Imagine there was a society in which blue eyed people are referred to with blee/bler pronouns, and green eyed people are referred to with glee/gler pronouns, and one day someone from that society saw a brown eyed person and had no idea whether to categorise them as blee/bler or glee/gler
Having heterochromia would really suck, people would probably be such dicks about it.
In a similar vein, imagine if we were like cats, and there were skin colors/patterns that only women could get.
Or albinism