I think it is the clash between male and female that upsets folks. They really want to see one side of the binary or the other.
I'm sure I ran into this a short petite guy. I wasn't masculine enough to be on the guy side of the binary. This made it tough to socialize.
But, having an hourglass figure, painted nails, and long hair makes it super easy to be on the feminine side.
Because everything "matches" everyone at work seems OK, even those one might guess may have issues.
Marion