Very good question. But it also makes you wonder why people simply must know which gender a person of undeterminable gender is. Other than just the fact that people are determined to solve any mystery they encounter (ex: everyone wanted to know what KISS looked like under their makeup in the 70s, lol).
I think there are several reasons many unconsciously expect everyone in public to "match" their natal gender. Also, everyone (of each gender) is socialized to treat females one way and males another. Body language and other subtleties can change depending on which gender we are interacting with or even approaching.
This socialization happens from a very early age, although sometimes it is a subconscious conditioning.
So we have these preconceived ideas (even the most liberal and open-minded of people) about how we will interact with someone based upon their gender (and to a lesser extent, age).
Some of those preconceived ideas are valid!! For example: I would NEVER yank a woman's chain the way I do male friends'.
Most females simply don't take kindly nor feel comfortable with that sort of joking or teasing.
Anyway, the point is that human beings aren't terribly flexible when it comes to gender issues. Socially, it evolves pretty slowly.
AND, of course there is the issue of attraction. The majority of the population is not (naturally) bisexual or homosexual. So when most straight guys see "a skirt", there is the hope of the possibility of chasing said skirt.
Uhhh, CDing throws a bit of a monkey wrench into that um, "automatic reflex", causing skirt chaser to "look (and pause) before he leaps".