This is my opinion only:
When the majority of women wear pants/jeans, they buy them in the women's department, they are not trying to fool others into thinking they are women. They don't try to disguise their voices, use fake names, use fake beards, etc. The norm in society now is: women wear pants. Over 50 years ago, it was not the norm. It took a while and a lot of women wearing pants for it to become commonplace.
This opinion is for CDer's only, does not pertain to trans. The problem that I see with men is that at the current time, there are no skirts or dresses specifically made for men that are sold in the men's department. Yes, there are fashion models of men wearing skirts but I haven't seen them in stores yet. So, if men did buy skirts and did not try to "pass as women" but as a man wearing a skirt, if lots of men flocked to the stores to buy skirts (as women did in the 50's & 60's to buy pants), and if men were comfortable being a man in a skirt, not trying to fool anyone with fake names, fake hair, fake boobs, and all that stuff, maybe CDers would not feel so out of place.
My question to CDers only is that if men in skirts and dresses were commonplace, would it be as much fun for you as it is now? If you didn't try to fool anyone into "acceptance", would it be as thrilling? Or would you just be happy that you don't have to hide (if you do hide)?