Well, the short answer is that there is a double standard in play.
Women have become more androgenous as they began to enter the workforce in more non-traditional roles. To counter the stigma of women in the "male" work roles they started dressing more maculine. This competition with men became the basis for their masculine attire.
Another reason, as I see it, men have often thought of women wearing their shirts as sexy.
But it still boils down to a double standerd.
Since men have been the competitive model that women emulate it has become acceptable for women to cross to male. But, since we men are the emulation model there is no basis there for men crossing to female as we are not competing in a women's world and since so much is made about image as a contributor to success there is therefore a stigma about men reversing the roles by dressing feminie. Guys just aren't supposed to act feminine. It's all about macho. Even the Villiage People (flamboyantly gay) singing about being a "Macho man" or as I like to sing along say Nacho man. hehehehe
BTW I think it is a lot of malarchy. But I still shy away from presenting female in public. Ah well!

If ya think I am full of _____ you may be right. The above is just my humble opine.

Cheers