Crossdressers are perceived as feminine men. Feminine men often elicit feelings of disgust and contempt from the belief that they cannot be protectors, are cowardly and morally weak combined with it being believed that to be feminine is to be weak because "women are the weaker, inferior sex"
For a man to embrace femininity he is rejecting his heritage which is a greater sin than for a woman who is born into it. Men "chose/choose to be effeminate" It is volitional. (This is actually a false believe IMHO because they are created by nature to be this way and probably it is natures way of reducing violence among men)
A weak man is one who is dominated by a woman. Henpecked, Cockhold, Submissive, Subserviant, ect... There is humiliation in these acts, where if a woman endures the same treatment there is sympathy.
Gender is to a large degree about double standards for both sexes beause they have served the interests (or it was at least believed so) of those who have embraced the double standards in the past.
I'm often amazed by how often crossdressers do not truly understand women. They emulate women without understanding their hearts, minds and needs.
A woman enters a relationship to gain that which she does not have and this is usually not found in an effeminate man, which can actually burden her or rob her of precious resources. Decent yes (masculine with a soft side) weak no and effeminacy implies weakness.
It is not just the clothes but more so what goes with the clothes that are problematic for women.
A woman may not feel she can count on a feminine man when things become difficult and than there is the sexual component.
Women do not object to the clothes "only" but everything they symbolize and imply. It can be an affront to her sexuality and to her sense of what makes up normal relations between intimates.(husband and wife)
Women do not care about crossdressers per se but being married to one.
Even many gay men have a problem with effeminate men and to embrace femininity is to be effeminate if you are a man, even if only temporarily while you are doing it.
It is the effeminacy (which is not trusted and viewed suspiciously) that is truly the problem, not the crossdressing, but the two go hand in hand.
There is not a patriarchal society anywhere that embraces effeminacy and most if not all actively suppress it and aspects of patriarchy very much serve the interests of women.