Well, this is not a good place to ask that question because we are crossdressers, not the women who left us because of our crossdressing.
I can only guess and my guess is that little girls dream of marrying a prince on a white horse and when they see their "prince" prancing around the house in a tutu and heels, it's a big letdown.
As crossdressers, it's hard to imagine this, but think how a normal man would feel if his wife cut her hair short, bound her breasts and wore a fake beard.