I don't think we are underdogs in society. Many of us just think we are. I've said many times, we are too hard on ourselves. I think most of us, being heterosexual, do not want to be perceived as gay. And for that reason, we worry about being found out. Most people just don't care. "Hey, that might be a guy". "Yeah, could be". "How bout those gas prices"?
I've seen obvious crossdressers, and didn't think much about it, except some of them weren't even remotely passable. But no one ran over and assaulted them. No one paid any attention. It's just not the big deal we make it out to be.