Both TS and CDers have lots of challenges. Some challenges are common to both, and each one has challenges that are different from the other, but we both have lots of hardships we face. Here are some of the challenges of being a CD.
1. Finding time to cross-dress, where you won't have to interact with the world
2. Work - unless you work from home, you can't CD at work. And if you're lucky to work from home, that will likely change and you will be forced to commute to an office and sit through rush hour traffic.
3. Wife and kids - most wives aren't accepting of CDers, and many fear the CD will eventually transition, even though that is highly unlikely, but you don't know who is gonna transition and who is not. You may have to sneak dressing time when no one is present. That can be very tricky to pull off without getting caught, and to even find time alone when you're married and have kids.
4. Family - parents, siblings, cousins - even if you're single or have an accepting SO, chances are your parents or siblings won't accept you and you're still struggling to keep it a secret from them. Try living near by and having to visit them in drab, be in drab if they visit you, give up a day of CDing to tend to family obligations, have a family member inadvertently discover your secret stash of clothes. And heaven help you if you live with family. Good luck finding time to CD.
5. If you're an out CD, you face all the risks a TS woman would face when out en femme. The only difference is you can revert to drab mode and cis-male privilege.
6. Fear of being caught by family, friends, SO, kids, work, neighbors.
7. Getting irritated if you have to go a long stretch without CDing. And what constitutes a long stretch? 1 month? 2 weeks? 1 week? 3 days? And if god forbid you're forced to go 2 or more years without CDing. Lucky you don't end up in the loony bin.
8. If you come out to others, they may think you're crazy, messed up in the head, insane, gender confused, a sinner, going to hell, and people may be afraid to hang around you because they'd be guilty by association. People may literally treat you as if you're on drugs. People might even think of you as a drug addict.
9. Finding women's clothes that fit a plus-size male body.
10. Shopping for women's clothes in drab, getting mistreated by an SA, or being caught shopping by someone you know. I've never had trouble with an SA in L.A., but I certainly fear getting caught by someone I know. And I have had trouble with SAs in less tolerant places like Virginia.
11. It's very expensive to maintain two wardrobes.
12. Double duty laundry.
13. Hassles of hair removal or beard covering. Some CDers have such stubborn beard growth that waxing or electro or laser is a must. Even those of us with average or below average beard growth, some of us opt for painful and expensive waxing, electro, and laser.
14. Self-acceptance. CDs and TSs both struggle with accepting who we are. This can be the hardest part of all, accepting being a cross-dresser.
15. Although unlikely, what if you really are TS and do need to transition?
16. What if you really do hate being a man?
17. Loneliness.
Edit: People actually think cross-dressing is selfish.