I don't think that people in general mistrust the CDing any more than strange foods, foreign places, or any unknown situation you might think of that people are unwilling to engage in. A lot of people stay away from strange foods & places even when they know that millions of others eat the foods and live there. I just got back from a visit to North Africa. Some friends were asking me why on earth would I want to go there, just as they might wonder why on earth I'm OK with being in a relationship with a man who explores femininity. But, there is an added element of shame involved in not being true to our anatomical reality, almost as if it's a betrayal (see the next point).
Yes, I did mean sexual orientation because this is a fundamental building block of who we are much more than just pastimes or hobbies. Our sexual orientation is what propagates our species!Also, the distaste for crossdressing (felt by some and not all people) has to do with a perceived deviation from the very gender norms that has propagated our species, which in turn is perceived by some people as a deliberate rejection of it. And the distaste for this is somehow stronger in men than women. Boys are socialized to believe there is nothing worse in life than to be perceived as a sissy by their peers. Women on the other hand are not socialized to fear appearing strong and competitive, in fact it's the opposite among women ... we're darn happy to escape the confines of subjugated gender that has plagued us for much of human history. Still, people generally seek to belong to their cultural groups and this translates to an aversion of anything that might be perceived as "not right" by our peers. And this applies to women married to men who deviate from the gender norms too. This is why so many CDers and/or their wives do not wish to be out to all their family, friends, coworkers and everyone they know in their day-to-day lives.
All of this is changing of course in our society. Many people are more open to things like same-sex marriage and gender variance than they were a generation ago, but still we as a society haven't reached 100% acceptance and I don't know that we ever will, simply because there will only ever be a small percentage of the population that is same-sex attracted and/or gender variant. We're more prone to universally accept changes that affect most of us rather than just a few of us. And don't forget that it is also much easier to be socially liberal when it comes to situations affecting other people, than when it hits close to home.
I did say the examples I gave were loose because there are so many variables in what makes relationships equitable. For example, does your wife earn the same income as you. Did she know about the CDing at the beginning of the relationship or did she find out after years of marriage and does she feel betrayed. Are the two of you sexually connected, or might she perceive that your sexual interests are diffused. Still, maybe for your wife it has nothing to do with trust, a good balance between the CDing and the attention you give her, or a good sex life, and it is more about going outside of the norms as mentioned above.
Has she told you specifically all the things that bother her about the CDing? If you knew, then maybe the two of you could tackle them one by one and make compromises. Your wife may never take to the CDing and happily participate, but hopefully she will come to recognize that YOU need to do this and she will be willing to provide you with the time and space to do so.