Actually, I think it's possibly harder having a husband who does this for non-gender/sexual reasons. OUR identity can get messed up with it then as I know I started to wonder if I was the lesbian! What would be more convenient for me is there IS a similarity between you and my H as I might actually understand then. Then he might be the more feminine, nice guy he said he was before I realised the truth!
And seriously, what about those men who dress as babies? Don't you find a curious similarity between their experiences (started in childhood, they don't remember why, just that they were drawn to it, it's not sexual but it is relaxing and they HAVE to do it as it's part of their identity and it progresses and increases over time) I find them very similar. In which case, some crossdressing might be more related to the other identity differences men have than TS. I say SOME crossdressing as I think there's obviously the gender kind that is related to TS and is a different condition entirely.
I do think people can crossdress for completely different reasons, though. And maybe no two men have the same reason? In which case, this discussion is completely pointless!
And now I've peaked my own interest and want to know more about Adult Babies, lol