Everyone has a different relationship and yours is obviously different from mine. That said, if my husband came to me and said that he wanted to have sex with a man (or another woman) because he needs the "freedom" to do it, I would give him freedom - in the form of a divorce. It is selfish for anyone to marry and then say they want freedom to do it with other people - unless that was part of the original arrangement when you married.
I don't really care if you are a CDer, TS, or any other initials that you feel you are entitled to use, wearing women's clothes does not make it ok to run around and cheat. Give your wife her freedom first.
Hopefully, things work out for you. Maybe your wife will also find someone else to have "freedom" with.