The boundaries thing seems a bit harsh. Bra and panties only? Heck, that's only the foundation for me, and I'd not be happy with it. However, I keep it away from my wife, and I think we both like it better that way. However, remember the historical story of Pyrrus, the great Persian general, who won his battle but did so at the expense of his entire force. From this we get the term "Pyrric victory". Were to to chuck you marriage and your family just so you could have more freedom to dress as a woman, you'd likely find over time that it would be a very empty and hollow victory.
Find your foundation for your love, build upon that, and then find your options for dressing. Otherwise, you're only moving towards misery.
