Where I live, I can go out fully (or even partly) dressed with no problems, and I do,though not as frequently as I used to just because I have other things to do. It is becoming increasingly difficult to justify the additional time to look good in public - I do not wear make up at home, or a wig. There is no fear in going out and I love interacting with women when I do - and most of them know I am a man and are relaxed about it - but few of them know the man as a man.
I work in a sector where I know with certainty that I would be OK at work wearing womenswear. I do not feel I am a woman and acting like one with colleagues would just be weird so it would only ever be LOOKING like a woman.
So if I can do all that, why don't I? I can see no benefit. I get as much from cross-dressing as I need or even want without this. I also like women viewing me as masculine. I prefer the company of women and enjoy the mild flirting and that usually stops when you put on a dress. But the main reason is that I have a certain status because of what I contribute to society and that is what I want to be remembered for, not to be remembered as the guy in the dress.
It is not fear. It is preference.