As a man who has never "fully" dressed (i.e., the whole nine yards, and basically a MIAD), this is a problem for me in public. I've not ventured out in public very much at all, and only had to really contend with this problem once when I could not find a single person bathroom. I chose the men's room, but now feel that was a serious mistake. I was lucky. No guy came in while I was in there. I've never used a women's restroom, and I'll be nervous as hell the first time I try. I understand your conundrum.