Tough one. It depends on how open you want to be with them vs. your own happiness. If you going out in public dressed is more important that what your neighbors think, then do it. If you think going out will cause more problems than it is worth than don't do it.
My only 'advice' I can give you though is would you expect them to change their behavior, or jobs, or presentation, etc to make you you more comfortable? Would you be uncomfortable if they worked menial jobs or dressed like farmers? Or would you accept who they are? Try not to live your life to make strangers happy or more comfortable. Why should you give up your freedom while others can enjoy theirs? Hope this helps.
Amber