If this is a worry then why have you gone out in public dressed as a woman at all? If you are going to be that way in public do not lie about it. If asked "who is that woman I saw" tell them it is me! You cannot hide it and be out at the same time.
If they threaten to oust you ask for the requirements that specifically say no transgender people are allowed.