"It's one of them."
If one has the presence of mind and chutzpah, one could get a bit teary and say "It makes me feel so bad when you say things like that. I can't help it if I look a bit masculine..." Be confident enough in yourself and it will really mess with their minds!