I certainly would take it as a complement . I guess I would ask how well do you pass are you obviously a cross dresser and look very masculine when dressed? Maybe the guys are dawn to masculine looking women. Or maybe just accepting you for who you are and just making conversation. The other answer would be you were partly read and they were trying to confirm or as it has been said they are one of us.