I know the feeling, and it's not just when I'm in girl's clothes. I consider myself slightly bisexual. Every now and then I will be attracted to a guy. I've been intimate with a couple guys and it was pretty great. Maybe you're just curious. There's nothing wrong with it.
