This gets into trying to define other people's sexual identity, very hard to do. Are they straight? Technically no, probably bisexual is more accurate, but not quite 100%. Many that I've met were attracted to females as well as transgendered males. They're attracted to femininity, regardless of whether the person they like is male or female.
And many are mainly looking for sex, probably most, but should we label them as pervs any more than the guys who are out looking strictly for sex with females? They're annoying if you don't want their attention and they're bugging you, no question about it, I've experienced that plenty of times.
But remember that there are plenty of people who consider us pervs, so let's not go there.
Carol