If you read Alison Bechdel's "Fun Home: Tragicomic," you'll see someone who is clearly a FtM crossdressser (she doesn't view herself as a transman but as an "unusual woman"). it kind of touches on it, though -- at one point, she imagines being in the Stonewall Riots and wondering whether she had the necessary "three pieces of women's clothing." It's not perceived the same way, even though at a young age she clearly wonders why she was not born male, and felt great distress about puberty.
You can spend a lot of time thinking about it, I suppose. Is it viewed as more frivolous due to the asymmetry of society's views with respect to gender? Butch women aren't typically seen as doing what they do for fun, and practicality isn't seen as frivolous. Crossdressers who go beyond fun might be considered somewhat something else at least in terms of how society tends to view crossdressers. I've heard that people who follow through with transition tend to be about equal in number. However, in any forums of this sort, there definitely is a bias towards MtF.
