If the point in question is *splitting* a constituency, it has to be wider than men using women restrooms. Presumably it's everyone under the umbrella, or perhaps on the spectrum (it wasn't specified). It's a perfect example of deriving an identity (the constituency) from a theoretical construct (the spectrum or umbrella). It also illustrates my conclusion ... that someone loses. In this case, it's transsexuals.
Back to the spectrum itself. What, exactly, is the subject of the spectrum? Gender feelings? Transition need? Homosexual leanings (in the original sexual inversion sense ... which is the great progenitor of the spectrum notion behind the gender spectrum)? Disorder? Dressing urgency? Age? Passability? Femininity/Masculinity? Behavior? Attitudes? Stereotype conformity?
Oh, you say, gender IDENTITY! As if THAT'S crystal clear ... (not) Let's start with the idea of gender as opposed to sex. It's an academic invention. That it is constantly conflated and confused with one's inner sense of SEX doesn't help. The transsexual dilemma is the conflict between their actual sex and what they are biologically geared to expect. In a MtF context, a male expects to see and experience a female body. It's a body mapping issue. That's it. It may manifest as conflict and dysphoria ... or not. It may manifest as pschological symptoms masking or substituting for the actual issue ... or not. It makes some suicidal ... and some not. But they all share the same issue. And they may experience it in terms of gender in the sociological sense ... or not.
In short, the spectrum idea doesn't help. It obfuscates complex phenomena via overgeneralization, to put it mildly. Beyond the overgeneralization, though, is the biggest swamp of all - that the spectrum is always ultimately plotted on a male to female axis. You can't reduce individual identity to such a scale, even if the scale's plot has some population-level validity. To do so is to make the most common mistake in practical statistics. Even if you use proper plots - and they are rarely linear anyway - if an individual falls so far outside its bounds that it's unmeasurable, it STILL doesn't force the conclusion (i.e., in this case that one is or isn't a woman or a man or something else). It's rubbish at the individual level. It's usability lies elsewhere, which is another topic. BTW, the same comments apply to multi-dimensional approaches.