If we do a good enough job, there's no issue as we won't be recognized.
For most of us, we are detectable only under close examination. Most people do not examine us closely enough to see anything but a woman.
Someone who sees me closely may detect something out of the ordinary. When fully dressed I could be a CDer, or I might just be a tall 50something woman who inherited some unfortunate genes. Politeness demands that those I interact with acknowledge me as I present and I conform to societal norms to make this easier for them.
Most CDers, while not adhering to society's norm concerning presenting as their birth gender, do adhere closely to society's norm for the gender binary while dressed. For whatever reason, we want to be acknowledged as female and we won't do anything that might preclude that.
Now, I'm not so careful when in male mode. I won't wear a skirt, but I will wear clothes and colors that are androgynous. Being misgendered in male mode doesn't bother me at all. My feeling is that I am not perceived as a CDer in this mode, simply a male with somewhat effeminate tastes.
The one bugaboo seems to be the matter of hair, both facial and body. Even GGs have a strong societal pressure not to have excessive facial or body hair. My electrologist makes the bulk of her income from this. Facial hair is probably the strongest gender cue, to the point where a "bearded lady" is a stereotypical circus freak. The combination of an undeniably male cue like a beard and undeniable female cue like a skirt and hose seems to be culturally jarring both to ordinary members of society and to many of us. Unfortunately, this is the only kind of CDer that many members of society have ever perceived, since the ones who blend go unnoticed.
This leads to a considerable problem. I desire so much to be accepted as a woman when dressed that there is no way that I would want to acknowledge that I am TG. If I am doing a good job in this I'm obviously not a good ambassador for the TG community.
I don't really have a solution to this, as being "Loud and Proud" about being TG is not compatible with presenting as a normal female.