Yes, you can. Just because someone puts on a dress once (for Hallowe'en, or on a dare, or to sneak into a college bathroom and snap pictures of women, or for a bank robbery) doesn't make them a crossdresser except in the most technical sense of the word, in which case you also have to consider any women who has ever worn a pair of pants or a man's shirt to be a crossdresser. In other words, you can't assume that, just because someone puts on a dress, that they are a CD or trans person. A person who dons a disguise (something intended to conceal their identity) to commit a crime is not the same as someone who dresses to acknowledge and express their identity and is not committing a crime. No law, much less one with minimal or non-existent penalties, will prevent people from donning disguises and entering the women's washroom to commit offences (that are already illegal!) if they really want to do so.
No Unequal Rights, Breitbart, and other organizations have published several lists of 'men who entered women's facilities' in order to bolster their discriminatory agendas. If you read through them with a critical eye you will find very few examples of people who are 'real' crossdressers or who identify as transgender. In the vast majority of cases they are cis men who have put on dresses as disguises, rather than as expressions of identity.
The NUER list contains 48 reports of crimes committed in five countries between 1991 - 2014 by 'men in women's clothing' (hereafter MIWC). Of those, there were only 7 cases in the US of men dressing as women &/or claiming to be trans in order to enter women's facilities to commit some sort of sexual offence (the majority of which were of a voyeuristic nature, e.g. peeping, taking pictures), or (on average) roughly one case every three years throughout the entire country. Even if you take all of their reports at face value, there are at most 25 cases of sexual offences by a MIWC in the US in those 24 years in any location (not just women's facilities, and therefore irrelevant from the perspective of these bills), or one per year on average. Remember, this is from a list compiled by an organization that has a vested interest in finding and publicizing as many such cases as it possibly can. And again, a significant number of the 'MIWC' in these reports are probably cis men in disguises.
If you are serious about 'protecting women and children', consider the 127767 reports of forcible (i.e. penetrative) rape that were reported to the FBI in 2012/2013 alone (realizing also that ~90% of sexual assaults go unreported). Oh, and don't forget the 31 trans women murdered in 2012/2013 simply for being who they are. 'Bathroom bills' are a joke, a solution looking for a problem that doesn't exist. Focusing on them is a way for lawmakers to curry favour with bigots while continuing to do nothing whatsoever about real issues and that should be recognized for the baseless and hateful attempts at pandering that they are.