There is some truth in this. All sorts of things go on in men's washrooms that are unrelated to the primary reason such facilities exist, and sometimes those things are sexual in nature. I'm sure women do kinky things in public toilets too, sometimes, but honestly, the problem is men in general, not just CDers. And mixing genders in washrooms will inevitably cause some problems.
Part of my reasoning that unisex washrooms are ultimately a good thing is that it may ultimately contribute toward reversing the sex-shaming that is so prevalent in North American culture. Part of the reason men are so :edit: up about sexuality is, IMO, due to how they're socialized about gender and gender roles - in other words, that unhealthy cocktail of misogyny, toxic masculinity and rape culture. A significant part of that is the puritanical/Victorian attitude toward sex and bodily functions that makes them something to be treated furtively and with shame, and that discourages frank acknowledgement of and honest discussion about sex in the misguided belief that ignoring it will make it go away (tell that to all the teen mothers in abstinence-only states). We need to move toward a more European attitude toward sex. on consent and on the non-binary nature of gender will go some way toward changing this. Sharing bathrooms is one step in this process.
Ultimately, then, I see the 'bathroom wars' and the current battles against LGBTQ (and particularly trans) people as opportunities to change our entire culture, to get rid of old ideas and to encourage respect, tolerance and acceptance where gender and sexuality are concerned. If we can discard a lot of the 18th century baggage we have around those issues, I think we can reduce the hyper-inflated emphasis we put on sex and change how people relate to each other in that domain. This alone should reduce sexual violence. (Conservatives may call it 'social engineering, and that's exactly what it is - just as seatbelts and no smoking laws were.) It will take a few more generations, but we need to keep going in this direction.