If I were in that situation, with the young children in the women's room... I would have used the men's room too. Hell has no fury like a mother who thinks she is protecting her children...whether or not that fear is actually justified, they may believe it regardless and act accordingly.

Others do as they wish and that is fine. I, personally, would never chance it with little girls in the women's room.