I have a theory about this. Keep in mind that its just a theory. When I was a young child, I was told a story that when I was a baby, that a friend of the family mistakenly thought that I was a girl. He said something like, "Oh, what a beautiful little girl", or something like that. Obviously, he was corrected by my parents. However, I believe I heard that even though I was just an infant. Years later when I was told (and sometimes teased) about it, I think a "connection" was made somewhere in my brain. I know I started dressing when I was young, sometime around four or five. I remember running up the stairs to hide when someone would come over and I was playing in my mother's heels. Anyways, I think the "connection" was established/re-established when I was told about the event (events?) and teased about it when I was four or five, right around the same time I started playing with my mother's shoes and, on the weekend at my father's house, my older sister's bikinis, nightgowns, and other clothes that she would leave hanging on the hook of the bathroom door.
I think it would be neat if others on this forum could/would ask their parents if they were ever mistakenly identified by someone OUT LOUD as a girl when they were an infant.