I guess I am the exception to the rule.
I never, to my blotchy memory, went through the arousal phase. I first wanted to just see what it felt like to wear female clothes. I liked it. then I was sneaking an occasional wearing of while my sister's clothes while she was still at school. I did not have a real female figure in my life because mom and dad had divorced and I and my sister lived with dad (when he was home, he was a SAC refueler pilot).