It doesn't even have to be traumatic. It only needs a random incident that creates a strong impression and this can be based on a myriad factors. Say there is a little boy who has a huge crush on a teacher who wears nylons and heels every day. Everything about her is fascinating and one day his hand brushes against her leg by mistake, he is amazed at the silky feel and struck with the intimacy of the incident. It becomes part of his arousal template.
Here's what Ian Kerner, PhD, LMFT (licensed psychotherapist and sexuality counselor) has to say about fetishes/kinks. He describes, near the bottom, how they may be imprinted. He also mentions the possibility of seeing something on TV that is locked in the mind and eroticized at a later time:
http://thechart.blogs.cnn.com/2011/0...-and-fetishes/
If you ask people who begin the CDing in later years, I'm guessing they will say they did have a fascination with women's clothes or a particular item of women's clothing for a long time even if it wasn't acted upon.
Most people who develop fetishes are men. This explains why there aren't many female crossdressers.