You are not alone! In my early years dressing was totally for sexual reasons, and remained that way for many years. In my case change has not happened quickly, but has morphed over a long period of time. Over the last 10 years it has jumped back and forth a lot. It is now at about an 80% to 20% split in favour of the clothes only. I have always thought that some of this change has to do with the 'change of life' which men also go through. There is a lovely sensual feel to my clothing choices and to me this is what makes it all feel so peaceful.
Pat's comment,"The sexual aspect of dressing, though common at the beginning, has been identified by psychologists as "an extinction behavior" meaning that the behavior disappears over time but the crossdressing remains." This is an interesting thought! To me, I believe we all have varying degrees of masculine and feminine traits, if someone is truly transgender, their feminine traits should manifest in many areas, not just in their clothes.