I kind of have a different take on this issue. For me I see it not as an attraction to the Cross-dresser but to Cross-dressing. In an algebra problem to solve for X you have to eliminate all other variables. If you take the man out of the dress he is no longer attractive. What was attractive was the act of dressing. To a CDer seeing cross-dressing done well can stir highly charged emotions such as envy or aspiration. From my own experience I believe cross-dressing is rooted in ones sex-drive whether or not the individual CDer does anything fetishistic or feels fetishistic about their cross-dressing. The act or just the thought of a man appearing as a woman in itself alone is the source of attraction.
Have you ever come across TG captioned pictures or photos? Pictures of very attractive GG models or the millions of "selfies" that flood the internet posted along side a short two paragraph story about how the picture is of some guy transformed into a woman. The picture of the woman alone is very attractive but add the story with the TG element and it gains an extra level of fascination to the CDer's mind. A neat test to have would be to have a man and a woman with an identical attractive appearance. Have them dressed in the same clothes and made up in the same style make-up and see who would rate more among CDers. I am sure the woman word attract them all but the man would earn extra simply because the act of cross-dressing strikes a certain unknown cord in us.
Just a thought.
