No, we don't.
^this is the assumption, made by those who DO do it for sexual pleasure. In order to feel that they are normal, they project their own behavior and feeling onto everyone else, in order to feel that their feelings are the same as the rest of us.
Not true, either. While plenty of women will wear men's clothes, it's not to change what they want their sexuality to display. But a few women WILL wear men's clothing, in order to display that they are masculine. Though not in great numbers, they are out there, most often seen in masculine gay women.
The problem is, most crossdressers can't tell you why they do it. There's this huge social stigma against men
behaving, feeling or displaying femininity, that few are willing to explore the reasons why they do it. To them, it's just simpler to tell others that they were 'born that way'.
The definitions aren't changing. But perhaps slowly, over time, more males will be willing to examine why they want to embrace femininity. But it's not going to happen any time soon. We still have a whole lot of men who can't accept that they have feminine feelings, so much so, that they can't even accept it as part off themselves. Look at all of those who have to go to the extreme off creating a fake feminine persona, to assign all those feminine feelings to, to the point where they need to refer to themselves in third person to avoid any connection between themselves and those female feelings.