It comes down to how one identifies. A good example is very butch lesbians--the kind you tend to mistake for men by appearances alone. They dress in men's clothes and pass as men (and sometimes even prefer male pronouns), yet are neither cross-dressers nor transmen because they don't identify as such. Rather, they still identify as women, hence the "lesbian" part.
A bit like how transwomen (or transmen) aren't cross-dressers because of how they identify and what that identification constitutes.![]()