A very succinct statement of the issue. I've written at length in a similar thread that we might view the physical brain (flesh and blood) as analogous to the hardware of a computer, and the consciousness with which we live and learn as analogous to applications software, and the primary neural "wiring" of the brain cells in various areas of the physical brain (responsible for, e.g., emotions) as a kind of firmware, instructions that facilitate consciousness without our being aware of their actions and operations.
At my advanced age, having dealt with this since early childhood and gone so far as to live through an eight-year full-time experiment, I am absolutely convinced that there is something different in the brains of crossdressers, homosexuals, and their fellow variants that make us this way. If it was in the software developed by nurture, it wouldn't be so stubbornly resistant to change, and looking at gross brain structure doesn't show us the details of the neural roadmap. The rainbow of different manifestations? There are millions of neural connections, and, accordingly, many millions of ways they might have been connected abnormally during gestation and early development.
Absolutely, something in our brains tells us to do this. That some of us stare it down early and others deny and delay for decades is quite another matter.