Steph, you are perfectly correct. GID shows at very early ages well before puberty.
But where you are at is very common, particularly for older (by which I mean 35-40+) TS's.
Look in once sense anything is manageable it just comes down to the compromises that you want to make and accepting that there will be pain & loss, but also gain whatever choice you make.
In the end (and only you and your wife can make this decision, perhaps with some professional help) you have to decide whether your relationship (and all the associated family, life, job, etc) is more important.
In which case it is about managing your GID. It will never go away and will always cause you angst. But on the other hand you get the pleasure and joy of you relationship (etc).
Now how to manage it is another thing, but a lot of people do (personally I think there are a lot of of people out there with GID of varying intensities).
Note that there is no cure, just management and it could take some time to achieve a reasonable balance .. and it might also be said you might fail (as I did for example).
On the other hand you chose 'the red pill' and that will fix your GID, but you will suffer pain and loss and regret, some of which may go on for a long time, even all your life. On the other hand you will have the joy and contentment of being yourself.
There are no easy answers to this and it is so personal and involves so many other people and factors that no one but you can choose. You will win and lose whatever the choice you make.