My own two cents.
It's all to do with ourselves - being and feeling more self-conscious, taking such a brave step outside into the big wide world. We are the centre of our own world, and when we take such brave steps, it's perfectly natural to think that others will regard our steps outside with just as much significance and importance that we ourselves ascribe to it. The truth is - other people don't ascribe the same level of importance. Other people ascribe no degree of importance. Other people are just going about their own business like they would any other day.
So a stranger clocks you? What happens? They may giggle, they may double take, it may even become a story that they tell their friends or partners later that same day. But they won't ascribe any importance to it, simply because it's not important.
I've experienced a wide range of emotions when I've gone out. From being fully dressed and presenting as a "woman", to mixing and matching, and wearing feminine attire while not hiding the fact that I'm a man. I've felt my heart pound against my chest, scared witless about encountering other people out in public - to other times not caring whatsoever and just going about my day. The reactions from the public on either occasion? Exactly the same. From either not noticing, to not caring, to noticing and not caring, to noticing and having a momentary giggle, maybe even a double take, but then continuing about their own business.
The point I'm making is that most of the fears, the excitement, the adventure, reside only in our own heads. No one else really cares. Certainly no one as much as we do, put it that way.
So just be yourself, go about your business. Wear what you want. And stop worrying about what other people may or may not think.