I am very hesitant to apply labels to behavior because a label sets a boundary as well and, sometimes, a source of motivation. We are all different in the details of this because we all have unique histories. Labels also tend to carry rather generalized specifications. I can think of a lot of labels that would fit me at different times, so I tend to steer clear of labels unless it is clearly applicable.
But, yes, I do understand the lack of desire under certain circumstances. My main thing about women's clothes, especially T-shirts, is not that it is a women's. A T-shirt is just a T-shirt. But I am very much into the feminine colors - bold and bright rather than often rather more subtle and not a lot of color variation available in men's. That seems to be changing a bit. And yes, many people associate certain colors with certain behaviors but that does not mean I am wearing pink for the reasons they might think, but on the other hand, sometimes I am. In other words appearances can lead to erroneous assumptions that may not fit at all. For example, some people will think a man in a kilt is wearing a skirt and therefore may be "one of those weird crossdressers." Maybe yes, maybe no.
I very rarely ever go all the way with my dressing anymore and for me I think a lot of it is age. Motivations change with age and often, but not always, become more and more casual about a lot of things.
So, I would say it is best to do that which makes you most comfortable for the moment or the situation you are in.