I admit to really hating the MIAD look but I will staunchly defend your right to dress that way to anyone. That being said, are you attracted to dresses and other typically female garments because of the actual way they look or because they are marketed towards women? If sartorial history was flipped and dresses were all along considered to be a male garment would you still want to wear one?
If I put myself on the spot, I would have to admit that it's probably the fact that certain items are supposed to be for women and not men is what makes them appealing. Things like "manties" are a complete turn off for me: http://www.manties.net/
When I found out that certain Bulgari fragrances were actually considered unisex and not just women's I did not like them as much (even though I still wear them on occasion). None of this is rational but I guess my brain is just wired that way. If three piece suits and ties were considered only women's wear I would probably be a lot more interested in them. As it is now, I kind of detest suits (the YSL Le Smoking being an exception) and I most definitely HATE ties.
I used to hate the color pink not because of gender connotations but because I thought it was kind of juvenile looking. The wide spread appearance and marketing of "millennial pink" has definitely worked on me and I now have several accessories and clothing in that pale pinkish color. I still find it fascinating that pink was originally for baby boys because it was considered a strong color and blue was for girls because it was seen as softer.
The whole idea of gender is really weird but I am definitely influenced by it.