You raise an interesting subject. Oddly women wear drab a lot, and though I don't find it attractive at all I don't think I should comment.
I sometimes wonder how people commenting about crossdressing is different to them commenting about people wearing clothes that just don't suit them at all. Why is it OK to comment about the former but not socially acceptable to comment about the latter because it might upset them.