Well I will not repeat what others have said.
But it is complicated .
The reasons stated are good but do not forget the The average person never is in contact with a crossdresser so it seems /diffent, out of the norm ,maybe scary because they are not informed, involved.
Remember some that are secret, closeted crossdressers does not help this situation ....this will take a long time to change people perspective. I wish more were open, proud embrace themselves..
Yes things are changing but it being a secret hidden does not help things IMHO.
( not saying you are closeted I do not know) but stating this does not help as many are.
YES you are of course allowed, probably only in a few countries is it Not.
Wear what you want .