No rules against it. It is simply social expectations and the resulting personal choices that stop many of us. I have no fear of being arrested for wearing what are seen as female clothes. I have a massive fear of ridicule and rejection amongst certain important groups, that I need to be excepted by to live the life I
very much enjoy as it is.