As many said before, you're not alone. I also feel ashamed sometimes. We do something that for most people is not "normal" even bad for some.
When I felt ashamed or guilty, I always remind to myself:
1) I am not hurting anyone,
2) Crossdressing is just too pleasant to quit.
Keep in mind that life is short. Enjoy it.