I don't think it's a matter of "if' you will be caught it's "when" you will be caught. And it doesn't have to be a wife. Children or parents living in the same house or having their own keys, neighbors with keys, etc.
It's also related to the extent of your dressing. Just throwing on a bra when your wife is at work or out of town is one thing, going all out with forms, makeup, and a wig while someone else is in the house or unaccounted for is something else.
So to answer the question, I got away with it for many years but I was just wearing a bra and water balloon forms and my wife's blouses from the laundry. She went out but came right back home because the tire light on the car came on. Caught me desperately trying to get out of the bra and blouse.
So anyway, now I am free to dress around the house and I have my own stuff. I am one of the lucky ones.