No, I've never been approached and asked a question like that.
People have a lot of nerve if they are either in a crowd or secure in their car, but on a face-to-face basis they will mutter to themselves, but rarely confront you.
The exception to this of course is kids. They have no qualms about it at all and young girls seem to have the most Chutzbah.