Point taken - and it's great to meet people like that. I find that often, even when I gently point out "hmmm, you know, that joke, was really problematical in some ways, although I'm sure you didn't mean it to be", that people tend to dig in their heels and become defensive about it. But you are absolutely right that sometimes they don't realize they are using bigoted language, because such things are awfully ingrained in us socially. One of my friends called out a local political party chairman on a transphobic joke he made in a public speech. She did it in a letter, privately, and very politely I might add. He wrote her back after a time, and admitted he had to do some soul searching, and apologized for the remark.
It really is best to point out to people, as politely as possible, as I think hostility accomplishes little, that jokes like that aren't OK.