One thing said to me a while back puts a lot of things in perspective. You think of yourself way more than others do. In customer service you see and hear a lot of things and a passing interaction like that is likely to not be memorable.
The other theory I go by is another persons opinion of me is none of my business.