You should be packing. That way you won't feel so afraid of people like that. The worst part is that he probably meant no harm, but never the less it did bother you. Glad is was a non-event in the end.