It sounds to me like the guy was checking you out and his partner didn't like the competition. Keep going out to the diners, the hell with anyone that doesn't like it.