Unfortunately, I have done the same thing. If he is anything like me, he appreciates your support and my guess is he will become more comfortable with it. Maybe come up with a plan before you head out. Code words maybe?