Yes.. And this could be an issue.. But since you never know if it's a drill or not, you have to do what's necessary to save your skin, and deal with the possible consequences afterward.
If I was that concerned with what my neighbors are thinking, I would maybe lie and say this girl I just met is into some freaky stuff, so she put make up on me but forgot to bring something, so she went home to get it, then the alarm went off.