The main thing is to avoid situations that put you at risk no matter how you present. There are areas of Denver even the people who live there don't want to be out in after dark.
It concerns me that some feel the need to carry a weapon. Time it takes to remove the weapon, take off the safety, aim (because you never shoot randomly) and fire is way longer than you have in most instances. The law enforcement people don't know you are the white hat when they arrive and you are just as good a target as the "perp".
This question comes up frequently and we become two camps, the aware and the defensive. I think aware, no matter how you are dressed in far better than relying on reflexes and hoping you don't shoot some poor by-stander. When I drive my eyes are constantly moving and I make mental notes of escape routes. I can usually tell you what color and style of vehicle I have been near the last 2 minutes and if possible who is in the car when the windows are not over darkened. In public I am a people watcher not because I fear them but because I am interested in how they present, so I pay attention more than most. On the other side of that coin I can't remember names for nothing. Long term memory isn't my best asset when it comes to people.
Fear breeds more fear. Just be aware and think about where you go.