It seems that the difference is whether you own it, or are embarrassed by it. That's why I call my purse a purse, not a masculine euphemism. If I were embarrassed and tried to hide it that would make me a target.

If Shakespeare's Petruccio, a man's man if there ever was one, can say "Crowns in my purse I have..." then I can carry a purse too!