If you can get your brother to listen for a moment remind him that he has put your health in danger based on a limited understanding of what he sees. Did he think in that one moment that all the years of love and support in the family could evaporate? Let him know that he is wrong in his assumption and if anything bad happens to you it is his doing. Don't argue with him, just lay out the facts.
Then you need to look for alternative living arrangements if possible. I would be very worried to live in an active gang neighborhood where the members of that gang didn't like me.