I've read several reports over the years that well over half of the bras that women buy are the wrong size. It's not hard to imagine then that they would feel discomfort just from the fit problems. Add to that the chaffing and moisture in summer and it's likely that most women would really rather avoid them altogether.
Seeing the commentary in this thread made me recall similar reactions to wearing nylons. Seems like you either love or hate certain types of clothes.
For myself, I have quite a few very comfortable bras that I wear with concave-back forms that fit my natural breast. Once they have warmed up they feel very "natural", or close as I imagine.