For me, it seems that I can always find some obligation to others that takes precedent over my wants. My greatest regrets are those occasions when I acted out of self-interest without concern for how it impacted those closest to me.
That being said, being retired and living alone are affording me much more opportunity to satisfy my own wants without infringing on anyone else.