Hmm, a mix bag there... :-\

So, your ex is your ex why?
I don't really want to know but, it is something to think about before you go on.

My understanding is that one sided open relationships like you are talking about are very hard to do. Where your wife is allowed out but, not you. All open, all closed, or controlled open to a single or small set of people seem to be the things that sometimes work... the one sided open appears to end bad most of the time.

So, your wife has a medical condition. ... Does she still see her girl friend? Still the half open relationship?

This is really a hard one to deal with and you probably should find professional help if you want to stay with your wife. Find someone who has experience with this sort of relationship... On the flip side... what part of you or in your past has you drawn to these people. I didn't understand it for a long time... about myself... that I was attracted to my wife for a fairly clear set of reasons... other than the physical. Knowing yourself in all of this would be a very good thing. I wish I could help... My only thoughts are...

learn about yourself and why you are here
remember why you aren't with your ex
think about the open vs closed

If your going mad and you may make a bad choice with a stranger which could be much worst than with your ex depending... TALK WITH YOUR WIFE.
If you have problems talking / listening look up "imago dialogue" on the net and get a book or a vid and PRACTICE TALKING WITH YOUR WIFE!