Do you think the reason they have no sex with their wives is perhaps a lack of interest in women, and had they come to terms with their sexuality earlier, they would not have married GGs?
Also, I understand you are not their keeper and there is no way you can determine whether or not you are making their relationships worse, unless you actually have a chat with their wives. These wives need to know the truths about their husbands so that they can just bite the bullet and move on from their relationships and perhaps find someone who is into them. But, have you ever considered going out with men who have taken the plunge after realizing they are not into their wives and have actually divorced them? Of course, if all these people are into open relationships, then my comments do not apply.
As with my comments above, if she is sleeping around too and they have an open relationship, then all bets are off, but what makes things worse in your situation is the fact that you KNOW the gf would be upset if she found out. Where are your principles? Feeling "guilty", yet going ahead anyway does not absolve you from any wrong-doing.
To both of you: even men in committed homosexual relationships have some sense of propriety and loyalty to their partners, unless, again, they are in open relationships.