It’s Just Sex or Leave Him

If you’re with someone who cheated on you twice w a hooker when he was out of town at a conference and again at an out of state training… is it just sex since it’s with a person to be in contact w or is this trouble