Jesus made it clear that it's perfectly permissible to divorce because of adultery. In the Old Testament an adulterer would be stoned to death and you'd be free to marry. Paul made it clear that we're not to divorce our unbelieving spouses. That we should stay married to them if they want to be married to us. But if they leave we are no longer bound to them.