I believe he threw in the towel when he realized the secret service detail was changed to make sure agents with no emotional ties to him were at his side with guns if he decided to overstay his visit in DC.