Familiarity can be powerful like that. You already know who you were with them, while becoming someone new requires you to figure yourself out again.