It doesn't matter who you drink coffee with as long as it makes you happy. Even with your condition, your therapist is right, no one should impose anything on you. If you don't feel comfortable doing it, then, don't. You are loved and cherished.. never forget that.