Food is a lot about culture, habits and traditions. Becoming vegan makes all of the traditions, savoir-faire and handicrafts linked to meat, dairy products, cheese and fish pointless, irrelevant, outdated. Some non-vegan people may find it to be offensive, even insulting. Because it questions their habits, their culture and traditions linked to food. It questions their identity, it questions who they are.
Furthermore, being vegan makes it difficult to share a meal with you and everybody knows that social relationships are strongly linked to food acquisition, preparation and sharing. Some people may think that you're putting some distance between you and them and eventually they may reject you.
Two years ago I was the only vegan living in a room-share of five and it sometimes killed the entente in the house because I coud not share a barbecue or even a plate of spaghetti Bolognese with my roommates. The benefits the environment, the animals and I got from being vegan counter-balanced those petty social inconvenients, so I was fine with the situation.
Nice people will understand your diet and there's always common food that vegan and non-vegan people can share at the table.