A lot depends on how a person perceives himself. If he considers himself not smart enough, handsome, pleasant or competent, those around him "feel" - see his insecurity in every look, word, movement, and refer to him accordingly. An uncertain, soft person, as it were, says all his appearance to others: "I can not be reckoned with, I will endure everything." And even if there is no "natural" coarseness in the nature of others, they act with an unsure person just as he himself "advises". Here the rule is: "what you ask for is what they give". No more, no less. If a person does not pretend to respect, he will not receive it.
It is interesting that people who do not suffer from insecurity, but are strongly turned inward - vivid introverts, are usually perceived by everyone else as simply very inconspicuous people. No one comes to mind somehow infringe on their rights, humiliate or exploit. Just their appearance and behavior, they signal: "I'm not here, I'm far away, do not bother me."
That is, how people relate to him, depends on the person himself.