I'm probably the same way, but I think a lot of it has to do with my ADHD. It's pretty rare that I will look anyone in the eye when I am talking to them. In fact, most of the time I won't even look in their general direction. It just feels unnatural to me. I know it's the proper thing to do, but I just haven't been able to condition myself to do it. Throw some degraded brain function on top and I don't think I would be able to handle it. It sounds like you have found a great way to cope, whether people appreciate that or not.
RE: He's So Shifty