I would be willing to bet that it is part Hollywood politics and part the rumor that James is quite tough to work with. I don't know this for certain but he seems like one of those artsy types or character actors who are just a menace on the set. I remember reading about Jim Carey being absolutely awful to work with as well because he thinks he knows better than the director or anyone else how to do a character. James is rumored to be the same. This might be why he doesn't get many awesome scripts because people are just like "screw that guy, he's too much of a pain in the ass to work with."