It will unnecessarily complicate things a lot, and ultimately it really doesn't matter that much, because regardless of how big voter is, idea is same: vote for all they like, don't vote for one you don't like, vote or not for return proposal.
True, big vote moves return proposal up a lot, but it also does same with other proposals voted for.