I don't know how the reward pool works.

The reward pool is not infinite. If we did not have downvotes people on average would get a lot less in my opinion.

That is speculation obviously but there would be no way to stop self voting or multiple accounts so that activity would ramp up.