I don't think putting penalty on the use of downvote is okay because if that is done that means the purpose of creating such option in the platform is neglected.

Which is also the same thing as removing the downvote button 🔘.
Which will make me ask this "if penalty is placed on the use of downvote, what will be the measures or punishment that will be used against anybody who defaults in the platform?