For some reason, the early Castlevania games were where I drew the line of enjoyment versus frustration in a tough game.

I think that is totally reasonable. I remember there were certain things that were hidden that there is no way that someone could have figured it out. I don't recall exactly what it was but in order to reach a particular area you need to to break a certain block in a wall and then stand there crouched in order for some whirlwind to come and whisk you away to another area. This doesn't happen at any other point in the game so honestly, how the hell were we meant to figure that out?

The only game I recall as a child that I had "draw the line" was Ninja Gaiden. The knockback on the birds that would attack you on platforms over and over resulted in me giving up on the game altogether.