Engine Light Node and Snapshot Update

In my last post over here I have proposed Hive-Engine light nodes. You are finding more information in the old post if you are wondering what that is.

In this post I will be giving an update on how light nodes are coming along and a few other small changes.

Light nodes

Since my last post, the light node PR has been approved by the engine team and was merged into the main repository. By the way the engine github has switched to a new repository, which you can find here https://github.com/hive-engine/hivesmartcontracts.

There was a bit of a discussion at the start about light nodes, but they are coming along quite nicely now. We have fixed some minor issues and adapted a few things including the default blocksToKeep. It was rasied to 864000 blocks, equivalent to about 30 days of engine block history, that light nodes should keep.

In the meantime there are already 18 running light node witnesses, 6 of them even in the top 20. In the long term the best would be to transition to 100% light nodes for engine witnesses, as the expenses of running a full node are too high compared to the rewards. The 30 day history should be sufficient for most use cases in my opinion.

The capped block / transaction history is basically the only difference of light nodes, other than that they are equivalent to full nodes. The largest benefit is, that they drastically decrease storage size to ~12GB, compared to ~130GB for a full node. Also the snapshots are only 4.5GB compared to 57GB for full nodes.

Talking of snapshots...

Light node snapshots

In addition to the full node snapshots, I am now providing light node snapshots twice a day, which can be found over here https://snap.primersion.com/light.
They are much smaller in size, which results in a much faster restoration time of about 30 minutes, compared to a few hours (~6?) for full nodes.

Restoration / setup script

I have created another PR a few weeks ago, which adds a script for easily fixing broken (forked) engine node databases or setting them up from scratch.
It can be found in the root directory of the repository (restore_partial.js) and the usage instructions can be found in the file:

/**
 * Script for helping to restore / repair hive-engine node databases. The restore mode will simply drop the
 * existing database and do a full restore.
 * The repair mode will delete all invalid blocks / transactions and drop all other collections and restore the
 * last valid state using a light node snapshot.
 *
 * These are the different modes:
 * - FULL node:
 *   - Restore by executing `node restore_partial.js`     (~30-60 minutes)
 *   - Drop    by executing `node restore_partial.js -d -s https://snap.primersion.com/` (~6 hours)
 * - LIGHT node:
 *   - Drop    by executing `node restore_partial.js -d`  (~30-60 minutes)
 *   - Restore is not supported as dropping is faster
 * */

HF26 beneficiary fix

In HIVEs upcoming HF26 a breaking change for beneficiaries was introduced, which simply changed the structure of beneficiaries. This resulted in engine nodes forking all the time, as nodes syncing blocks from HF26 HIVE nodes were parsing beneficiaries wrongly. You can read more about the change in the PR over here if you are interested https://github.com/hive-engine/steemsmartcontracts/pull/153.
The new beneficiary format now uses an object with type and value inside the extensions instead of an array e.g.:

"extensions": [
    {
      "type": "comment_payout_beneficiaries",
      "value": {
        "beneficiaries": [
          {
            "account": "alinares",
            "weight": 300
          },
          {
            "account": "hiveonboard",
            "weight": 100
          },
          {
            "account": "tipu",
            "weight": 100
          }]
      }
    }]
}]
],