18th update of 2021 on BlockTrades work on Hive software

blocktrades update.png

Below is a list of Hive-related programming issues worked on by BlockTrades team since my last report.

Hive network upgraded to Equilibrium

The most significant accomplishment was successful upgrade of Hive via hardfork 25 (the Equilibrium release), and a lot of time was spent monitoring the upgrade and supporting Hive apps as needed during the transition.

The Equilibrium upgrade did have one unanticipated side-effect: the change in curation rules resulted in votes cast after the hardfork being much stronger than votes cast before the hardfork with respect to curation weight, which meant that votes cast in the days before the hardfork generally didn’t receive much in the way of curation rewards. This was a temporary effect that has now been resolved since all posts being actively voted on now were created after the hardfork, but hopefully we’ll have enough traffic on our next iteration of the testnet that we’ll be able to detect such issues ahead of time.

Hived work (blockchain node software)

Improvements to Testtools and tests used to verify hived functionality: https://gitlab.syncad.com/hive/hive/-/merge_requests/272

Added new –exit-before-sync flag to hived’s command-line interface (useful for dumping a snapshot without then syncing, see https://gitlab.syncad.com/hive/hive/-/issues/66 for more details on why this option was added):
https://gitlab.syncad.com/hive/hive/-/merge_requests/232
https://gitlab.syncad.com/hive/hive/-/merge_requests/273

We fixed the previously reported bug that requires a hived to be restarted after loading a snapshot:
https://gitlab.syncad.com/hive/hive/-/merge_requests/274

We have been analyzing the performance of our new “blockchain converter” tool for creating testnets quickly that mirror the mainnet and we’ve identified some code associated with nonces as the potential bottleneck.

Hivemind (social media middleware)

We’ve added a new programmer to our hivemind team and he’ll initially be working on testing and minor bug fixes as a means of learning the code base. His first merge request is here:
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/518

We added code for checking consistency of the hive_blocks table (this is part of previously mentioned plan to ensure robust operation in the case where hivemind’s postgres process shuts down suddenly or a rollback fails):
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/516

We’re continuing work on improving performance of the update_rshares function immediately after massive sync of a hivemind instance. We’re trying two different alternatives to improve overall performance: 1) changing massive sync so that it updates rshares for paid posts on-the-fly, reducing the work of update_rshares to only updating rshares for unpaid posts (this approach requires introducing a new hived virtual_operation) and 2) adding an index (at least temporarily just after massive sync) to speed up update_rshares. Both approaches are currently undergoing testing.

We also resumed research into the reason why some hivemind nodes consume more memory than others. It has been suggested that it may be related to differences in python or python-library installations on the different systems, which I’m inclined to believe at this point, as we’re no longer seeing unexpected memory consumption on any of our production nodes running the latest official hivemind version. So if we’re unable to replicate this issue in our forthcoming tests, we’ll likely drop this issue soon, after merging in our diagnostic changes that identify sources of memory usage better.

Hive Application Framework (HAF)

Our primary dev for this work is currently on vacation.

I had hoped we would still be able to work on the psql_serializer plugin (which feeds data from hived to hivemind under the HAF system) in the meantime, but the dev tasked with that was tied up with other issues (e.g. fix of snapshot problem). A new dev has been assigned to work on psql_serializer this week (the previously tasked one is going on vacation for two weeks).

Condenser and wallet (open-source code base for https://hive.blog and other similar frontends)

We reviewed and merged in a number of community-contributed upgrades to condenser and its wallet.

From @quochuy: https://gitlab.syncad.com/hive/wallet/-/merge_requests/102

From @guiltyparties:
https://gitlab.syncad.com/hive/wallet/-/merge_requests/101
https://gitlab.syncad.com/hive/condenser/-/merge_requests/268

From @eonwarped: https://gitlab.syncad.com/hive/condenser/-/merge_requests/269

What’s next?

Several of our Hive devs are either on vacation now or going on vacation this coming week (they had been delaying their vacations to be available for the hardfork and any potential problems that might arise afterwards). So we’ll only have 8 BlockTrades devs working on Hive for the next two weeks, and our progress will inevitably slow some during this time.

After all our Hive devs return from vacation, we’ll take a couple of weeks to begin planning what work to schedule for the next hardfork (HF26). I have some preliminary ideas for improvements that our team will work on, but we’ll make a full list of proposed changes, then begin to prioritize what we want to fit on the roadmap for HF26. My plan at this time is to stick to our existing “upgrade Hive protocol every six months” schedule if possible.

Also, as during previous hardforks, our roadmaps aren’t fixed in stone, so we may consider making other proposed changes even after the initial roadmap is published, assuming the changes aren’t too big.

Note that the above process doesn’t mean we don’t have clear development goals prior to the completion of the HF26 roadmap. For one thing, we will be making performance upgrades to hived that don’t require an actual hardfork, and these changes will generally be released as they are completed.

Even more importantly, at this point our highest priority tasks revolve around the creation of the HAF framework and HAF-based applications, and this is all layer 2 work that doesn’t require any Hive protocol changes that would necessitate a hardfork. In other words, we can also release HAF and associated apps to the dev community as soon as they are ready, without the labor and scheduling issues involved in getting nodes to upgrade as part of a hardfork.

H2
H3
H4
3 columns
2 columns
1 column
43 Comments