It’s been several weeks since my last report, as I was first involved in preparing for HiveFest, then the holidays hit and I decided to focus on coding while a lot of our staff was on vacation and I didn’t have as many management tasks to deal with. In particular I wanted to make some large scale changes to hivemind code while there were fewer changes going on from other programmers.

In the meantime, I’m happy to announce that we’re about to tag our first “versioned” release of hivemind: v1.24.0. The major version is being set to 1, moving out of the “beta” status that hivemind has been in for so long and to represent the major overhaul we’ve done to the hivemind code base, 24 to indicate the associated hard fork version of hived to use, and the third number (currently 0) to report on minor revisions that improve performance, add additional features, etc. This is a major milestone for our Hive work, and it means we’re now finally freeing up to take on big new tasks.

Hived work (blockchain node software)

We’re still working on creating a hived plugin that can directly write the needed data into hivemind’s database during hive reindexing and normal block reception. Most of the data being provided by get_block_api is of no interest to hivemind, so using this API to get the data is unnecessarily wasting cpu, in addition to slowing down hivemind. I expect that using the plugin approach will lead to significant speedup in the initial sync time for hivemind and it should also reduce normal hivemind live-sync write time. Work is ongoing here:
https://gitlab.syncad.com/hive/hive/-/commits/km_live_postgres_dump/

Work has also continued on the code for implementing expiration of governance votes (witness votes, proposal votes, and voter proxy changes) when an account ceases activity, work is in progress here (one of our new programmers is working on this as an introduction to the core blockchain code):
https://gitlab.syncad.com/hive/hive/-/merge_requests/160

In the last developer call, someone suggested (happily I think), that nowadays it sounded like we had at least 5 core developers who worked on hived software at BlockTrades since Hive started. I knew the number was higher, but I didn’t know the exact number, so I decided to check on it and it looks like there have been 10 (note this doesn’t include additional programmers working on other Hive software such as hivemind and condenser).

We also have some other guys that have experience with the hived code base from Steemit days, who are currently on other projects, so it’s fair to say we’ve have plenty of core programmers for the 1st layer, in case anyone was concerned on this point.

Hivemind (2nd layer microservice for social media applications)

As mentioned in intro, we’ve about to release an official version of hivemind, and a lot of work in the last few weeks was required to get there. Here’s a short list of some of the work that’s been merged to the release code base recently:

https://gitlab.syncad.com/hive/hivemind/-/merge_requests/426
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/411
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/395
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/387
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/428

And this was my “holiday work” on optimizing queries :-)
https://gitlab.syncad.com/hive/hivemind/-/merge_requests/454
It allows for a quite big speedup, and our production hivemind server itself only has load of around 1 now (there’s also a hived located there, so total load on system is around 1.5) while handling a large portion of Hive’s API traffic. In short, hivemind can now handle a fairly huge amount of traffic without requiring a powerful server (my initial guess is this single hivemind server could handle around 8 times this traffic without a problem). This means that Hive has become a lot more decentralized since your average unix enthusiast could now inexpensively setup a server to serve all our traffic.

I’ll prepare a detailed report on the results of optimizing the latency of individual API calls later in a spreadsheet, because the timing information there could be quite useful to 2nd layer app developers.

We also have pending work to hivemind that hasn’t yet been merged, as we still have a long validation time on changes (because of time required for a “full sync” to 50m+ blocks), and we wanted to get out an official release and move releases for API nodes back to “master” branch and leave develop branch for experimental changes (due to urgency of development and frequent updates to database schema, we were using develop branch as our release branch for API nodes ever since HF24). As part of this official release, we ran a full sync test on the version just prior to optimizations from MR454 above, but changes to 454 are unlikely to affect that result, since it’s mostly about API optimization.

Condenser (open-source code for sites such as https://hive.blog)

We finished and deployed decentralized list changes, but haven’t had much time to test them yet in production. AFAIK they are working ok, however. I’ll probably do some testing later today.

What’s the plan for next week?

• Continue creating hivemind tests.
• We still have a few more optimizations that can be done similar to the changes done in 454, probably will complete them this week.
• Move hivemind tests into hivemind repo instead of tests_api to simplify practical testing.
• Begin design analysis for slimmed-down 2nd layer microservice and for development of smart contract platform based on it.
• Continue work on speedup of hivemind full sync via the hived plugin as the slow sync time has a big impact on the speed of hivemind CI (which sets an upper limit on how fast we can validate changes). This work is also required for the slimmed-down 2nd layer microservice.
• Continue work on governance changes for HF25(details on this work here: @blocktrades/roadmap-for-hive-related-work-by-blocktrades-in-the-next-6-months).