Running on @atexoras.witness

To be honest, and so far, this looks like a very interesting thing. And this for me @forykw is only "experimental" because we all are in the very early stages of a new beginning of an amazing decentralization path and it is only humble to assume you can fail.

Where to begin...

I have followed a couple of things as guidance, but because I usually need just a direction, and then I follow my own path. For the not so certain ones, I would advise following @rishi556 👉 Guide (best one so far, at least for me). There are a couple of reposts I did at @forykw, so check them out too.

My own "sauce"

Unlike most of the crypto community, I am not used to the cloud world or web development for that matter. Hence my experience is somehow more "Dinausour" alike (even if I am not that old).

$ tail -f node_app.log | grep round | grep atexoras
2021-02-08 23:01:01 info: [Blockchain] scheduled witness atexoras.witness for block 4699606 (round 14859)
2021-02-08 23:07:57 info: [Blockchain] scheduled witness atexoras.witness for block 4699689 (round 14870)
2021-02-08 23:08:12 info: [Blockchain] witness atexoras.witness signed round 14870
2021-02-08 23:10:15 info: [Blockchain] scheduled witness atexoras.witness for block 4699754 (round 14880)
2021-02-08 23:16:53 info: [Blockchain] scheduled witness atexoras.witness for block 4699862 (round 14895)
2021-02-08 23:22:49 info: [Blockchain] scheduled witness atexoras.witness for block 4699952 (round 14908)
2021-02-08 23:35:31 info: [Blockchain] scheduled witness atexoras.witness for block 4700130 (round 14933)
2021-02-08 23:36:03 info: [Blockchain] witness atexoras.witness signed round 14933
2021-02-08 23:36:45 info: [Blockchain] scheduled witness atexoras.witness for block 4700150 (round 14936)
2021-02-08 23:54:44 info: [Blockchain] scheduled witness atexoras.witness for block 4700470 (round 14982)
2021-02-09 00:11:40 info: [Blockchain] scheduled witness atexoras.witness for block 4700763 (round 15024)
2021-02-09 00:14:42 info: [Blockchain] scheduled witness atexoras.witness for block 4700814 (round 15031)
2021-02-09 00:16:19 info: [Blockchain] scheduled witness atexoras.witness for block 4700843 (round 15035)
2021-02-09 00:28:00 info: [Blockchain] scheduled witness atexoras.witness for block 4701035 (round 15063)
2021-02-09 00:32:05 info: [Blockchain] scheduled witness atexoras.witness for block 4701108 (round 15073)
2021-02-09 00:34:17 info: [Blockchain] scheduled witness atexoras.witness for block 4701145 (round 15078)
2021-02-09 00:34:49 info: [Blockchain] witness atexoras.witness signed round 15078
2021-02-09 00:41:49 info: [Blockchain] scheduled witness atexoras.witness for block 4701261 (round 15095)
2021-02-09 00:42:09 info: [Blockchain] scheduled witness atexoras.witness for block 4701271 (round 15096)
2021-02-09 00:42:37 info: [Blockchain] witness atexoras.witness signed round 15096
2021-02-09 00:42:37 info: [Blockchain] scheduled witness atexoras.witness for block 4701273 (round 15097)
2021-02-09 00:43:34 info: [Blockchain] scheduled witness atexoras.witness for block 4701286 (round 15099)

Example: My simple way to check when I am about to sign a block or when I do one.

I mostly followed @rishi556 for purposes of comparison but then, I have used (not because I needed) a 4-core and 8GB RAM VM... this is basically because I am using the VM as a sort of "mini-workstation" for the witness node (experimental) where I run other stuff that might occupy memory and use more CPU. Gives me a bit more flexibility on my "tests" and learning curve.

@rishi556 (you are being tagged! LOL) was right about the minimum requirements... 2-core is more than enough, and 2GB RAM + 2GB SWAP or 4GB RAM is pretty enough for the witness node at the moment.

Although, I wish to lay some extra details about these...

There are a couple of initial phases that might make use of more CPU+RAM, depending on the IO subsystem. Those are the Mongo DB restore and the re-sync phase after starting the node. But for sure, if you are IO bound, those will not make any use.

My few HIVE (cents)...

These are my own preferences and somehow biased by the business world experience I work with.

1. [Easy] Disable Swapiness (so that swap is only used when there is no more RAM free) - sysctl -w vm.swappiness=0 and then add vm.swappiness=0 it to /etc/sysctl.conf
2. [Advanced] Around the same kind of optimizations and depending on if you are using a VM or bare-metal, increase the TCP buffers... would make some difference on syncs at least (give you have the disk IO). This is offcourse depending on your connection to the Internet. Google it, or check some standards I usually follow for +10Gb networks.
3. [Easy] If you like a great monitor... I have never found a better one than nmon, so far. On Ubuntu, just do sudo apt install nmon -y. Then just type nmon followed by c (CPU), m (Memory), d (disk) and n (network). For more type h for help and you will easily get to all the other options. Any extra questions, let me know. I use this tool for more than 15 years.
4. When using the newest versions of Ubuntu (20.x), and while updating nodejs to the latest version, use instead npm ci on top of the cloned repo, to install the dependencies. There was a change on how npm works and if you use npm i it might install the latest versions of depedencies which are not guaranteed to work with the current state of the hive-engine code. Note: Althought its a great way to understand if there are any "warnings" to watch for. I have learned this today.
5. I prefer to clone tags and then work my way around git options to update the node. Will share more of my setup on a followup post. But for who understands, here is my advice. Use more than one pm2 instance to setup things, in a prod, preprod, test, dev environment. Then work things via a script to automate. (this might be overkill for most, but makes things easier over time).

Some other notes

The right way in my view to do "any" kind of updates (unless you have 2 mongo DBs running in parallel) is to:

1. Disable the current witness first (node witness_action.js unregister)
2. Go offline (pm2 stop <app_name>)
3. Update/backup/etc (will be covered on the next post too)
4. Start the node (pm2 start <app_name>)
5. Then register/enable things again (node witness_action.js register)

Otherwise, you can miss blocks, depending on your round schedule and approval weight.

Simple HA Solution...

Using the same IP... useful for most people. Still need to mangle around pm2 because I saw some HA features and what I am saying might be actually stupid. But this will work anyway. This is manual but possible to automate, process.

1. Configure two instances on pm2 (same clone, different directories, same ports, same configs). Instance PRIMARY and SECONDARY.
2. Then when you need to update, go to the SECONDARY (non-running one) and update it. Check all you wish before starting it.
3. Stop the PRIMARY instance.
4. Copy the startHiveBlock block number of the PRIMARY instance (on config.json) to the SECONDARY one.
5. Start the SECONDARY instance.

Note: This is assuming you don't need to replay the node.

For different IPs HA deployments, you will most likely need to run separate mongo DBs and hence it's just like another node. This would be the "Enterprise" architecture in my view.

If that comes around, I will post about it, otherwise DM me somewhere.

Network/Security

This will be a topic for the next post. As I am running late for tonight.

ToDo's

• A script to kill the node and unregister witness if it fails to ping the network for a certain amount of time. To avoid network instability.
• Find out the best way to run live HA (high-availability) mode, where the node does not need to unregister from the network.
• Fine-tune the backup/restore scripts.
• Think about snapshots (I think I will try ZFS)

If you wish to vote for @atexoras.witness

You first need to stake WORKERBEE and then either go to https://tribaldex.com/witnesses and vote for your witnesses or if you are running from the node that is your witness... simply execute the following script:

node witness_action.sh approve <witness>

or the other way around...

node witness_action.sh disapprove <witness>

Being <witness> any of the accounts showing at https://tribaldex.com/witnesses

One step closer to the best decentralized hybrid blockchain!

Thanks to everyone else sharing their insights and skills across this, and if you have a node running, consider joining the Discord server, and let the admins know.