Hello

I've mentioned that one of the things I've been working on lately is a financial project — essentially an accountant that should eventually be able to understand both ordinary money and cryptocurrencies.

I thought I'd give a small glimpse into what that actually means.

The first problem is rather obvious in hindsight:

Before the software can do any accounting, it needs to know what actually happened.

I've been on Steem/Hive for ten years, and over that time I've accumulated quite a few accounts. Some were personal, some had particular purposes, one belonged to my old company, and some ran little automated experiments.

For the current reconstruction we're dealing with quite a few Hive accounts accumulated over the years.

Fetching their histories produced about 3.7 million account-history records, spread over 3,731 API pages.

One account alone had more than 342,000 records.

And simply downloading them isn't enough.

A blockchain operation tells us what the blockchain recorded. It doesn't necessarily tell an accountant what that operation meant.

A transfer between two of my own accounts isn't income merely because one account received money. Having the key to an account doesn't necessarily mean I own the assets in it. A Hive-Engine custom_json operation can show that an account requested something, but that alone doesn't prove that the sidechain successfully executed it.

So we've been building the reconstruction rather conservatively.

First we preserve the original evidence. Then we record what facts that evidence actually supports. Only later do we start asking accounting questions such as ownership, transfers, gains, losses, cost basis and taxation.

If two sources disagree, we don't quietly choose one.

If something is unknown, it stays unknown.

If we later discover that two records describe the same economic event, we reconcile them without deleting the original evidence.

At the moment the reconstruction workspace occupies about 24 GB on my computer.

➜ ls -alh hive-bulk.sqlite
-rw-r--r-- 1 ambience ambience  19G Sep 10 21:03 hive-bulk.sqlite

➜ du -sh bulk-attempts
4.7G    bulk-attempts

That doesn't mean Hive has stored 24 GB of information about me. Most of that is our own working database, indexes, checkpoints, provenance and deliberately retained intermediate material. The finished system should be considerably smaller.

But it does illustrate something I hadn't really appreciated when I started:

Making an accountant that supports cryptocurrency isn't primarily a matter of teaching it how to add numbers.

First you have to teach it how to know where those numbers came from, what they actually represent, and when it doesn't know enough to answer yet.

And apparently ten years on a blockchain gives it quite a lot of homework. :)

^{Yep, I might need to clean my workspace up a bit too.}