Working on HAT has led me to learn a lot about blockchain architecture and I would like to share the journey with you all. I'll start with sharing the lessons I learned about how transactions are designed in the scope of a blockchain.
If you want to learn a bit more about HAT and its intended purpose, you can read the intro post and the brief whitepaper.
In a nutshell, it is a sidechain (separate blockchain) that Hive accounts can interact with to create tokens and monetize content.
Block explorer: https://hat-view-testnet.imwatsi.com/
Development node: https://hat-testnet.imwatsi.com/
Design
Transactions are at the core of blockchain architecture and I had to design the basic structure of a transaction on the HAT chain. This means how it is formulated, what it contains and how it is broadcast, processed and then stored.
I had to define the different types of transactions that would be supported. So far, I have these:
- Transfer: moving tokens from one HAT account to another
- Airdrop: moving tokens from the system HAT account to a specified account
- Stake: move tokens into a time-based lock, to gain associated stakeholder-based privileges
You can see some of the definitions in this script.
I'm experimenting with two broadcast/storage approaches:
- Serialized JSON (as is used by Hive)
- Custom CSV (my preferred)
While CSV takes up less storage/bandwidth resources, it makes it necessary to develop extra parsing techniques for things such as the memo field in a transfer operation. So I developed my own custom CSV parsing function, specifically for use with HAT transactions.
Here's a code snippet of the custom CSV parsing function:
def parse_transaction_payload(payload):
elements = []
buffer = ""
in_line = False
for ch in payload:
if ch == '"': in_line = not in_line
if ch == "," and not in_line:
elements.append(buffer)
buffer = ""
continue
buffer += ch
elements.append(buffer)
return elements
Signatures
I had to learn how the private/public key signing and signature verification works.
Code snippets of signature verification:
sig = base64.b64decode(signature)
pubkey = unhexlify(repr(Base58(authority, prefix="STM")))
pk = ecdsa.VerifyingKey.from_string(pubkey, curve=ecdsa.SECP256k1, hashfunc=sha256)
message = chain_id + transaction
digest = sha256(message.encode('ascii')).digest()
try:
valid = pk.verify_digest(sig,digest)
return valid
except:
return False
And to sign a transaction, I'm using the code below. Do note that I recently took a deep dive into how Hive signs transactions and learned about canonical signatures and how they're used to avoid transaction malleability.
I am working on new methods to sign transactions that take this into account and will share my progress when I push new code.
For now, this is a snippet from a transaction signing script:
privkey = unhexlify(repr(Base58(private_key)))
sk = ecdsa.SigningKey.from_string(privkey, curve=ecdsa.SECP256k1)
message = chain_id + transaction
digest = sha256(message.encode('ascii')).digest()
sig = sk.sign_digest_deterministic(digest)
signature = base64.b64encode(sig).decode()
In a few days, I will share a public script that people can use to sign transactions and broadcast to the HAT chain themselves, through my node.
Storage
When storing the transactions in the database, the data is saved as the raw CSV strings that they are broadcast as. This means during block replay, you can evoke the same processes as those evoked from the live transaction reception and processing route, without much code work.
I might do a post with a more complete explanation of the database architecture I am using for HAT and how it all works together to store blocks, transactions and transaction history.
If you want to see the DB schema as it is now: https://github.com/imwatsi/hive-attention-tokens/blob/master/hive_attention_tokens/chain/database/schema.py
That's all I have on HAT's transaction design... Thanks for reading.
HAT is a sidechain and protocol for attention tokens and content monetization, controlled by accounts on the Hive blockchain.
If you want to learn a bit more about HAT and its intended purpose, you can read the intro post and the brief whitepaper.
Block explorer: https://hat-view-testnet.imwatsi.com/
Development node: https://hat-testnet.imwatsi.com/