Create and Broadcast Custom JSON on the Hive Blockchain with Python

Want to build a Dapp or play-to-earn game on the Hive blockchain?

Hive blockchain is very developer-friendly. There is much more to Hive than blogging. What many people do not realize is, much like Etherium, the Hive blockchain offers utility AND currency.

This, after all, is what attracted games such as Splinterlands.

Yes, the infrastructure is all there to create content, comments, social apps, but Hive also offers custom data on the blockchain in the form of JSON. This can represent anything you want (up to the size limits), from voting in a poll or staking a token, through to transferring ownership of in-game assets.

An example of Splinterlands* using this feature of the Hive blockchain to enable players to send gift cards?

* (Splinterlands was formerly Steem Monsters, hence the "SM", I learned today)

{'expiration': '2022-02-11T22:50:33',
 'extensions': [],
 'operations': [{'type': 'custom_json_operation',
                 'value': {'id': 'sm_gift_cards',

How Custom JSON Works

Before you can do pretty much anything on the Hive blockchain, you need Resource Credits (RCs). Unlike other chains, such as Etherium with "Gas Fees", these credits build up and replenish over time.

As you can see below, my little test account that I use for development still has some credits left, even after working on this code.

When you create a custom JSON operation, a signed transaction is broadcast to the Peer to Peer network, and then a witness validates it. If it is valid, then your transaction is then included in the next block. Every three seconds, the Hive blockchain produces a new block that includes all the most recent transactions since the previous one.

Unlike with, say, a MySQL database insert, you won't immediately get a unique record ID back that can be used for later instant retrieval. After your operation is broadcast, it can take up to a minute to see it appear on the irreversible stream of blocks. These are the blocks that have all been confirmed by a consensus of witnesses and won't be rolled back. If you don't care about that, you can get a swifter stream that includes the head which has not been confirmed yet.

Hive blocks are limited to 65.5KB in size and within each transaction, our Custom JSON operation is allowed to use up to 8KB. That might not sound like much but it is plenty for most needs (my first computer had only 3.5KB available to BASIC, and that had to include your code!), and of course, you can spread out your data over multiple transactions if necessary.

For our purposes, the main parameters we need to worry about are

• ID - This is just an identifier so you can find your operation later, perhaps the name of your app or the name of the action/event in your Dapp. For example, Splinterlands needs to differentiate between 'id': 'sm_token_transfer' and 'id': 'sm_sell_cards'
• User ('required_auths' / 'required_posting_auths') - The user that matches the key to be used.
• JSON - Our actual data payload in JSON format.

You can read more about the parameters on the Hive.io site.

Before we broadcast our own operations, let's see how others are using the feature ...

Reading the Hive Blockchain

This code will pretty-print the current block at the time of running:

from beem.instance import set_shared_blockchain_instance
from beem.instance import set_shared_hive_instance
from beem import Hive
from beem.blockchain import Blockchain 
from beem.block import Block
from pprint import pprint

# Put some space into the terminal
print("\n\n\n\n")

# Authenticate
hive = Hive()
set_shared_hive_instance(hive)
set_shared_blockchain_instance(hive)

# Read blocks, even if they can get rolled back
blockchain = Blockchain(hive, mode="head")
current_block = blockchain.get_current_block_num()

# Print the block
pprint(Block(block=current_block).json())

After setting up the usual Beem stuff, and connecting to the Hive blockchain, we simply get the most recent block number and then request the JSON from that block.

You will see a big mess of stuff fly past like Neo watching the matrix.

Have a look through the data you get back and you will see all the things that can be done on our little blockchain!

Now we know how to read the data from the firehose stream of blocks, we will be able to find our data when we need to. Remember we won't be able to just ask for our transaction by ID, we will need to search through until we find it.

Creating and Posting a Custom JSON Transaction

We create our transaction using TransactionBuilder(), after building the JSON - both the required JSON for the transaction and also the custom JSON we will be using for our application:

payload = {"data": "some data"}
new_json = {
      "required_auths": [],
      "required_posting_auths": [user],
      "id": app_name,
      "json": payload
    }
tx.appendOps(Custom_json(new_json))
tx.appendWif(wif)
signed_tx = tx.sign()

To ensure the legitimacy of the data, and that you are who you say you are, the transaction is signed using your key.

Of course, where I have dummy data you would replace it with the custom values that you need.

All that is left is to broadcast the transaction.

broadcast_tx = tx.broadcast()

Finding Our Custom JSON

Recall the data won't be there instantly? We need to search through the blocks to find what we are looking for, so I made this function that takes a start block and an end block, then iterates over them.

If our transaction isn't found, we can then set the start block to the previous end block, and use the current block as the new end block number and try again.

def get_transaction(start_block, end_block):

    for block_number in range(start_block, end_block):
        print(".",end="")
        block = Block(block=block_number)
        for transaction in block.json()['transactions']:
            id=transaction['operations'][0]['value'].get("id","")
            if(id==app_name): 
                print("BLOCK NUMBER: " + str(block_number))
                return transaction
    return None

Full Code Example

Copy this to a new python file (ensuring you have Beem installed) and run it.

You will be asked for your Hive username and your posting key, don't worry they are not stored anywhere, just used for signing and broadcasting the test data.

I've set my app_name that is used to set the ID to "mh", you can choose whatever you like, just don't upset the Splinterlands folks with duplicates of their names ;)

Hopefully the comments will make the rest self-documenting, but feel free to post in the comments ...

from beem.transactionbuilder import TransactionBuilder
from beem.instance import set_shared_blockchain_instance
from beem.instance import set_shared_hive_instance
from beembase.operations import Custom_json
from beem import Hive
from beem.blockchain import Blockchain 
from beem.block import Block
from beem.account import Account
from pprint import pprint
from beem.nodelist import NodeList
from time import sleep

# Application/tool identifier we will be searching for, eg. MH for Maker Hacks
app_name="mh"

# Put some space into the terminal
print("\n\n\n\n")

# Get user and key
user = input("Enter your account name: ")
wif = input("Enter your posting key: ")

# Authenticate
hive = Hive(keys={'posting': wif},node="https://api.hive.blog") # (Big apps tend to create their own nodes)
set_shared_hive_instance(hive)
set_shared_blockchain_instance(hive)

# Set up objects
blockchain = Blockchain(hive, mode="head")
current_block = blockchain.get_current_block_num()
tx = TransactionBuilder()

# Set up dummy datas
payload = {"data": "some data"}
new_json = {
      "required_auths": [],
      "required_posting_auths": [user],
      "id": app_name,
      "json": payload
    }
tx.appendOps(Custom_json(new_json))
tx.appendWif(wif)
signed_tx = tx.sign()
broadcast_tx = tx.broadcast()

# Send to the blockchain
print("CURRENT BLOCK: " + str(current_block))
print("BROADCAST: ",broadcast_tx,"\n\n")
print("Transaction broadcast complete, please wait while we find the transaction ", end="")

# Searches blocks for our app transactions
def get_transaction(start_block, end_block):

    for block_number in range(start_block, end_block):
        print(".",end="")
        block = Block(block=block_number)
        for transaction in block.json()['transactions']:
            id=transaction['operations'][0]['value'].get("id","")
            if(id==app_name): 
                print("BLOCK NUMBER: " + str(block_number))
                return transaction
    return None

# Start and end blocks to search
start_block = current_block
end_block = blockchain.get_current_block_num()

# Loop until we have a transaction
transaction = None
while(transaction == None):

    # Wait a second
    sleep(1)

    # Get the transaction if it exists
    transaction = get_transaction(start_block,end_block)

    # New start and end blocks to search if we didn't find anything
    start_block = end_block
    end_block = blockchain.get_current_block_num()


# Got a transaction
pprint(transaction)