Programming - Implementing a Blockchain using Python and Flask [Dev Blog 1]

[Image 1]

Introduction

Hey it's a me again drifter1!

As you may already know, I'm studying Computer Science and already have a Bachelor's degree. Well, I'm currently going for a Master's degree as well, and in one of the courses, I took the challenge of implementing a blockchain on my own. And the challenges that I will be facing during the implementation of the system will be the topic of a series of articles now.

Its not my first attempt though, as I already had a team-up with other fellow students. During a development and design course, we crazy dudes chose to implement a proof-of-work blockchain in C. You can find this implementation here. The programming language C is of course way to low-level, and so we have had quite a few problems in making the system more advanced. I implemented the wallet GUI, and miners in OpenMP and CUDA for that project! This implementation is very efficient, and quite a few open-source libraries, such as Ulfius and Jansson, have been used! I highly suggest you to check it out! A major reformation of the whole code-base (sharing code between the various systems etc.) is also currently in our to-do list for the future.

Either way, now for the new project! To get straight to the point, for this other course, I chose to design and implement the complete system on my own. To make things simpler for myself, I chose to use the language Python, and the main library that I chose to use for the communication between the various systems is Flask. This system will not be implementing a proof-of-work only consensus, but I will try to experiment with proof-of-stake and hybrid schemes as well. I already have implemented the rough communication between the various nodes of the system, and am already on the way of implementing the actual blockchain structure and REST API endpoints for that as well. But, before that, I would like to explain how the nodes are initially connected to the network, how they keep track of other nodes etc.

I hope that you will enjoy these kind of dev blogs that I will be making for this project. It's a little different from the tutorial stuff that I usually upload. It will be a nice change of pace for me as well.

So, without further ado, let's get straight into it!


GitHub Repository


Coding Environment Setup

Getting started with Python and Flask is a piece of cake. Of course, Linux systems are preferable, and so I chose the WSL2 + Docker route, just to be safe.

For coding I generally use Visual Studio Code, and when using Docker Containers, I think its a no-brainer! I just chose a Python Container and installed the additional Flask and Requests modules using simple pip commands. The required extensions of VS code also show up directly, and so its very quick to setup. I might add a complete guide on the GitHub repo sometime soon, or even upload a ready-to-go Docker image as well.

Lastly, for sending REST API requests I chose Insomnia, which is open-source and very easy to use as well.


Node Communication

When a new node wishes to enter the blockchain, it of course has to somehow find other nodes that are already a part of it. An easy way to implement this behavior is by updating the known nodes in 3 steps:

  1. Check if the known nodes are online/reachable, and remove those that aren't
  2. Retrieve the connections of known nodes and add them to the known nodes
  3. Retrieve known nodes by contacting a DNS server
Of course, not all steps have to be done each time, as it would be way to time-consuming, and so one of the settings of the client/node is the limit of how many nodes should be known in order to skip the next step.

Connection Check Endpoint

The endpoint that each client has, that is used for checking if the node is available, is simply: ip_address:port/

A GET request is done to that endpoint and if this request is successful, the node is considered online. In other words, the request and response parts are empty.

Here an example for checking if the local client 127.0.0.1:55000 is online.

Initial Connection Endpoint

Both the client, and the DNS server, also have another endpoint used for the initial connection to the network. Each node contacts the DNS server first through a POST request on ip_address:port/ with its node information in JSON format in the request. The callback than updates the local to that client (or DNS server) nodes.json file accordingly.

DNS Server

The DNS server simply checks if the nodes are online/reachable using those endpoints of the client, and removes clients accordingly. A thread is spawned for that purpose, which contacts all known nodes after a specific interval of time has passed.

Client

The clients also spawn a thread, which executes the previously mentioned steps of node communication.


Node Structure

A node is defined as the ip address - port pair, where the Flask app is running on. All objects are treated as JSON, and so a node is defined as:

{
    "ip_address" : "127.0.0.1",
    "port" : 55000
}
The node.py file in the common directory specifies the class/object and a function for creating the JSON format easier:
class Node:
    def __init__(self, ip_address, port):
        self.ip_address = ip_address
        self.port = port
def __repr__(self) -> str: return self.ip_address + ":" + str(self.port)

def json_construct_node(ip_address, port): return { "ip_address": ip_address, "port": port }
def json_node_is_valid(json_node): if len(json_node) == 2: keys = json_node.keys() if ("ip_address" in keys) and ("port" in keys): return True else: return False else: return False

Nodes are kept in a local (to each client / dns server) file with the default name nodes.json, as an array of this construct.


Updating the Local Nodes File

Three endpoints are available in order to access/modify the nodes.json file that each client has:

With the first one, one simple retrieves all known nodes of the specific client or DNS server, and so no request body is needed. An array of JSON nodes is returned, as shown below.

For the remaining two, a JSON node is feeded into the request, in order for that node to be added or removed, depending on the HTTP verb.

The Flask endpoints and callbacks for the nodes are implemented in the node_endpoints.py file of the common directory. Using the json module and a settings structure that specifies the location of the file, the code for all these endpoints is simply:

def node_endpoints(app: Flask, settings: Node_Settings) -> None:
@app.route('/', methods=['POST']) def initial_connection(): json_node = request.get_json()
if (json_node_is_valid(json_node)):
# send request to local endpoint requests.post("http://" + settings.ip_address + ":" + str(settings.port) + "/nodes/", json=json_node)
return json_node else: return {}
# Node Endpoints
@app.route('/nodes/', methods=['GET']) def retrieve_nodes(): json_nodes = json.load(open(settings.nodes_path, "r"))
return json.dumps(json_nodes)
@app.route('/nodes/', methods=['POST']) def add_node(): json_nodes = json.load(open(settings.nodes_path, "r"))
if request.get_json() not in json_nodes: json_nodes.append(request.get_json())
json.dump(obj=json_nodes, fp=open(settings.nodes_path, "w"))
return json.dumps(json_nodes)
@app.route('/nodes/', methods=['DELETE']) def remove_node(): json_nodes = json.load(open(settings.nodes_path, "r"))
if request.get_json() in json_nodes: json_nodes.remove(request.get_json())
json.dump(obj=json_nodes, fp=open(settings.nodes_path, "w"))
return json.dumps(json_nodes)

As you can see, these endpoints access the local nodes.json file, and the first one even does a request to the local (to that node) endpoint for adding nodes.


Terminal Arguments

Lastly, this first implementation also uses the argparse module of Python, and so the various settings, which are specified for the client and DNS server, can be easily put as arguments to the program. For example, the DNS server has the following settings:

  • ip_address (default: "127.0.0.1")
  • port (default : 42020)
  • directory (default : "../.dns_server")
  • nodes_filename (default : "nodes.json")
  • update_interval (default : 60 - in seconds)
which are easily specified as arguments as follows:
parser = ArgumentParser()
parser.add_argument("-i", "--ip", default=None, type=str,
                    help="ip address (default : 127.0.0.1)")
parser.add_argument("-p", "--port", default=None, type=int,
                    help="port (default : 42020)")
parser.add_argument("-d", "--dir", default=None, type=str,
                    help="directory (default : ../.dns_server)")
parser.add_argument("-n", "--nodes", default=None, type=str,
                    help="nodes filename (default : nodes.json)")
parser.add_argument("-u", "--upd_int", default=None,
                    type=int, help="update interval (default : 60)")
args = vars(parser.parse_args())
settings = DNS_Server_Settings( ip_address=args["ip"], port=args["port"], directory=args["dir"], nodes_filename=args["nodes"], update_interval=args["upd_int"] )

For example, in order to execute the client at port 55000:


RESOURCES:

References

  1. https://www.python.org/
  2. https://flask.palletsprojects.com/en/2.0.x/
  3. https://docs.python-requests.org/
  4. https://docs.microsoft.com/en-us/windows/wsl/install-win10
  5. https://www.docker.com/
  6. https://code.visualstudio.com/
  7. https://insomnia.rest/

Images

  1. https://pixabay.com/illustrations/blockchain-block-chain-technology-3019121/
The rest is screenshots...

Final words | Next up

And this is actually it for today's post!

I will keep you posted on my journey! Expect such articles to come out weekly, or even twice a week, until I'm finished!

Keep on drifting!

H2
H3
H4
3 columns
2 columns
1 column
8 Comments
Ecency