Today we’re delighted to announce the release of a new consensus mechanism that we believe will radically change the world of distributed computing. Whilst we rarely engage in the hype that’s all too common within the crypto sphere, this one is worth shouting about — because we’ve created the world’s first (as far as we’re aware!) completely decentralised, open source, highly asynchronous, Byzantine Fault Tolerant consensus mechanism.
PARSEC (Protocol for Asynchronous, Reliable, Secure and Efficient Consensus) has been built to power the SAFE Network. PARSEC will be released under a GPL v3 licence (with linking exception). It will be free for anyone to build upon and likely prove to be of immense value to other decentralised projects facing similar challenges.
If you aren’t already aware, the SAFE Network is the autonomous, decentralised and secure Internet that is currently available to the public in its Alpha testing phase. It’s been in development by MaidSafe for over a decade, with a community of thousands around the world. As one of the most established projects in the decentralised world, there’s been a lot of interest recently as people are starting to realise that the SAFE Network is about far more than simply decentralised storage. And what’s more, it doesn’t use a blockchain…
PARSEC solves a well-known problem in decentralised, distributed computer networks: how can individual computers (nodes) in a system reliably communicate truths (in other words, events that have taken place on the network) to each other where a proportion of the nodes are malicious (Byzantine) and looking to disrupt the system. Or to put it another way: how can a group of computers agree on which transactions have correctly taken place and in which order?
Or to bring it down to basics with a couple of simple SAFE Network examples: PARSEC consensus does such things as enabling a user to store a piece of data and enables the Network to confirm a Safecoin transaction.
Hasn’t this been solved before?
In a word, no.
To explain why, let’s take a look at Bitcoin. The network in that case reaches consensus by using a blockchain. A blockchain is basically a shared ledger that everyone relies on. It’s a record of consensus — an agreed history of everything that has taken place on the network. Because the Bitcoin network (just like the SAFE Network) is permissionless — in other words, you cannot prevent anyone from taking part who wants to — Satoshi built a system that has a couple of key characteristics: only one node can update the global ledger at a time; and, crucially, there’s no way to identify which node that might be in advance (thanks to proof-of-work). Consensus is achieved and defended by protecting the identity of that node until their job is done.
This is a huge deal. But, as we’ve discussed many times before, the SAFE Network cannot — and does not — use a blockchain.
Because as powerful and innovative as Satoshi’s creation has been, blockchain technology comes with some fairly significant downsides. If you’re building a secure, autonomous, decentralised data and communications network for the world like we are with the SAFE Network, then the limitations of blockchain technology when it comes to throughput (transactions-per-second), ever-increasing storage challenges and lack of encryption are all insurmountable problems for any system that seeks to build a project of this magnitude. Hopefully there will be improvements along each of these fronts over time. But the very design of blockchains means that their use case isn’t suited to a global internet that deals with vast amounts of data that needs to be both private and secure.
So despite being big fans of blockchain technology for many reasons here at MaidSafe, the reality is that the data and communications networks of the future will see millions or even billions of transactions per second taking place. No matter which type of blockchain implementation you take — tweaking the quantity and distribution of nodes across the network or how many people are in control of these across a variety of locations — at the end of the day, the blockchain itself remains, by definition, a single centralised record. And for the use cases that we’re working on, blockchain technology comes with limitations of transactions-per-second that simply makes that sort of centralisation unworkable.
So with blockchains unable to meet the challenge of powering the SAFE Network, we’ve explored a number of other approaches to consensus since we started working on the problem back in 2006. You may have heard of leader-based consensus systems. Broadly speaking, these systems work by having the individual nodes sending their transactions to a chosen leader who then decides on the order of transactions that the Network will follow. However, whilst these solutions (with names such as PBFT, Raft and Paxos) may be useful in closed (permissioned) Networks where all of the nodes are known or authorised, they are not suitable for our purposes — because as soon as you have a known leader, you introduce problems. As a couple of examples, having a leader means that you now risk having a malicious leader that sends huge amounts of messages and you also have a target for others to attack if they want to bring down the Network (for example, with a DDOS attack).
Another approach to seeking consensus in public networks is known as Proof of Stake. Here, each node is forced to stake real value on the consensus that they see as being correct (the idea being that no one wants to lose their own money by attempting to subvert the consensus process). There are many different flavours here. Some approaches have clear weaknesses (in the sense that they will lead to the centralisation of power) whilst others appear more promising. However, each suffers from the same issue as Proof-of-Work blockchains for our purposes: a lack of highly asynchronous Byzantine Fault Tolerance.
The concept of Byzantine Fault Tolerance is a crucial one. It means that it is mathematically guaranteed that all parts of the Network will come to the same agreement at a certain point in time. Exactly what PARSEC achieves.
This is very different to any blockchain-based consensus mechanism. With blockchains, the probability that the consensus cannot be reversed increases with every additional block that is added to the blockchain — but crucially, it never reaches 100% certainty. Put simply, this is down to the way in which blocks are added in blockchain technology and not something that will change in the future. With PARSEC, consensus is mathematically guaranteed as certain (as well as having a throughput that dwarves blockchain tech). And this is a huge thing.
What’s more, PARSEC is highly asynchronous. This means that there is no trusted setup nor any synchronous steps involved, as might be required in common coin implementations or threshold signature schemes. In other words, any consensus mechanism has to be able to work perfectly, even when different events on the Network are reaching nodes at wildly different times and allowing for the fact that nodes may suffer technical issues around the world or the Network could be attacked.
Recently, there’s been a lot of discussion around technologies such as Hedera Hashgraph. They’ve looked at solving many of the same challenges that we have but ultimately there are a number of significant differences to PARSEC which we believe make our consensus mechanism more powerful.
Most significantly, the Hashgraph consensus is closed source, restricting its use significantly. It is also unusable for our purposes as it requires a fixed set of known nodes. Furthermore, a network using the Hashgraph consensus is only proven to reach agreement if it is guaranteed that there is no sophisticated adversary on the Network.
On top of this advantage that PARSEC has when it comes to unconditional performance and resilience to adversaries on the Network, PARSEC has also been developed in a way that reflects our core values. Anyone who has looked a little further into the Hashgraph algorithm will see that it has only been shown to work so far on a network in which the nodes are identified and do not change — in other words, a permissioned network. But at MaidSafe, open source is in our DNA. We’re building a system that guarantees privacy, security and freedom for every individual on the planet who can get online.
Therefore, PARSEC has been designed and built for everyone to use. We are, as ever, keen to engage and collaborate with any other decentralised projects who are currently exploring DAG-like technologies (such as Byteball and IOTA, amongst others) who would benefit from not requiring controllers, trusted nodes or any such centralised components, provided that the focus is always on building open source, permissionless networks. Free of charge and free (as in freedom) forever. The result? We now have a consensus algorithm that provides the best performance of any asynchronous consensus algorithm in the world; with better maths proofs (in the sense that they prove that there can be no stalling even with max byzantine adversary); and crucially the absence of any patent or restriction on usage.
With PARSEC, we’ve created a solution that gives everyone the ability to have highly asynchronous vote ordering in a BFT-way. In other words, the events appear in an agreed order that every computer can sign and accept is good.
PARSEC will reduce significantly the number of testnets and code that the SAFE Network requires before launch. The next stage will be to add this to add, remove, split, merge and secure message relay. At that stage, routing will be complete and we will be releasing the Alpha 3 network. This marks the end of the last significant area of research before launch, after which we will be finalising Vault rules, Safecoin farming, SOLID integration and all of the client API’s and bindings.
As a result, the work behind PARSEC is summarised in the first of three papers that we will be releasing over the coming weeks. Next up will be a paper that details how Group Membership works within the Network, whilst the final paper that we release will explain Sharding. And the exciting thing about these papers are that these elements have already been implemented.
The Protocol for Asynchronous, Reliable, Secure and Efficient Consensus is the most elegant solution to the highly asynchronous byzantine fault tolerance problem in existence today. PARSEC has been fully proven mathematically and MaidSafe are now planning its implementation. PARSEC will be a vital part of the SAFE Network. But any team that is working on distributed consensus problems can now use this technology. We’re making it open source, and available for anyone to use. We’re looking to build and improve on top of this and we strongly believe that the best way to do this is to open it up to the world for comments and contributions.
The implications for distributed computing are significant and we look forward to hearing the response from others in the field to the work that we are releasing today. This is a technology that will power the SAFE Network, the only truly next-generation decentralised autonomous internet that promises privacy, security and freedom by default. And it’s a technology that you can use for your own projects as we all look to build the future together.
You can read the full details in the White Paper and RFC (Request For Comment). We look forward to receiving your feedback. Please join us on this journey as we build the only autonomous, decentralised internet for data and communications that promises security, privacy and freedom by default (https://safenetforum.org/).
