Since the creation of a technology which is known today as blockchain,that came into existence more than a decade ago with the publication of bitcoin whitepaper back in 2008.Bitcoin started as a decentralized peer to peer payment system,but has gained gross recognition over the years as a new technology.Just like every new innovation bitcoin was flawed in terms of scalability and its cost of operation became relatively high.
In the quest to solve the limitations facing bitcoin,On 2104 a new blockchian foundation was laid with a different infrastructure which enabled developers to create various kinds of blockchain applications using “smart contracts.However with time etherum failed to solve the scalability problem with its 15 tps(transaction per second).
Various blockchain projects have emerged and tackled these issues all with its various approach,various blockchain has tried to replace the proof of work consensus with proof of stake consensus,blockchain like Eos use Delegated Proof of Stake (DPoS),where block proposers are elected by voting rather than by an on-chain algorithmic process,other projects like IOTA replaced the chain-of-blocks data structure with a DAG (Directed Acyclic Graph)data structure, which breaks the limitation of sequential processing of transactions,all these and many more are different measures adopted to solve the problems facing the blockchain system.While these methods have tried to reduce the problems of scalability,this approach has left other aspects of the blockchain lagging such as security and decentralization.
Zilliqa for instance was able to solve the issue of scalability while optimizing security and decentralization through sharding:This involves the creation of multiple groups (i.e. shards) of validators and lets them process transactions concurrently,thereby increasing the number of transactions processed per second.However Zilliqa sharding solution fell short in two ways:
By the end of this post i will introduce a new blockchain armed with the specific tools to end the limitations facing the blockchain and how its going to achieve that.
Hamony is a next generation pos (prove of stake )blockchain that is fully scalable,secure and energy efficient based on sharding.Hamony infrastructure is aimed at creating unprecedented scalability and complete decentralization for billions as the growth and use case in the industry continues to grows massively. Hamony will seamlessly address the problems of existing blockchains like applications which were not previously compatible on existing blockchain,including high-volume decentralized exchanges, interactive fair games, Visa-scale payment systems, and Internet-of-Things transactions. Hamony wants to scale trust for billions of people and create fair economy.
Below are a few of the current blockchain limitations and how Hamony aims to solve them:
However Hamony is ready to address these issues of scalability after enormous research and engineering practice. Hamony has been able to solve these issues using sharding,not only the communication and transaction but also blockchain state,thereby creating a fully scalable blockchain.
Hamony has made a breakthrough in shard security thanks to its distributed randomness generation(DRG) process which is unpredictable,unbiased,scalable and verifiable.Hamony also reshards the network in a non-interruptive manner to prevent against slowly adaptive byzantine adversaries from infiltrating the network.
Scalable Networking Infrastructure:Hamony with RaptorQ fountain code,Adaptive information dispersal alogarithm and Kademlia routing are employed to achieve fast efficient cross sectional
shard transaction that scale logarithmically with the number of shards and fast propagation of block within shards or across another network.
Consistent Cross-Shard Transactions:Hamony supports cross-shard transactions,with shards directly communicating with each other. An atomic locking mechanism is usedto ensure the consistency of cross-shard transactions.
The consensus protocol has been the core component of any blockchain.the first consensus protocol ever used is that which powers bitcoin:POW(proof of work). This determines how securely quickly blockchain validators reach consensus. POW is a process where active miners in a network race to find the solution to a cryptographic puzzle,the winner is then rewarded with the native token and is entitled to propose the next puzzle in the network or block. The POW consensus system is relatively secure with the assumption that more than 50% of the hashing power of this protocol is controlled by honest nodes,leaving the rule of this consensus that, the longest chain acts as a basic one,therefore proof of work consensus can also be referred as chain based consensus.
PBFT (Practical Byzantine Fault Tolerance) is another consensus protocol that has been used been around almost a decade now. in PBFT protocol one node is elected leader with the rest off the nodes as validators,the consensus process in PBFT there are two fundamental stages the prepare stage and the commit stage.
Hamony has made a lot of improvements in the aforementioned protocols including the PBFT system,enhancing this system and fixing its glitches we have a new consensus protocol based on fast communication across nodes,the Fast Byzantine Fault Tolerance (FBFT) is the key,in this system instead of asking validators to broadcast their votes,the lead node runs a multi-signature signing process to collect the validators votes in a O(1) -sized multi-signature and then broadcast it. So using this method of communication we only have a single multi signature broadcast,thereby reducing the communication complexity across validators.
The use of this single multi signature process is inspired by ByzCoin’s BFT which makes use of the Schnorr signature scheme for constant-sized multi-signature aggregation and forms a multicasttree among validators to facilitate the message delivery. however this process involves a secret commitment round which leads up to a two round-trips for a single multi signature. Hamony has made massive improvement in this by using BLS (Boneh–Lynn–Shacham) multi-signature to reduce it to a single round-trip for a single multi signature. In comparison between ByzCoin’s BFT and FBFT has proved to be 50% faster than, Aside from this Hamony adopts RaptorQ fountain code to speed up the block broadcasting process to its maximum.
In summary Hamony's FBFT consensus protocol involves these few steps:
- The leader constructs the new block and broadcasts the block header to all validators.Meanwhile, the leader broadcasts the content of the block with erasure coding. This is called the “announce” phase.
- The validators check the validity of the block header, sign the block header with a BLS signature, and send the signature back to the leader.
- The leader waits for at least 2f + 1 valid signatures from validators (including the leader itself) and aggregates them into a BLS multi-signature. Then the leader broadcasts the aggregated multi-signature along with a bitmap indicating which validators have signed.Together with Step 2, this concludes the “prepare” phase of PBFT.
- The validators check that the multi-signature has at least 2f + 1 signers, verify the transactions in the block content broadcasted from the leader in Step 1, sign the received message from Step 3, and send it back to the leader.
- The leader waits for at least 2f + 1 valid signatures (can be different signers from Step 3)from Step 4, aggregates them together into a BLS multi-signature, and creates a bit map logging all the signers. Finally, the leader commits the new block with all the multi-signatures and bitmaps attached, and broadcasts the new block for all validators to
commit. Together with Step 4, this concludes the “commit” phase of PBFT.
The validators of Harmony’s consensus are elected based on Proof-of-Stake. Therefore, the actual
protocol differs slightly from the one described above in a sense that a validator with more voting
shares has more votes than others, rather than one-signature-one-vote. So instead of waiting for at
least 2f + 1 signatures from validators, the leader waits for signatures from the validators who
collectively possess at least 2f + 1 voting shares.
Since late 2017 blockchain sharding has gained popularity as a scalability solution,numerous blockchain projects have researched and tried to implement this solution. Zilliqa was the first public blockchain that featured this solution,claiming a transaction speed of 2800 tps(transaction per second). Zilliqa used POW as identity registration process. Zilliqa has two basic fundamental character,the directory-service committee and multiple shard committee with hundreds of nodes which receive and process transactions separately then all shards are collected and merged at the directory-service committee. Zilliqa does not act as a state sharding solution because all existing shards will have to hold the entire blockchain state in other to process transactions.
While in Academia sharding solution has been resharced by publications like Omniledger and RapidChain which propose a scaling solution to state sharding where all existing shards contain a subset of the blockchain state. With a multi-party computation scheme called RandHound in place to produce a secure random number which is assigned randomly to differently to different shards. This method also poses a security risk,with worry a particular shard can be corrupted which can affect the rest,Omniledger prevents this corruption by reshuffling all nodes in the shards at a fixed time interval called epoch ,Bounded Cuckoo Rule is used to reshuffle the nodes without interruptions thereby creating an unpredictable reshuffling pattern.
Hamony has incorporated these solutions in its blockchain,creating a provably secure,scalable and efficient blockchain based on POW.Hamony has a beacon chain and multiple shard chains. The beacon chain serves as the randomness beacon and identity register, while the shard chains store separate blockchain states and process transactions at the same time. Hamony makes use of Verifiable Random Function (VRF) and Verifiable Delay Function (VDF) for randomness generation.
A shard is a subset blockchain that processes transactions independently while storing its data in the blockchain. A shard is entirely independent yet maintains a cross shard communication with other shards.
Cross shard communication is plays an important role in the sharding of a blockchain. feature helps to extend the boundary of shards,enabling them to share data across shards,improving the utility of a single shard. There are three types of cross shard communication:
Hamony has analyzed the three and adopted the rapid-chain driven method(shard-driven). Using this communication protocol shards can send messages across different shards seamlessly with no cost due to its communication nature. Hamony also makes use of the kedemila protocol to reduce communication complexity to as simple as O(log(N).
The beacon chain is just like every other shard chain,but performs extra duties that other shards do not,like generating random numbers and accepting stakes from from prospective validators. The beacon shard also serves another purpose-that is strengthening the security and consistency of the shard chain by including the block header from each shards which is then forwarded to the beacon chain using cross chard chain communication. the beacon chain now validates the block header using :
The hash of its previous block,and
Adding different shards block header to the beacon chain is very important for the security of shards, for instance it makes it difficult for the network to experience a single shard attack or takeover and reduces the network cost of broadcasting block headers among different shards.
The process of assigning nodes to shard have taken different approaches such as randomness-based sharding, location-based sharding , and centrally-controlled sharding, out of the three processes randomness based sharding has provably being the most secure and efficient method of assigning nodes to shards. In randomness based sharding process an agreed random number is used to determine the sharding assignment for each node in a shard. A secure randomness based sharding process must be:
Hamony has implored VRF and VDF to create a secure,verifiable,unbiased and scalable solution.First, Hamony's DRG protocol complexity is O(n) , which in practice is at least an order of magnitude faster than RandHound. Second, unlike RapidChain’s simple VSS-based approach, Hamony is unbiasable and verifiable. Third, compared to Ethereum 2.0’s solution, Hamony's uses BFT consensus to provide finality to the random number.
In summary the randomness genration follows these procdure:
- A leader sends an init message with the hash of the last block H(Bn−1) to all the validators.
- For each validator i , after receiving the in it message, a VRF is computed to create a random number ri and a proof pi : (ri, pi) = V RF(ski, H(Bn−1), v) , where ski is the secret key of validator i and v is the current view number of consensus. Then, each validator sends back (ri, pi) to the leader.
- The leader waits until it receives at least f + 1 valid random numbers and combines them with an XOR operation to get the preimage of the final randomness pRnd .
- The leader runs BFT (discussed in §2) among all the validators to reach consensus on the pRnd and commit it in block Bn .
- After pRnd is committed, the leader starts computing the actual randomness Rnd = V DF(pRnd, T) , where T is the VDF difficulty and is set algorithmically such that the randomness can only be computed after k blocks.
- Once Rnd is computed, the leader initiates a BFT among all validators to agree on the validity of Rnd and finally commit the randomness into the blockchain.
Hamony an Epochs refers the predetermined time during which the sharding structure is fixed and each shard continuously reach consensus withe the same set of active validators. At the beginning of each epoch using the DRG protocol a random number will be generated and the sharding structure will be based on this randomness. Active validators who wish to process their transactions will have to stake their token before the beginning of an epoch.
In every public blockchain the risk of Sybil attack cant be overlooked ,for instance bitcoin miners will have to complete a cryptographic puzzle in order to create the next block which is POW,in similar blockchains like Zilliqa or Quarkchain POW is used to prevent Sybil attack,
Hamony on the other hand makes use of POS to guard against Sybil attack, in Hamony in order to become a validator candidates will have to skate a certain amount of tokens,the number of tokens skated will determine the number of shares given.
A voting share is a virtual ticket that allows a validator to cast one vote in the consensus. Validators can acquire voting shares by staking tokens,according to the number of tokens staked The number of tokens required for a voting share in a consensus is automatically adjusted. At the beginning of each epoch, new validators voting shares will be
randomly assigned to shards and validators join the shards where there shares have been assigned. In their respective shard a consensus is reached when each shard possess at least 2f + 1 voting shares.
To make sure that the security of a single shard is intact and to prevent a large scale attack, the amount of voting shares by malicious validators needs to be kept below 1/3 of all the voting shares in that shard. Hamony adaptive thresholded PoS ensure maximum security by automatically adapting the price of share, also rather than entrusting the shares to individuals in a shard,the shares are controlled by the shard.
After considering the secure protocol for sharding,to reduce the risk of malicious validators from overrunning a single shard and eventually the whole shards. if the sharding structure of a shard is fixed and systematic,malicious validators can still overtake the system by corrupting the validators in the shard.
They are three models of attackers:
In order to protect the shards from a potential security takeover Hamony implores the use of Cuckoo-rule based resharding mechanism. At the end of each epoch validators who wish to retrieve their shares are evicted from the shard and new validators now join the shard with their shares.The new validators who staked during this epoch gets new voting shares. These voting shares will be randomly assigned to the shards who have more than the median of the total voting share in the current epoch . after this a constant number of the voting shares from all shards will be randomly re-distributed to the other half of the shards who have less than the median of total voting shares. Using this resharding mechanism the voting shares in all active shard will be balanced while maintaining maximum security.
Due to the large amount of data in a blockchain state,speedy synchronization in a shard is very slow,going aboutt it in the traditional way of downloading and reconstructing the shard could take days to complete. for instance redownloading the entire history in the etherum network could take days to achieve which is not efficient for a network built on resharding mechanism.
To make this process fast and efficient,Hamony first allows new validator to download the current state of that shard,so validation of transactions can start quickly. To make sure that the current state of the blockchain is valid without having to download the whole history to verify,the verification of the current state can be done the new by running new node to trace the historical block headers and validates the headers by checking their unique signatures. However signature verification traditionally takes time to verify the signatures. Hamony has a solution,the first block of each epoch will include an extra hash pointer to the first block of the last epoch,using this idea the verification will be fast and up to date to the current blockchain state.
A shard is an independent transaction processor with a state of its own,that is why it accepts and process transactions independently. In Hamony, the blockchain state of a single shard is linked to a specific account and smart contract,although a validator can have his tokens spread out in multiple shards,but they do not operate on the same smart contracts they all must have different smart contracts and bound to a single shard. A validator can move his balance across other shards using the cross shard communication mechanism.
Effective Networking is one of the limitations of blockchain technology,Hamony however has initiated and engineered new protocols to deal with ineffective networking and improve networking utilization.
Hamony as a solution has adopted kademila as the routing mechanism for cross shard messages. In the Hamony network different nodes from different maintains a unique routing table. for instance if a message wants to be sent from shard X to shard Y,the nodes in either shard will look for the closest node in the routing table with a specific ID. With kademila cross shard communication complexity has been completely reduced to a minimum from O(N) to O(logN),while reducing the network load.
Broadcasting is very vital in various instance in the blockchain like ,when a lead validator wishes to send a proposal to various validators for consensus purposes. Hamony has implored the use of erasure code rather than the peer2peer traditional system where the network load is always maximum. Using the erasure code a sender first sends chucks of the encoded message to each validator. This process reduces the load on the sender to O(M + e) where e is the size of erasure code. Therefore using this erasure mechanism the networking load of the broadcast sender is reduced to its minimum unlike in P2P protocol.
Hamony uses the RaptorQ fountain code mechanism. Each message is encoded into symbols, and symbols are sent over the wire until the receiver acknowledges successful decoding of the message using the symbols that it received. Unlike using fixed-rate codes such as Reed–Solomon where the transmission fails once the symbols have been, fountain code enables infinite, just-in-time generation and use of encoding symbols.
Hamony implores the use of ICE (Interactive Connectivity Establishment) to identity the NAT mechanism behind every nodes operation so as to employ the quickest and most efficient way to tackle the node. Such protocols include STUN, TURN and IGDP.
Hamony’s network layer, in order to solve this problem, introduces a clean separation between node identity (cryptographic key pair possessed by the node) and node locators(network/transport-layer locator where the node can be reached) using the industry-standard Host Identity Protocol Version 2 (HIPv2). HIPv2 lets locators of a node change over time while keeping the node identity, by providing mechanisms for locator discovery, node-to-node security association, and tunneling of upper-layer traffic associated with local/remote node identity as
endpoints.
After the successful commitment of a new block,all validators who participated in signing the new block will rewarded with according to their voting shares. The transaction fees are also rewarded to validators.
This feature is put in place to discourage any dishonesty and misbehavior from validators in a shard,if detected by the network.for example if a leader fails to finish the consensus protocol and triggers the leader change option,the leaders staked token will be slashed,if validators sign a new block for dishonest a purpose when detected by the network all their tokens will be slashed. However if a validator was able to prove the the misbehavior of another validators and verified by the system,the slashed tokens will be transferred to the reporting validator.
Blockchains operating on proof of work are prone to a special type of attack where honest validators keys stolen are compromised by attackers seeking to gain control over the network or a single shard. With these keys they are able to create fake and dishonest intended block where unsuspecting validators fall prey to their scheme. Honest validators can compromise their keys by a lack of security on honest validators, or more commonly, by the fact, after a validator withdraw their token, he could financially benefits if an attacker which would be looking to buy its private key, or
when signing a new block,before a new block can be signed 2/3 of the validators must sign the block which could be compromised thus granting the attacker the right to choose the next set of validators.
Proof of work blockchain protects the network from such long range attacks by giving honest validators an objective method of fork choice. In a proof of work blockchain, the fork choice to select the lead chain is the accumulated amount of work done in terms of hashes computed.
In a proof of work system the only option(objective) that can be used to select between forks is the total weighting of signatures used to approve each block. If two different block are compared using weighed signature this will lead to the following formula = Safety = Block approval key weight - Compromised key weight.
The “Block approval key weight” means the voting power of the keys that signed on the block. If,
by stake weight, more private keys are compromised than were used to approve of a block, then
the block can be forked. Until then, validators will always prefer the original, legitimate version of
the block.
Harmony maximizes the safety of each block in its proof-of-stake blockchain by maximizing this equation. It is infeasible to dis incentivize leaking private keys in the long term. Harmony instead incentivizes validators to maximize the approval weight of each block after a quorum has been achieved. This is done by requiring validators to sign each quorum-approved block before allowing those validators to withdraw their stake. These new additional signatures only need to exist within the blockchain, and they do not need to be generated at consensus time for each block. Because of this, the new signatures can be added to subsequent blocks when validators decide to withdraw their stake, and so they may freely improve the safety of the chain without impacting its liveness.
Hamony is on boarding new updates to ensure a transparent network where fraud or misbehavior of validators can be proved easily and ensuring that the cross shard communication mechanism is kept hack proof and transparent to avoid attackers hacking into the network.
Pangaea is purely an experimental zone for subscribers to test the limit of Hamony's network, thy can earn rewards and learn a lot while participating in the zone.
During the live presentation and experiment of the network, it generated a lot of traffic in its first 24hrs with figures below:
Cryptokitties a company project of Animoca,has suffered a lot due to its salability issues on the etherum network. Cryptokitties has chosen to partner with Hamony in order to collaborate on the development of a new system for crypto collectibles.
Hamony is capable of aiding in high throughput rate demand by the financial system that deals with lending and borrowing which require high transaction rate that the etherum network cannot keep up with due to its infrastructure. Teaming up Hamony with its linear sharding solution which adapts to the increasing demand on the network.
@originalworks,the images in this post are curtosey of @originalworks and Harmony Website.Harmony Website
Harmony Onepager
Harmony Whitepaper
Harmony Medium blog
Harmony Telegram group
Harmony Twitter
Harmony Linkedln
Harmony Instagram
