In the development of the blockchain, the impossible triangle of decentralization, security, and performance is the most discussed topic. How to improve the performance of the blockchain and make it commercially possible while ensuring the first two? On September 5th, at the POD conference hosted by Odaily and strategically co-organized by 36Kr Group, Dr. Jia Yaoqi, the co-founder and technical director of Zilliqa and the National University of Singapore, gave his answer in a speech at the venue.
Jia Yaoqi said that the current solutions to improve the throughput and scalability of the public chain, such as increasing the block size, proxy consensus mechanism, and off-chain solutions, are not perfect solutions. First of all, due to the limitation of computer storage capacity and computing power, it is impossible to increase the block size to 100 times; in addition, although the agent consensus mechanism improves the transaction speed, the relatively small consensus group also means that it is more vulnerable to attacks, and The color of centralization is strong; while off-chain solutions and side chains lose the decentralization and openness and transparency of the blockchain itself to a certain extent.
In this regard, the sharding technology cuts the entire network into different groups and conducts different transactions, making it possible for the entire network to process different transactions in different shards, and finally achieve joint computing and joint settlement of transactions. effect, so as to achieve a high throughput.
As an emerging technology, sharding technology also has pain points and challenges. For sybil attacks (malicious attackers create different malicious nodes, which may be assigned to different shards to facilitate more malicious transactions). Jia Yaoqi proposed that the proof of work can be used to allow the nodes to determine their identities, and at the same time, the network shuffles every once in a while to randomly assign nodes to different shards. When the number of nodes in each shard is not less than 600, each shard The probability that a third of the nodes on a slice are malicious is less than one in a million. In addition, in view of the risk of double-spend attacks caused by parallel computing of different shards, Jia Yaoqi said that her team has developed a new smart contract sharding mechanism, through deterministic allocation, to ensure that different smart contracts can perform calculations in multiple shards at the same time.
secondary title
The following is the transcript of Jia Yaoqi's speech:
Thank you very much for the invitation of Mandy and Odaily. Today I would like to introduce to you that Zilliqa has done research on some methods of blockchain, especially public chain expansion, in the past three years, especially as a pioneer of blockchain sharding technology. Research team, some recent progress in this area.
Everyone's understanding of the blockchain and the public chain may be more through some of its existing characteristics, such as its decentralization, immutability, and its openness and transparency. At the same time, the blockchain, especially the public chain, also has some bottlenecks, such as the low throughput that everyone knows now. Here we use commuter trains as an example and analogy. For example, like the existing public chain, it can achieve 10-20 transactions per second, which is similar to a commuter train, each train can only accommodate 10 people, and others have to queue up.
Compared with traditional payment systems, such as various credit cards, they can reach 8000tps, that is, each train can accommodate 8000 people, so that there will be very few people in line. Some problems caused by the low throughput of the current public chain system are traffic congestion. For example, we now have hundreds of people, millions of people want to take such a train, but each train can only accommodate ten people, what can we do? First of all, increase the fare, for example, from the initial 10 yuan, now we increase it to 1000 yuan, and the 10 people with the highest bid will be able to take the latest train.
At the same time, for the commuter train system, this would create a significant congestion. Even though each train can accommodate 10 people, but there are thousands of people queuing behind it, its throughput will eventually cause such a big problem. We can see that there are many projects and teams, and everyone is exploring how to achieve a public chain platform with high throughput and scalability.
Among the existing solutions, the first is to increase the block size. There are these popular blockchains out there, and some of them have blocks that are 2MB. If the block size is simply doubled or even ten times increased, the corresponding throughput can be increased by two to ten times accordingly. However, this system is more like a bus we take. You can make it a double-decker, and even ten floors can be stacked on top of it, but due to various height restrictions, you cannot build 100 floors on a bus.
This is analogous to the blockchain, because the current network has bandwidth limitations, including the storage capacity of each computer, and the limitation of its computing power, which leads to increasing the block size, which is not a 100-fold solution.
At the same time, some teams and scholars have proposed a proxy consensus mechanism. At present, the existing public chain has 10,000 or 20,000 nodes to run the consensus protocol of the entire system, process transactions and generate new blocks. So can we reduce such a node of 20,000 people to, for example, 1,000, or even 50, or 21 people to vote to form a consensus? The answer is yes, we can reduce such a consensus protocol to only 50 people processing transactions to reach consensus and generate blocks.
From the perspective of computer technology, the fewer the number of nodes running the consensus protocol, the faster the consensus will be formed. When you have a relatively small node group, the block generation speed and the speed of the consensus protocol can reach tens of thousands of transactions per second. However, most of the time we select an agency organization based on the amount of capital, which will lead to the selection of a small group that may not represent the interests of the general public. At the same time, due to the relatively small target of the agent organization, it is more vulnerable to attacks, and once some agents are attacked, the entire blockchain system will be affected.
Another popular expansion solution is the side chain and off-chain solution. At present, the most used scenario of off-chain transactions is small transactions. For example, to buy coffee, users can deposit a deposit on the direct chain, and then buy coffee dozens of times a month at the coffee shop. The need for transactions to be placed on the chain, thereby reducing the dependence on the chain itself. However, such a solution also loses the decentralization and transparency of the public chain itself.
Meanwhile, earlier this year, Vitalik made a post on his Twitter that sharding was coming. As the first team to conduct research on sharding technology, our team started sharding research in 2015, and published related papers at the top computer security conference CCS in 2016. This is also the first academic paper to apply sharding technology to Academic articles on public chain throughput improvement.
As you can see, this is some preliminary experiments done by our team. At 3600 nodes, our system can achieve more than two thousand transactions per second. In simple terms, sharding technology is to divide the entire network. For example, if the entire network has 10,000 nodes, we can divide it into different groups, and each group handles different transactions. The entire network processes different transactions in different shards, and finally achieves the effect of parallel processing transactions, thereby achieving a high throughput.
Fragmentation technology also faces many challenges and problems, such as how to ensure good security, uncompromising decentralization, and efficient execution of smart contracts.
Next, I will briefly share with you how we solve these challenges and problems. The first challenge is how to divide different nodes into different shards. There is a lot of knowledge in this. A malicious attacker can create different malicious nodes and put them in the same shard, thereby facilitating more malicious transactions.
How do we solve such a problem? First of all, I just mentioned such an attack, which is called a sybil attack academically, that is, attackers can create a large number of such malicious nodes to capture your network. To prevent such sybil attacks, our system requires each node to do proof of work. Each node must perform proof of work at regular intervals, such as two hours, to declare its valid identity. Afterwards, the system randomly distributes the nodes into different shards, with each shard having more than 600 nodes. Through calculation, we can prove that the probability that one-third of the nodes in each shard is malicious is one in a million.
After the system handles security sharding, how to place different transactions in different shards is the next question. For example, Alice is an attacker. He can create two transactions to different shards. One transaction is to transfer 5 yuan to Bob, and the other transaction is to transfer the 5 yuan to himself. The two shards process different transactions, but Alice spends her 5 dollars twice, which is what we call a classic double-spend attack.
For the double-spend attack just mentioned, we can prevent the attack by sharding different transfer transactions according to the sender's address. However, there are currently a large number of smart contract transactions on the public chain, and these transactions bring more challenges to sharding. Still the example just now, Alice and Bob, they both use a smart contract for selling movie tickets, Alice buys the last movie ticket from this smart contract in one shard, and Bob also buys the last movie ticket in another shard movie tickets. In the end, they all bought the movie ticket with their own balance, but because there is no communication between different shards, does the last movie ticket belong to Alice or Bob? This is a very serious problem caused by data inconsistency.
For ordinary transfer transactions, only the sender will cause the state of the sender and receiver to change. However, in a smart contract, different senders can cause the state of the smart contract to change, and at the same time, the smart contract can also cause the state of different users to change.
In order to solve the problem of inconsistency in the status of smart contract shards, we can use the two-phase commit protocol to lock the status of related smart contracts in different shards. Alice and Bob buy the last movie ticket in different shards. We can let Alice's transaction run first and lock the state of the smart contract of Bob's shard. However, the lock protocol will prevent smart contracts from running simultaneously in multiple shards. In order to solve such a parallel computing problem, our team has developed another method, which is to solve the smart contract conflict problem just mentioned through deterministic allocation.
We first study a large number of smart contracts, and then classify the transactions of these smart contracts:
1. Ordinary transfer transactions.
2. Simply call the smart contract.
3. It is more complicated, the user can call the smart contract, and the smart contract calls the smart contract again.
Then we assign these three types of transactions to different shards. For ordinary transfer transactions, we can directly determine which shard it belongs to for processing based on the sender's address. For the second category, it can be determined which shard it belongs to based on the address of the sender and the smart contract. We will hand over the special transactions of the third category and the second category to special shards for processing. After theoretical analysis and verification, we can put these three types of transactions in different shards for parallel processing through our algorithm, and at the same time prevent double-spending attacks and the data inconsistency problem just mentioned.
While solving the problem of smart contract sharding, we also found that the smart contract itself has many problems. For example, the smart contracts you saw in the past two years were frozen due to various loopholes. The reason is that the smart contract itself is more complicated and has more unpredictability. Based on our existing research, our team developed its own smart contract language Scilla, which can provide better security and facilitate formal verification. At present, some relatively large advertising companies and software companies have cooperated with us to use Scilla for smart contract development.
At the end of this year, or early next year when the Zilliqa mainnet goes live, you should see that sharding technology has greatly improved the throughput of the public chain.
These are my sharing, thank you all!
