Puzzle Ventures Why is ZKization of Consensus Layer Needed?

Why is ZKization of Consensus Layer Needed?

Original author: Zoe

TL; DR

• Since the competition among various public chains, to Danksharding in Ethereum’s roadmap, to layer 2 solutions like op/zk, we have been continuously discussing the scalability of blockchain – what to do when a large number of users and funds come in? Through a series of articles to follow, I want to show everyone a future scenario, which consists of three parts: data access, off-chain computation, and on-chain verification.

Trustless Data Access + Off-chain Computation + On-chain Verification

• Chainlink releases v0.2 Staking platform and specific details
• Two pictures will help you understand why Filecoin has strong resilience and how to easily resolve the scenario of significant QAP loss.
• Evening Must-read | Security Issues that Airdrop Hunters Need to Be Alert About

• “Proving consensus” is an important part of this blueprint. This article explores the significance of using zero-knowledge proof consensus on Ethereum’s PoS, including: 1. The importance of decentralization for EVM. 2. The importance of decentralized data access for web3 scalability. Proving the full consensus of the Ethereum mainnet is a complex task, but if we can achieve zkization of the consensus layer, it will help Ethereum scale while ensuring security and trust, enhance the robustness of the entire Ethereum ecosystem, reduce participation costs, and allow more people to participate.

1. Why does proving consensus matter?

Using zk to verify the consensus layer of Ethereum L1 makes sense in two main directions. Firstly, it can compensate for the shortcomings of current node diversity and enhance the decentralization and security of Ethereum itself. Secondly, it provides the basis for the availability and security of various layers of protocols in the Ethereum ecosystem, including cross-chain security, trustless data access, decentralized oracles, and scalability.

1. Perspective from Ethereum

For Ethereum to achieve its decentralization and robustness, it needs a diverse environment of clients. This means more people participating, especially ordinary users running clients based on different code environments. However, it is not feasible to require every user to run a full node because it requires a lot of resources, and few people can afford at least 16 GB+ RAM and Fast SSD with 2+TB, and these requirements are still growing.

The current goal is to achieve a lightweight node that provides the same level of trust as a full node (minimizing trust) but with lower costs in terms of memory, storage, and bandwidth. However, currently lightweight nodes do not participate in the consensus process, or only receive partial protection from the consensus mechanism (Sync Committee). This goal is referred to as “The Verge” in Ethereum’s roadmap.

Goal: verifying blocks should be super easy – download N bytes of data, perform a few basic computations, verify a SNARK and you’re done— The Verge on Ethereum’s Roadmap

“The Verge” aims to bridge the client gap, and a key step is how to achieve a trustless lightweight node that is as secure as today’s full node, filling the “client gap” and enabling more people to actively participate in the decentralization and robustness of the network.

https://www.ethernodes.org/network-types

https://clientdiversity.org/

2. Perspective of Protocol Stacks on Ethereum

From first principles, we need to solve the problem of combining on-chain data access with off-chain computation verification. Currently, the use of on-chain data is relatively primitive and insufficient. In many cases, the data required for protocol adjustments is too complex to be computed on-chain, and the cost of obtaining data through trustless means is high, requiring a large amount of historical data access and frequent digital computation. For individual users and projects, our ideal situation is to achieve decentralized, end-to-end trustless data transmission and read/write, based on which, facing more users in the future, we should achieve as low computational costs as possible, while considering security, usability, and economy. Specifically, the following aspects should be considered:

1. Decentralized and trustless oracles: Currently, centralized oracles are used in protocols to avoid direct access to a large amount of historical data on-chain, which increases unnecessary trust costs and reduces composability.

2. Data read/write for data-sensitive and asset-related protocols: For example, DeFi protocols need to make some parameter adjustments during operation, but whether it is possible to access historical data and perform more complex calculations without trust, such as adjusting AMM fees based on recent market fluctuations, designing on-chain derivative trading price models and dynamic fluctuations, introducing machine learning methods for asset management, and adjusting borrowing interest rates based on market conditions.

3. Cross-chain security: Currently, zk-based light node solutions are superior in terms of security, capital efficiency, statefulness, and diversity of information transmission. The current Succinct TeleLianGuaithy cross-chain solution and the Polehedra cross-chain solution on LayerZero are both based on Sync Committee for lightweight node block header zk verification. However, Sync Committee is not the Ethereum PoS consensus layer itself and has certain trust assumptions, leaving room for further improvement in the future. Currently, due to economic costs, technical limitations, and user experience considerations, developers usually rely on centralized RPC servers such as Alchemy, Infura, and Ankr when using on-chain data.

II. Where is Blockchain Data? Trust Assumptions for Different Data Sources

There are two sources of computational data in blockchain: on-chain data and off-chain data. Calculation is performed based on these two types of data, corresponding to on-chain and off-chain directions. For example, the need to adjust DeFi protocol parameters mentioned earlier.

Data Access, computation, proof and verification

There are two notable characteristics of on-chain and off-chain data reading, writing, and computation:

1. In order to achieve decentralization and security, it is best to verify the data we obtain, that is, “Don’t Trust, Verify”.

2. Often involves many complex and expensive computational processes.

If no suitable technical solution is found, the above two points will affect the usability of the blockchain.

We can illustrate different ways of obtaining data through a simple example. Suppose you want to check your account balance, what would you do?

One of the safest ways is to run a full node yourself, check the Ethereum state stored locally, and obtain the account balance from it.

Full Node Benchmark. Sync mode and client selection will affect the required space. Reference: https://ethereum.org/en/developers/docs/nodes-and-clients/run-a-node/; https://docs.google.com/presentation/d/1ZxEp6Go5XqTZxQFYTYYnzyd97JKbcXlA6O2s4RI9jr4/mobilepresent?pli=1&slide=id.g252bbdac496_0_109)

However, the cost of running a full node yourself is high and requires self-maintenance. To save trouble, many people may directly request data from centralized node operators. Although there is nothing wrong with doing so, similar to operations in Web2, and we have never seen these suppliers engage in any malicious behavior, but this also means that we have to trust a centralized service provider, which increases the overall security assumption.

To solve this problem, we can consider two solutions: reduce the cost of running nodes and find a way to verify the credibility of third-party data.

So it’s better to store only the necessary data. In order to access data more efficiently, reduce the cost of trust, and independently verify data, some institutions have developed light clients, such as Rust-based Helio (developed by a16z), Lodestar, Nimbus, and JavaScript-based Kevlar, etc. Light clients do not store all block data, but only download and store block headers – a “summary” of all information in a block. Light clients can independently verify the received data, so when you get data from a third-party data provider, you no longer need to fully trust the data provided by that provider.

https://medium.com/coinmonks/ethereum-data-transaction-trie-simplified-795483ff3929

The main characteristics of light nodes include:

• Ideally, light nodes can run on mobile phones or embedded devices.

• Ideally, they can have the same functionality and security guarantees as full nodes.

• However, light nodes do not participate in the consensus process, or they are only protected by part of the consensus mechanism, namely the Sync Committee.

The Sync Committee is the trust assumption of light nodes. Before The Merge, starting from December 2020, the Beacon Chain underwent a hard fork called Altair, which aims to provide consensus support for light nodes. Unlike PoS full consensus, the group of validators (512) that make up this group is a smaller dataset, and they are randomly selected at longer intervals (256 epochs, about 27 hours).

Light clients such as Helios and Succinct are taking steps toward solving the problem, but a light client is far from a fully verifying node: a light client merely verifies the signatures of a random subset of validators called the sync committee, and does not verify that the chain actually follows the protocol rules. To bring us to a world where users can actually verify that the chain follows the rules, we would have to do something different. How will Ethereum’s multi-client philosophy interact with ZK-EVMs?, by Vitalik Buterin*

This is why we need to verify Ethereum’s entire consensus layer in order to usher in a more secure, more usable, more diverse protocol and a future that is widely adopted. Zero-knowledge (zero-knowledge) technology is currently the best solution in this regard.

III. The LianGuaith to Prove Consensus Using ZK

To build an environment without the need for trust assumptions, we must address the issues of light client credibility, decentralized data access, and off-chain computation verification. Zero-knowledge proofs are currently the most recognized core technology in these areas, involving but not limited to underlying solutions such as zkEVM, zkWASM, other zkVMs, and zk Co-processors. Proving the consensus layer is an important part of this. The PoS algorithm is very complex, and implementing them in a ZK way requires a lot of engineering work and architectural considerations. Let’s break down its components first.

1. Key Steps in Consensus Formation in Ethereum 2.0

(1) Validator-related algorithms

These include the following steps:

• Becoming a validator: Validator candidates need to send 32ETH to the deposit contract and wait for at least 16 hours to several days or weeks for the Beacon Chain to process and activate them as formal validators. (Refer to FAQ – Why does it take so long for a validator to be activated)

• Performing validation duties: Involves random number and block proof algorithms.

• Exiting the validator role: The ways to exit as a validator can be voluntary or due to punishment (slashing). Validators can initiate “exit” at any time, and there are limitations on the number of validators exiting each epoch. If there are too many validators trying to exit at the same time, they will be placed in a queue and still need to fulfill their validation duties before being processed. After a successful exit, validators will be able to withdraw their staking funds after 1/8 of an eek.

(2) Random number-related algorithms

• Each epoch consists of 32 slots, and 2 epochs in advance are used for random grouping, dividing all validators into 32 committees. These committees are responsible for consensus on each block in the current epoch.

• There are two roles in each committee: a proposer and builders, randomly selected. This separates the processes of transaction ordering and block construction (see proposer/builder seLianGuairation – PBS for details).

(3) Block attestation and BLS signature-related algorithms

• The signature part is the most critical part of the consensus layer.

• Each slot’s validation committee votes (using BLS signatures) and requires a 2/3 majority to build a block.

• In the Ethereum PoS consensus layer, BLS signatures use the BLS12-381 elliptic curve, which is LianGuaiiring-friendly and suitable for aggregating all signatures, reducing proof time and size.

• In proof of work, blocks may be reorganized (re-org). After the merge, the concept of “finalized blocks and safe head” is introduced on the execution layer. To create a conflicting block, attackers need to destroy at least 1/3 of the total staked Ether, making PoS more reliable than PoW to a large extent.

https://blog.ethereum.org/2021/11/29/how-the-merge-imLianGuaicts-app-layer

By the end of June 2023, “Puzzle Ventures Evening Self-Study” introduced Hyper Oracle’s zkPoS (using zk methods to verify the Ethereum consensus layer). For details, please refer to zkPoS: End-to-End Trustless

(4) Others: such as weak subjectivity checkpoints

One of the challenges faced by trustless PoS consensus proofs is the selection of weak subjectivity checkpoints, which involves social consensus based on social information. These checkpoints are revert limits because blocks before weak subjectivity checkpoints cannot be changed. See: https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/weak-subjectivity/

Checkpoints are also a point to consider in the zkification of the consensus layer.

2. Proving Consensus Layer ZK Tech Stacks | Tech Stacks to Prove Consensus

In proving the consensus layer, proving signatures or other computations themselves are very expensive, but in contrast, verifying zero-knowledge proofs is very cheap. When choosing the method of using zero-knowledge proof consensus layer, the protocol needs to consider the following factors:

• What do you want to prove?

• What are the application scenarios after the proof?

• How to improve the efficiency of the proof?

Take Hyper Oracle as an example. For proving BLS signatures, they chose Halo2 instead of Circom used by Succinct Labs for the following reasons:

• Both Circom and Halo2 can generate zero-knowledge proofs of BLS signatures (BLS12-381 elliptic curve).

• Hyper Oracle is not just doing zkPoS. Its core product is a programmable on-chain zero-knowledge oracle (Programmable Onchain zkOracle). Among them, zkGraph, zkIndexing, and zkAutomation are directly facing users, and zkWASM virtual machine is used to verify off-chain calculations. Although Circom is easier for engineers to get started, it has poor compatibility and cannot ensure that all the logic of the functions can be used.

• Circom-LianGuaiiring will be compiled into R1CS, which is incompatible with zkWASM and the Plonkish constraint system of other circuits, while the Halo2 LianGuaiiring circuit can be easily integrated into the zkWASM circuit. In comparison, R1CS is not ideal for batch proofing (Proof Batching).

• From an efficiency perspective, the BLS circuit generated by Halo2-LianGuaiiring is smaller, the proof time is shorter, the hardware requirements are lower, and the gas fee is lower.

https://mirror.xyz/hyperoracleblog.eth/lAE9erAz5eIlQZ346PG6tfh7Q6xy59bmA_kFNr-l6dE

Another key point in using zero-knowledge proof to prove the consensus layer is recursive proof, which means proofs of proofs, packaging previous events into a proof. Without recursive proof, the proof size would eventually be O(block height), which includes each block attestation and its corresponding zkp. Through recursive proof, apart from the initial state and the final state, we only need a proof size of O(1) for any number of blocks.

Verify Proof N and Step N+1 to get Proof N+1, i.e. you know N+1 pieces of knowledge, instead of verifying all N steps sequentially. Returning to the initial goal, our solution should target “light clients” with computational and memory constraints. Even if each proof can be verified in a fixed time, the verification time will become very long as the number of blocks and proofs accumulates.

3. The End Game: Diversified Level 1 zkEVM

Ethereum’s goal is not only to prove the consensus layer, but also to achieve the zero-knowledgeification of the entire Layer 1 virtual machine through zkEVM, and ultimately achieve diversified zkEVM to enhance the decentralization and robustness of Ethereum. The current solutions and roadmap for Ethereum regarding these issues are as follows: “lightweight” – smaller memory, storage, and bandwidth requirements

• Currently, it achieves storing and verifying only the block headers through light nodes.

• Future development also requires further efforts in verkle tree and stateless clients, involving improvements to the mainnet data structure.

“trustless security” – achieving minimum trust equivalent to full nodes

• Basic light node consensus layer has been implemented, namely Sync Committees, but this is only a transitional solution.

• Use SNARK to verify Ethereum Layer 1, including verifying Verkle Proof of the execution layer, verifying the consensus layer, and SNARKifying the entire virtual machine.

• Level 1 zkEVM is used to achieve the zero-knowledgeification of the entire Ethereum Layer 1 virtual machine and to achieve diversified zkEVM.

Possible Risks

In an ideal situation, when entering the era of zk, we need multiple open-source zkEVMs – different clients have different zkEVM implementations, and each client waits for a proof compatible with its own implementation before accepting a block. However, multiple proof systems may face some problems because each proof system requires a peer-to-peer network, and a client that only supports a certain proof system can only wait for the corresponding type of proof to be recognized by its verifier. Two main challenges that may arise include “latency challenge” and “data inefficiency”. The former is mainly due to the slow generation of proofs, allowing malicious actors to create temporary forks during the time difference when generating proofs for different proof systems. The latter is because you need to generate zk proofs of multiple types, which requires preserving the original signatures. Although theoretically the advantage of zkSNARK itself is the ability to remove original signatures and other data, there are some contradictions here that need to be optimized and resolved.

4. Future Outlook | What is the Future?

In order to bring more users to web3, provide a smoother experience, create higher usability, and ensure the security of applications, we must build infrastructure for decentralized data access, off-chain computing, and on-chain verification.

The proof of consensus layer is one important component. In addition to Ethereum PSE and the zkEVM layer2 mentioned earlier, there are also protocols that are using zero-knowledge proof consensus to achieve their own application goals. This includes the Hyper Oracle (Programmable zkOracle Network), which plans to use zero-knowledge proof to obtain data from the entire consensus layer of Ethereum PoS; Succinct Labs’ TeleLianGuaithy, which is a light node bridge that achieves cross-chain communication by verifying the Sync Committee consensus and submitting state validity proofs; and Polyhedra, which was originally a light node bridge but now also claims to use devirgo to achieve full node full consensus zk proof. In addition to cross-chain security and decentralized oracles, this combination of off-chain computing and on-chain verification may also be involved in fraud proof in optimistic rollups, integrating with OP L2; or in intent-based architecture, providing on-chain proofs for more complex intent structures, and so on. Here we are talking not only about the off-chain ecosystem surrounding Ethereum, but also the broader market beyond Ethereum.

There are still many aspects of this topic that are worth further research. For example, on August 24th last week, a16z published an article arguing that “stateless blockchains cannot be achieved”. Another example is the question of whether weak subjectivity checkpoints and the security of the Sync Committee are mathematically sufficient. Interested colleagues are welcome to contact ([email protected]) to continue discussing this topic with the author.

Once again, thank you to my colleagues for their guidance and feedback, Alex @ IOBC (@looksrare_eth), Fan Zhang @ Yale University (@0xFanZhang), Roy @ Aki Protocol (@aki_protocol), Zhixiong LianGuain @ LianGuai (@nake13), Suning Yao @ Hyper Oracle (@msfew_eth), Qi Zhou @ EthStorage (@qc_qizhou), Sinka @ Delphinus (@DelphinusLab), Shumo @ Manta (@shumochu)

We will continue to update Blocking; if you have any questions or suggestions, please contact us!