Written in front: This article consists of 15 sub-articles written by 20 professional Ethereum developers. They are divided into different clients, Ewasm, formal verification, Remix, Plasma, ETH 2.0, Solidity, status channel, ZoKrates and other subdivisions. The research and development work was summarized and the original text was published on the Ethereum Foundation (EF) official website .
Here is the translation:
- Beijing's "regulation sandbox" experiment is about to begin, and the road to blockchain compliance begins?
- Circle transformation: from Goldman Sachs, Baidu sought after, to give up bitcoin payment and change to stable currency
- Filecoin mainnet test, "first move" miners may be eliminated
- QKL123 Quote Analysis | Short-term disk is constantly being tested; how will Bitcoin evolve in the future? (1113)
- Bakkt was confirmed in the third quarter, and the super bitcoin bull market is really coming?
- Why do you say Bitcoin prices never look back?
(Figure: Ethereum devcon conference)
Since the last report, Ethereum's research and development has made comprehensive progress. From improving network conditions to the subsequent Istanbul hard fork release and the development of Eth1.x and Eth2.0, all core areas of Ethereum functionality and sustainability are progressing.
The focus of this series of articles is on teams and attempts covering the Ethereum Foundation and the wider Ethereum ecosystem, which are working to improve Ethereum as a whole. In this article, we will introduce the latest situation of many teams highlighted in the previous report, and the core elements of the Ethereum ecosystem, such as Eth2.0 research, Geth, Solidity, and other ecosystem work.
I. Aleth / C ++ Ethereum
- EVM & other consensus
- Testeth tools
2. Ecosystem Support Program
- Taiwan Special Funding
- Growing non-financial support
- Website improvements
- Execution Environment (EE)
- Fast Cryptography
- Speed, Metering, Size
Fourth, formal verification
Seven, the Python ecosystem [PyEVM / Trinity / Web3.py / Vyper]
Nine, research [CBC]
十 、 Research [Plasma]
XI. Research [Serenity / Eth2]
Twelve, security [security / consensus test]
- Solidity 0.6.0
- SMT Checker
- Yul Optimizer
14. Research on State Channels
- What's the next step for the status channel?
I. Aleth / C ++ Ethereum
Author: Andrei Maiboroda
The Aleth team is one of the client teams that are following up the Istanbul upgrade and Eth1.x chain development. Aleth 1.7.2 client fully supports Istanbul upgrade.
1.1 EVM & other consensus
Important milestones of the evmone project:
After the initial version of this experimental fast EVM project was released, we focused on squeezing more performance out of it. The 0.2.0 version is 66% faster than the previous version. Subsequently, version 0.3.0 made evmone compatible with the Istanbul upgrade specification. For those interested in exploring concepts with evmone trials and more information on optimizing EVM, please refer to the slides on optimization techniques for EVM implementation provided in the Devcon5 conference presentation and articles on EVM's efficient gas calculation algorithms .
The EVMC project (a cross-language API implemented by EVM) received the required updates with the support of Istanbul. They are all packaged into EVMC 7.0.0 .
In the aleth interpreter, developers have also made some optimizations to eliminate unnecessary state access in certain opcode implementations. These invalid accesses became apparent due to the EVM test suite provided by the evmone project .
The aleth interpreter also switched from using boost :: multiprecision to the intx library . This is a step to release the aleth interpreter without relying on boost, but it also allows for some interesting benchmarks to see how inefficient a 256-bit integer boost implementation is for our needs.
After the Istanbul upgrade was completed, we selected several EIPs for future hard fork Berlin: EIP-1380 will reduce the gas cost of self-invocation, and EIP-2046 will reduce the gas cost of precompiled static calls. These implementations can be activated in the testeth tool, see below:
1, 2 Networking
We have implemented an optimization that has been adopted by other mainnet clients: broadcasting new blocks to peers immediately after the PoW check, rather than waiting for full verification and execution. We also put a cap on the number of previously discarded transactions remembered by the transaction pool (now 1024 such transactions).
The client version reported during devp2p handshake has been fixed and allows the version of Aleth to be displayed correctly on ethernodes.
1, 3 RPC
In order to better conform to the input / output format used in other client RPC interfaces, we made a lot of modifications. As the problem of unnecessary block transaction re-execution is solved, many methods have achieved significant performance improvements. This may be noticed in many use cases of frequent RPC requests, such as when using retesteth in aleth .
1, 4 Database
Rebuilding the index from the existing block database has been completed and optimized, which will allow us to optimize and modify the layout of the index database in the future.
1, 5 Testeth tools
The test team reorganized the consensus test folder structure. Now the Testeth tool has supported it. All the tests involving fork rules before the Istanbul upgrade are in the LegacyTests suite.
You can now generate a status test and run it with a string like "fork name + EIP_number" instead of the regular fork name in the expect section. This allows anyone who plans to create new EIP prototypes in aleth to generate tests for them before they fork to accept EIP (this is the basic idea of the EIP-centric fork process , currently used by all core dev communities). As an example of the mechanism, the two new EIPs mentioned above (EIP-2046 and EIP-1380) can be activated in testeth, and we created two state tests to illustrate this feature.
We also fixed and optimized the testeth feature to run any custom test files (beyond the predefined consensus test structure) and make its output more consistent with go-ethereum's evm tool. This allowed us to integrate testeth into the goevmlab project, and now aleth's EVM implementation participates in cross-fuzzing with the EVMs of the other three major clients.
- Provides a better build process for those who build aleth from scratch;
- Improvements to docker images;
Ecological Support Project
Author: ESP Team
2.1 Taiwan Special Funding
Recently, we issued five grants of $ 2,000-5,000 to the Crosslink Taipei project, these are one of a series of grant programs designed to recognize the contributions of communities around the world.
2.2 Growing non-financial support
We are continuing to expand our definition of “support” for projects where regular funding is not appropriate. Some of the non-financial support we provide includes feedback from expert consultants, connecting teams that are doing similar work, AWS credit, invitations to events, and more.
(Note: meaningless, omitted and translated)
Author: Alex Beregssaszi and Paul Dworzanski
Since the last update, the focus of the Ewasm team has shifted to Eth 2.0 research and working closely with other teams.
At present, Eth 2.0's phase 2 (phase 2) execution layer development work is actively proceeding, and it is parallel to the development work of phase 0 (phase 0) and phase 1 (phase 1). Regarding Phase 2, the developer has made a number of proposals, and the Ewasm team has been working to build a prototype and benchmark design on the basis of Scout.
3, 1 Scout
The Scout specification is the minimal interface to the execution environment (EE). This minimal interface is sufficient for prototyping stateless EE, which is required to validate stateless models and can provide information for Ewasm and Phase 2 design.
There are three ways to implement Scout:
- Scout written in Rust, designed for rapid prototyping and collaboration (it uses a Wasm interpreter that supports analysis);
- Scout.ts written in Typescript for rapid prototyping and browser support;
- ScoutOne written in C ++, designed for performance and production use, can be embedded in Eth 2.0 clients;
3.2 Execution Environment (EE)
Unlike the Eth 1.0 stateful model, Eth 2.0 is stateless because the state is stored off-chain, and only the hash representing the state is stored on the chain, and the witnesses are passed as part of the transaction.
Stateless models present new challenges, which require prototypes and measurements to verify its feasibility.
The Ewasm team has made great efforts in prototyping and measuring stateless EE, and we classify it as:
1. Execution environment (EE) that must be compatible with Eth 1 data structures and execution ;
- SMPT (Stateless Merkel Patricia Tree) uses RLP serialization to verify the content and transaction data, and uses the Eth1 signature scheme.
- EVM implemented in Assemblyscript;
- biturbo (formerly known as TurboToken), uses Multiproof to encode verification data more efficiently, and also supports EVM execution;
2. No need for backward compatible design ;
- KMM (Katajainen Makinen Merkle) Token EE, optimized for verification content size and execution time;
- Groth16 verification program implementation for zk-SNARKs supporting Eth 2.0;
- STARK verification program implementation;
Of note is the active study of the interaction between Eth 1 and Eth 2 chains. To help evaluate the "handover" scheme, where Eth 1 is moved to the execution environment (EE) on Eth 2, the above Eth 1 EE will be prototyped. The team is also actively evaluating the options for connecting the two networks and their impact on EE design.
Our ultimate goal is to provide a good development experience for existing and new DApps.
This EE work has been fed back into the design of Scout and Eth 2.0.
3, 3 Fast Cryptography
For the Ewasm virtual machine to succeed, we must execute expensive crypto on the chain. Fortunately, crypto often has runtime bottlenecks in the bigint algorithm. First, we benchmarked various implementations of cryptographic primitives to identify bottlenecks. We then designed a fast native bigint API to address these bottlenecks. Finally, in collaboration with its creator, we extended the highly optimized websnark library to call this bigint API.
The results are encouraging: by implementing this bigint API in the interpreter, we are close to native speed in elliptic curve operations! We can now perform pairing close to the native speed, which is the most recent success in Ewasm's R & D efforts.
This work allowed the aforementioned EE prototype to run under the performance constraints of Eth 2.
3, 4 Speed, Metering, Size
Ewasm has many other items related to speed, metering (reducing metering runtime overhead and accurately approaching runtime), and bytecode size. From Wasm engine optimization to econometric analysis to bytecode conversion, the Ewasm team is working to design the best execution system possible.
We have been working for Ewasm's tools.
The combination of Eth 2 and the bigint API is provided for Assemblyscript and Rust .
If Wasm bytecode needs to be extended, develop an extensible tool called chisel to provide various conversions (such as bytecode size reduction and import / export adjustments) required by Ewasm and non-Ewasm applications.
Fourth, formal verification
Author: Leo Alt
The new formal verification team is developing relevant tools, providing formal models and proofs to other teams of the foundation, and working with members of the Ethereum FV community.
Recent work focuses on:
- Solidity's SMTChecker, which is an infinite model checker for Solidity smart contracts;
- KYul, Yul semantics are further used in K.KYul to support Solidity compilers by calculating non-optimized and optimized Yul code's bisimulation proofs;
- With the support of members of the FV community, lead the development of smart contract specification language. The canonical language is used to describe contract attributes, and is intended to be simplified and supported by many FV tools
- Support Eth2 research team and Runtime verification in Beacon Chain verification work;
- (Together with Dapphub) maintain HEVM, which is a fully compatible Haskell EVM implementation;
- Support test teams to use the differences found in different EVM implementations to extend the coverage of Ethereum testing;
Author: Péter Szilágyi
After the Geth team released the v1.9.0 client in July this year, we have been busy iterating the existing code base, fixing any issues found, and preparing the client for the Istanbul hard fork . In addition to these maintenance changes (a total of 8 versions), the Geth team has laid the groundwork for some new things in the future.
We have built a treasure trove of 5 EIPs around the ENR (Ethereum Node Record) for network and peer discovery.
The fork id work has been deployed on top of the Ethereum protocol, allowing Geth nodes to cleanly divide the network between incompatible machines. These records are also indexed by a new discovery service exposed via DNS (not yet finalized), which makes eclipse attacks more difficult and will allow Ethereum to be locked in UDP environments (DNS via HTTPS) Running.
In terms of performance, we are working on multiple aspects. On the spectrum side, we are trying to reduce the network load by making transactions (and possibly even blocks) more intelligent. On the other hand, we are working on a state catcher that can track active chains and serve as an acceleration structure for EVM execution and state synchronization. In the meantime, we are developing various configuration options for Geth that allow users to discard parts of the database that are not useful to them (without affecting network health), thereby saving valuable SSD capacity. These are promising ways forward, and we will share some numbers in the near future.
We have also invested a lot of work on lightweight client infrastructure, allowing server operators to assign and manage client priorities and resource permissions through RPC APIs. Although in the long run, light client incentives are planned to work through a point-to-point protocol, the current feature set has allowed operators to collect payments outside Geth and synchronize with Geth's internal accounting. This will immediately allow anyone to create a paid light server service (note that it is still in the prototype stage). Currently, we are doing development work for a full P2P payment layer.
Author: Samuel Furter, Holger Drewes, Marc Garreau, Everton Fraga and Richard Moore
- Web3.js ;
- Ethers.js ;
- EthereumJS ;
- Grid ;
However, current projects are not forgotten. We know very well that we must care about and further develop tools that are widely used within the community. Therefore, the following is an update on each subproject to reveal what happened to these projects in the last quarter.
Since the last EF blog post, we have released multiple Web3.js patches and switched to semantic versioning. These patches add support for TypeScript, extend transaction signature functionality, add transaction confirmation workflow attributes, add JSON-RPC method
getChainId , add
connected events to subscriptions, use the methods
supportsSubscription and other utility functions to extend the provider interface, To use bloom filters.
More details about the new features and improvements can be seen in the release announcement on GitHub .
Currently, our focus is to reduce the size of bundles, improve performance, add reconnections to
WebsocketProvider , and improve TypeScript DX.
In addition to these improvements, we are fortunate to see Chris join the EF-JS team. Chris is currently supporting Web3.js development, and of course he will also participate in the development of all other EF-JS packages.
6, 2 Ethers.js
We are ready for v5 for public use, and adoption has been steadily increasing. Thanks to everyone who tried it and reported related issues.
The focus of v5 is to add extensibility and improve APIs for framework developers, including a new framework ethers-app, with a focus on dapp developers.
As the number of new issues has decreased, we expect v5 to be available soon, with only a few minor changes in the pipeline.
6, 3 EthereumJS
In terms of EthereumJS , the most noteworthy is the release of different components for the Istanbul upgrade: the VM received a larger v4.1.0 update in September, and we are currently removing the final bug to make the VM fully comply with the official test suite. Other updates to mention in the Istanbul upgrade environment are related to transactions, blocks, and public (hard fork and network logic) libraries.
6, 4 Grid
Developers have made several major upgrades to the Ethereum Grid since the last EF blog post update. The application is now in your operating system taskbar and provides a simple user interface to download, configure, and run Ethereum clients and tools. The plug-in system will continue to improve with each new integration, but equally exciting is the Grid application. The application can be used to test RPC methods, query block data through Geth's GraphQL, sign transactions with Clef, and more. The team has been busy writing tutorials on what to do with Grid, and you can see them on Medium.
Seven, the Python ecosystem (PyEVM / Trinity / Web3.py / Vyper)
Author: Piper Merriam
Recent work has been to improve stability and documentation. The current development focus is on adding asynchronous support for the library.
7, 2 Trinity
Developers are developing a beta version of the Trinity client, which includes the newly developed "Beam Sync" (https://firstname.lastname@example.org/intro-to-Beam-Sync-a0fd168be14a). We also focus on working with the wider client team ecosystem to solve some of the larger issues (such as state explosions) and find the migration path from Eth 1.x to the 2.0 world.
7, 3 EthPM
We continue to focus on ecosystem tools.
ethpm-cli development continues, allowing installation of packages from different sources, as well as building and publishing packages.
Author: Yann Levreau
Regarding Remix, we have a lot of updates to share with you. In recent months, our team has been working:
- Improve the Remix plugin and work with the community in various ways-@GrandSchtroumpf;
- Implement WebSocket plugin for Edi Sinovcic's github integration;
- Help Quorum integrate their Remix plugin;
- Work with Waffle (Ethworks) to develop their plugins;
- Work with the VSCode Ethereum team to integrate the plugin engine as a VSCode extension;
- Make loading plugin resources completely decentralized (currently using IPFS technology);
- Remix library: @Aniket has recently joined the team and is responsible for improving, maintaining and promoting the remix repository https://github.com/ethereum/Remix. This includes remix debugging (transaction debugging), remix testing, remix solidity, and key work in the remix analyzer (new module added: ETH incoming loop warning);
- Remix library: complete the Remix simulator and add more tests to Remix debug @ iurimatias;
- Improve IDE's file explorer to support folders and standard features. Work with Ethworks to develop new themes. Use the Monaco editor. Set compiler version based on Solidity pragma-@Lianahus @ryestew @Aniket @GrandSchtroumpf;
- We introduced the Remix desktop version here https://medium.com/remix-ide/remix-desktop-8c1e9e946ee1, and now we are working on it. Note that it will be officially released soon! -@ yann300 @lianahus
- Remix Workshop is a plugin that runs in the Remix plugin API;
It allows to create tutorials, register for tutorials, and allows students and learners to actually run the tutorials. The range of tutorials being produced is very broad (such as Solidity, Vyper, common Ethereum concepts, etc.). It depends on the creator of the tutorial! The first POC was successful, and we are now moving towards releasing the first version with the help of different teams in the community for suggestions and feedback. -@ granchtroumpf @ ryestew
- Seminars and outreach activities outside the community: On the one hand, we also contribute to education in various ways. We expect this to become even more important by 2020. -@ryestew @Aniket @GrandSchtroumpf @team
- Organize seminars / conferences (ESE, Consensys, Devcon, Dappcon);
- Meet with organizations and people outside the community. It is not necessarily technical, but it is more to let people know what is blockchain and what is Ethereum;
- Build a tool for it (see Remix workshop)
Author: Aditya Asgaonkar
The recent work of the Casper CBC R & D team focuses on:
- Describe the minimal CBC Casper protocol and the activity verification strategy in the unified framework (named VLSM). Related public documents WIP will be released soon;
- Formal verification of VLSM;
- Use CBC Casper elements to improve Eth2.0 design, such as https://ethresear.ch/t/cross-shard-messaging-system/6201;
- Community promotion work, such as this AMA: https://www.reddit.com/r/ethereum/comments/dsiz9j/ama_we_are_the_cbc_casper_research_team/
Stay tuned for new progress!
X. Research (Plasma)
Author: Plasma Group
Since May, we have been working hard to advance the advancement of scalable technology. There are 4 different teams in the ecosystem building common Plasma specifications for multiple blockchains, including OmiseGO, Matic, Cryptoeconomics Lab, and Plasm. Since many of our peers are working hard in the payment direction, we have begun to work in areas with less scalability research, such as application development and general composability. Developments in this area include the Optimistic virtual machine (a universal controversial language for layer2 construction) and our Uniswap Devcon 5 demo (a scalable payment and transaction game that can be found on the Unipig exchange ). The game is built on an optimistic rollup chain, a design that was created during a conversation with Barry Whitehat and Vitalik at an Ethereum expansion conference in early June.
At the end of the year, we are preparing to launch a dissertation for OVM and write more in-depth documentation for Optimistic Rollup. Currently, those interested in learning about it can find a description of it on our blog, where we describe how to build it for arbitrary smart contracts, and some early documentation outlining plasma to optimistic rollup:
Our forum: https://plasma.build/t/rollup-plasma-for-mass-exits-complex-disputes/90/1
Plasma conference video: https://youtu.be/5RpYoU6xD_M?t=1136
Eleven (Serenity / Eth2)
Author: EF team
After Devcon5, Danny and the Eth2 research team started a weekly Eth2 quick update series, and recently started a series of work focusing on Eth2 verification. As we are nearing the launch stage 0, there will be more latest news and progress, such as the link below, please also keep an eye on the EF blog!
In general, we are moving forward towards the stage 0 testnet and mainnet launch. The specification and prototype work of the first phase is parallel, while the research and development of the second phase is also actively underway.
Verification: Staking for Ethereum 2.0 2019-11-27
Eth2 Quick Update # 4 -2019-11-21
Eth2 Quick Update # 3 -2019-11-08
Eth2 Quick Update # 2 -2019-10-31
Eth2 Quick Update # 1 -2019-10-23
Twelve security (security / consensus testing)
Author: Martin Holst Swende
In terms of security, much has been done about the Istanbul hard fork. For example, the old python-based fuzzer (Evmlab) has been rewritten in Go and used to create EIP target fuzzers. These fuzzers are used to generate test applications (discovery of implementation flaws in Besu and Nethermind) and to run millions of test cases comparing Geth and Parity-as of the end of November, we also have Aleth and Nethermind VM on the same fuzzer framework run. So we now have up to four EVMs doing differential fuzzing.
At the same time, we also run libfuzzer-based fuzzers on Geth and Parity, which is led by @cryptomental.
Not long ago, we announced on the reward page that we had allocated $ 15,000 for EIP's security audit. Among them, we awarded $ 5,000 to Neville Gretch contract-library.com and Hubert Ritzdorf (Chainsecurity), respectively, for their assessment of the security impact of EIP-1884.
Several other bounties have also been awarded, most of which will be shared publicly soon.
Author: Christian Reitwiessner
The Solidity language and compiler continue to be stable, and some features requested by the community have been added. This includes the option to output Solidity code for a variety of flexible formats, stability and security changes. The team is working on a new 0.6.0 release and a 0.5.x branch update.
Solidity supports experimental preview output of webassembly code using the -ewasm switch. We have extended most of the phases of the Yul optimizer to handle eWasm code. We are currently researching the conversion of EVM-type Yul to eWasm-type glue code and have a working prototype for generating eWasm binary code required for deployment contracts.
13, 2 Solidity 0.6.0
We are almost done with groundbreaking changes and we expect to release Solidity 0.6.0 later this year. Some new changes include:
- Requires explicit "virtual" and "override" keywords to override functions;
- ABIEncoderV2 is no longer experimental;
- The rollback function is divided into an "accept Ethereum" function and an actual "rollback" function;
- Abstract contracts must be marked with "abstract";
- Structs and enums can be defined at the file level;
- It is not allowed to arbitrarily set the length of the storage array;
- Support push () to add new default initialization elements to the dynamic storage array;
- Add a "leave" statement to the Yul / Inline assembly (Assembly) to return from the current function;
- Support multiple return values in NatSpec;
- Support better command line error message format;
- Metadata hash now defaults to IPFS and can be switched to Swarm or deleted;
- Allow removing "revert reason strings" from binary files;
13, 3 SMT Checker
SMT Checker has a new model checking engine that supports loops and allows checking assertions when considering an unlimited number of transactions. Read more about the changes here: https://medium.com/@leonardoalt/smtchecker-toward-completeness-1a99c02e0133;
We are currently working on supporting function calls in the new engine, which will enable multi-contract analysis, even if the calling code is unknown.
13.4 Yul Optimizer
The Yul optimizer can now consider the side-effects of user-defined functions to optimize between these function calls. It can remove redundant sload and mload calls, and can take into account conditional local values of variables.
- If using standard json, the compiler generates bytecode only for the selected contract, and if no bytecode is requested, it stops after doing a syntax analysis.
- The option –error recovery can be used to recover from most parser errors, so you can create AST-like things for invalid inputs;
In addition to the changes listed above, we have implemented many minor bug fixes and features.
14. State Channels (Research)
Author: Liam Horne
In the past few months, the state channel R & D team of Ethereum has achieved pleasing results.
The most exciting thing is that the status channel is online on the Ethereum mainnet. Connext is a micropayment service company built on our work and has been in use since September 2019. The scalability and UX enhancements brought by state channels are no longer theoretical, they are now benefiting users. Maybe try it !
Behind the scenes, R & D personnel have been busy for the past 6 months. This summer, two major state channel research groups, Counterfactual and Magmo, unified their work into a single project and agreement and referred to them simply as "StateChannels". This unification allows us to move faster, and also provides an easier experience for Ethereum application developers, who don't need to think about which channel standard they support.
More specifically, in the past few months, we have done the following:
- Merged CF and Magmo code bases ;
- Implemented a client API record here , and documented the protocol here ;
- Rewritten our Solidity contract code library to save a lot of gas for "happy case" (612K → 165K) and "challenge case" (1100K → 200K);
- The ForceMove protocol is written in a formal specification language called TLA +. This allows us to find many interesting protocol optimizations and identify non-active attack vectors. You can read more on our research forum .
14. What is the next step for the status channel?
We are developing two demo applications that are built entirely on the client API and run in the browser through our reference hub.
Launch new projects, recruit new contributors, and continue to make the status channel developer-friendly.
Author: Jacob Eberhardt
We are happy to share with you the latest developments in ZoKrates, which aims to be a powerful and user-friendly toolkit on the Ethereum platform.
Good news for ZoKrates developers: ZoKrates code development in the browser is now supported in Remix . You can find the ZoKrates plugin via the plugin manager on the left.
The community has been hoping for a richer type system, and ZoKrates now supports complex user-defined types in the form of structures and multidimensional arrays. To enable seamless interaction with ZoKrates programs using these new types from the outside world, we have added a JSON-ABI, which allows easy programmatic access.
To improve the readability of ZoKrates, we have refactored the module system and changed the end of the ZoKrates source code file to
.zok . Internally, the re-implementation of the parser based on the formal DSL syntax (mentioned in the previous update article) has been successfully completed.
Finally, we have made more optimizations to the compiled program to reduce the execution and proof generation time. For science work, we presented these results and the application using ZoKrates at Devcon V in Osaka!