Sam Kaplan
January 9, 2023
Recently the Ethereum community has started to shift from viewing Ethereum and L1s as the primary blockchain for day to day transactions, to being just security layers that L2s build on top of. Vitalik, in his post “Endgame" proposes a future where more centralized L1s like Solana and less centralized L1s like Ethereum converge with centralized block production and decentralized final security guarantees. This thinking is reflected in the Ethereum roadmap which has deprioritized (potentially forever) execution sharding. Data sharding has changed from true multi-chain sharding to simpler “danksharding.”
To me, this betrays the core promise of Ethereum: having a single general purpose blockchain. Instead, we’re going to have a plethora of L2s likely reliant on centralized operators to produce blocks.
It doesn’t need to be this way. Really, what 99% of use cases want are just what’s at the core of blockchains: a way to have shared state, and to execute state transitions. A sufficiently powerful generic virtual machine like the EVM is already able to support almost all current use cases of L2s. This is just the same argument that was used for Ethereum in the first place.
The big question is scalability. How can we scale the L1 without L2? The answer is simple: sharding. In particular, hierarchical sharding allows us to create an infinite hierarchy of shards. Shards belong to shards, belong to shards, belong to the root layer. The challenges to sharding are twofold: correctness and censorship resistance. We don’t want people to be able to successfully include invalid state transitions such as spending coins they don’t have. In addition, we want to make sure that participants in the network aren’t prevented from spending their coins or interacting with the network.
Zero knowledge proofs make correctness easy. Each shard block would include a proof of its correctness and these can be merged all the way up to the root block. Censorship resistance is harder. Vitalik’s earlier sharding proposal solves this problem with the use of committees. For each block, validators (who have staked ETH) are randomly assigned to a shard and perform data availability sampling which guarantees that the block data will be available. In addition, the same security logic in the ongoing proposer/builder separation work can be applied on a shard level to ensure that any valid transaction is able to be included in a shard block. Using random committees of validators allows each shard chain to retain the same security properties as the whole chain.
While this scheme works well for medium numbers of shards, as blockchain use cases proliferate, the number of shards will become quite high, especially as hierarchical sharding is introduced. Since each shard block needs to be validated by a committee of validators, the maximum number of shards will be Total Number of Validators / Committee Size. Ethereum right now has about 400,000 validators. Let’s say that each committee will include 100 validators. Then, the chain will support ~4,000 shards. That may seem like a lot. But, if we expect blockchain applications to explode, Ethereum to win, and most transactions to happen on L1, then this will not be enough in the long term. It’s possible that the number of validators can be increased, but the number of nodes that can comfortably coordinate in the core consensus process is likely limited.
To support more shards, I propose the following. Recognize that the limiting factor here is the number of validators. What do we do when we have a scarce resource? Create a market! Each shard will have a Contracted Validator who makes a deposit. This Contracted Validator could be provided by a generic service provider and get fees from the shard blocks it’s validating. Or, the primary users of the shard could operate the Contracted Validator. Every block time, the Contracted Validator produces and certifies a block for its shard. This is rolled up to its parent shard until the root chain.
The Contracted Validator is kept honest through a dispute process. If a user is unable to access the shard block data (data availability) or is unable to get its transaction included in a shard block, then they can make a bid for a Network Validator. In this bid, the contesting user will include a commitment to the shard chain they’re seeking to dispute. If there are 4,000 available committees, then the top 4,000 bids will be selected and committees will be assigned to each. Once a committee is assigned, the contesting user will reveal the identity of the shard chain to the committee members. The committee will then verify the shard chain. If the Contracted Validator is not performing its duties, then it will lose its deposit and a new Contracted Validator will be chosen.
Since any misbehaving Contracted Validator is guaranteed to lose their deposit, this should make the equilibrium state that all Contracted Validators perform their duties correctly. The only issue comes if there’s a coordinated attack or if the size of the network grows to the point that there are more shard blocks with errors than the number of committees. But, given that there will be thousands of committees, this system will be able to correct thousands of shards per block, and hundreds of thousands of shards per hour. Each correction costing the Contracted Validator their deposit.
This system doesn’t come without downsides. In particular, because this is essentially a fraud proof system, there is necessary waiting even once a transaction gets included in a shard block. For instance, if a Contracted Validator validates lots of blocks, but doesn’t make them available, then all those blocks will be invalid once the dispute process has taken place. Of course, there are lots of ways to mitigate this. For instance, users could bid for Validators to validate their shard proactively. Code running in other shards could see that a given shard block was officially validated and not wait for confirmations. Given the large number of validators, this could work for 99% of shards in the short term and medium term. Though, this somewhat reduces security by adding more competition for validators.
Another downside is data loss. If there are hundreds of thousands of shards, what happens when all the network participants in one shard disappear?
Actually, these aren’t downsides, but the tenets of a new model of blockchain programming. Just as large programs move from monoliths to micro services, blockchain applications need to learn to love eventual consistency as a first class citizen. In addition, shards should stop being viewed as generic, and rather should be viewed as homes for specific applications and communities.
So, you might be asking, isn’t this just basically the same as L2s today? Aren’t Contracted Validators just centralized block producers where if something goes wrong, a fraud proof is posted to the main chain? All scalability solutions have common features. But, this solution is much simpler. Everything works together in one happy, extremely interoperable ecosystem. Instead of pushing fundamental and universal problems to the L2, we solve them once and for all in an extremely scalable system. Above all this will provide a great developer experience.
Imagine you’re writing a smart contract deployed to Your Shard. As part of your code, you need to exchange DAI for USDC. Well, with great developer tooling, you can write cross-shard eventually consistent code easily.
function convertDAIToUSDC(LocalAsset<DAI> inputDAI) { return await Uniswap.convert(DAI, USDC, inputDAI); }
This simple function will compile to perform the following steps:
In this example, the developer experience is dead easy even though the task will require multiple cross-shard transactions across many blocks. While in this example, a simple conversion requires four cross-shard transactions, this will certainly be significantly optimized in the future including automatically parallelizing and batching transactions from different parts of the contract.
I believe that there will be a single L1 winner. Almost all applications can be implemented with just state and state transitions backed by a general purpose computer. As L1 scaling is solved and developers can easily build scalable blockchain applications, a million flowers will bloom, rewriting our world to be more efficient and much, much more interoperable. This will unleash developers and entrepreneurs to be even more creative and add an entirely new layer of abstraction to society. Ethereum is the largest general purpose blockchain and its vision and focus on true decentralization position it to be the L1 winner - to be the center of a new Internet. I hope that the Ethereum community remembers its initial promise: one blockchain to rule them all.