-
Notifications
You must be signed in to change notification settings - Fork 396
docs: aggregation mode #1964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
docs: aggregation mode #1964
Changes from 11 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
e2faf9f
docs: [wip] agg mode
MarcosNicolau 70815b7
docs: example
MarcosNicolau 714d1b1
docs: add to summary.md
MarcosNicolau d0b7c4b
docs: improve deep dive
MarcosNicolau f0904bf
docs: add proof commitment section
MarcosNicolau df68f82
docs: more polish
MarcosNicolau 6552f2c
docs: summary fix deep dive
MarcosNicolau d786e36
docs: add submission note
MarcosNicolau 5e554bd
docs: misspell + eip-4844 link
MarcosNicolau b518e55
Update docs/2_architecture/agg_mode_components/1_deep_dive.md
MarcosNicolau fa91181
Update docs/2_architecture/agg_mode_components/1_deep_dive.md
MauroToscano 27ca399
Update docs/3_guides/3.1_aggregation_mode.md
MauroToscano 17226ee
Update docs/2_architecture/agg_mode_components/1_deep_dive.md
MarcosNicolau 53b2f52
docs: move blob capacity to rust code
MarcosNicolau 18aad92
Merge branch 'testnet' into docs/aggregation-mode
MauroToscano File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| # Aggregation Mode Deep Dive | ||
|
|
||
| The Aggregation Mode runs **once every 24 hours** and performs the following steps: | ||
|
|
||
| 1. **Fetch Proofs from the Verification Layer** | ||
| Queries `NewBatchV3` events from the `AlignedLayerServiceManager` and downloads the batches from `S3`, starting from the last processed block of the previous run. | ||
|
|
||
| 2. **Filter Proofs** | ||
| Filters proofs by supported verifiers and proof types. | ||
|
|
||
| 3. **Aggregate Proofs in the zkVM** | ||
| Selected proofs are aggregated using a zkVM. | ||
|
|
||
| 4. **Construct the Blob** | ||
| A blob is built containing the [commitments](#proof-commitment) of the aggregated proofs. | ||
|
|
||
| 5. **Send Aggregated Proof** | ||
| The final aggregated proof and its blob are sent to the `AlignedProofAggregationService` contract for verification. | ||
|
|
||
| > [Note] | ||
| > Currently if you want your proof to be verified in the `AggregationMode` you need to submit it via the `VerificationLayer`. As explained above, in the next run the `AggregationMode` will fetch your proof from the `VerificationLayer` batches in spite of its verification status. | ||
|
|
||
| ## Aggregators and Supported Proof Types | ||
|
|
||
| Two separate aggregators are run every 24 hours: | ||
|
|
||
| - **Risc0**: Aggregates proofs of types `Composite` and `Succinct`. | ||
| - **SP1**: Aggregates proofs of type `Compressed`. | ||
|
|
||
| ## Proof Commitment | ||
|
|
||
| The **proof commitment** is a hash that uniquely identifies a proof. It is defined as the keccak of the proof public inputs + program ID: | ||
|
|
||
| - **For SP1**: | ||
| The commitment is computed as: `keccak(proof_public_inputs_bytes || vk_hash_bytes)` | ||
| - **For Risc0**: | ||
| The commitment is computed as: `keccack(receipt_public_inputs_bytes || image_id_bytes)` | ||
|
|
||
| ## Multilayer Aggregation | ||
|
|
||
| To scale aggregation without exhausting zkVM memory, aggregation is split in two programs: | ||
|
|
||
| 1. **User Proof Aggregator** | ||
| Processes chunks of `n` user proofs. Each run creates an aggregated proof that commits to a Merkle root of the user proofs inputs. This step is repeated for as many chunks as needed. Usually each chunks contains `256` proofs but it can be lowered based on the machine specs. | ||
|
|
||
| 2. **Chunk Aggregator** | ||
| Aggregates all chunk-level proofs into a single final proof. It receives: | ||
|
|
||
| - The chunked proofs | ||
| - The original [proofs commitments](#proof-commitment) included each chunk received | ||
|
|
||
| During verification, it checks that each chunk’s committed Merkle root matches the reconstructed root to ensure input correctness. The final Merkle root, representing all user [proofs commitments](#proof-commitment), is then committed as a public input. | ||
|
|
||
| ## Verification | ||
|
|
||
| Once aggregated, the proof is sent to Ethereum and verified via the `AlignedProofAggregationService` contract. Depending on the proving system, the contract invokes: | ||
|
|
||
| - `verifySP1` for SP1 proofs | ||
| - `verifyRisc0` for Risc0 proofs | ||
|
|
||
| Each function receives: | ||
|
|
||
| - The public inputs | ||
| - The proof binary | ||
|
|
||
| The program ID is hardcoded in the contract to ensure only trusted aggregation programs (`chunk_aggregator`) are accepted. | ||
|
|
||
| If verification succeeds, the new proof is added to the `aggregatedProofs` map in contract storage. | ||
|
|
||
| ### Proof Inclusion Verification | ||
|
|
||
| To verify a user’s proof on-chain, the following must be provided: | ||
|
|
||
| - The proof bytes | ||
| - The proof public inputs | ||
| - The program ID | ||
| - A Merkle proof | ||
|
|
||
| The Merkle root is computed and checked for existence in the contract using the `verifyProofInclusion` function of the `ProofAggregationServiceContract`, which: | ||
|
|
||
| 1. Computes the merkle root | ||
| 2. Returns `true` or `false` depending if there exists an `aggregatedProof` with the computed root. | ||
|
|
||
| ## Data Availability | ||
|
|
||
| When submitting the aggregated proof to Ethereum, we include a **blob** that contains the [commitments](#proof-commitment) of all the individual proofs that were aggregated. This blob serves two main purposes: | ||
|
|
||
| - It makes the [proof commitments](#proof-commitment) publicly available for **18 days**. | ||
| - It allows users to: | ||
| - Inspect which proofs were aggregated | ||
| - Get a Merkle proof to verify that their proof is included in the aggregated proof | ||
|
|
||
| ### Blob capacity | ||
|
|
||
| As dictated in the [eip-4844](https://eips.ethereum.org/EIPS/eip-4844) Each blob can hold: | ||
|
|
||
| - `FIELD_ELEMENTS_PER_BLOB = 4096` | ||
| - `BYTES_PER_FIELD_ELEMENT = 32` | ||
|
|
||
| Which results in a total theoretical capacity of: | ||
|
|
||
| `FIELD_ELEMENTS_PER_BLOB * BYTES_PER_FIELD_ELEMENT` = `4096 * 32` = `131.072 bytes` | ||
|
|
||
| However, this full capacity can't be used due to how KZG bytes to elliptic curve points are encoded. Specifically: | ||
|
|
||
| - Ethereum uses the BLS12-381 curve, whose scalar field modulus is slightly less than `2^256`, in fact, it's closer to `2^255`. | ||
|
MarcosNicolau marked this conversation as resolved.
Outdated
|
||
| - That means the 32-byte field elements can't represent arbitrary 256-bit values. | ||
| - To stay within the field modulus, we **pad the value with a leading `0x00` byte**, ensuring it's below the modulus. | ||
| - This reduces the usable payload to **31 bytes per field element**. | ||
|
|
||
| So the _actual usable capacity_ per blob becomes: | ||
|
|
||
| `4096 * 31` = `126.976 bytes` | ||
|
|
||
| ### Current Bottleneck | ||
|
|
||
| Since each [proof commitment](#proof-commitment) is exactly **32 bytes**, the maximum number of proof commitments that can fit in a single blob is: | ||
|
|
||
| `126.976 / 32` = `3968 proofs` | ||
|
|
||
| This is the **current upper limit** on how many proofs we can include in a single aggregation run. | ||
|
|
||
| ## Scaling Beyond Current Limits | ||
|
|
||
| To increase throughput we can: | ||
|
|
||
| 1. **Send Multiple Blobs per Transaction** | ||
| Up to **6 blobs** can be included per transaction, supporting up to **23,808 proofs per run**, which is more than we can aggregate in one day. | ||
|
|
||
| 2. **Run Aggregation More Frequently** | ||
| Reducing the interval between aggregation runs. | ||
|
MarcosNicolau marked this conversation as resolved.
Outdated
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,208 @@ | ||
| ## Aggregation Mode L2 integration example | ||
|
|
||
| This guide demonstrates how to build a toy L2 application that integrates with Aligned Aggregation Mode. The L2 does not post state diffs or any data to Ethereum, only commitments. The prover has to prove that: | ||
|
MauroToscano marked this conversation as resolved.
Outdated
|
||
|
|
||
| 1. The state database used in the proof must match the commitment stored in the on-chain contract. This is validated by computing the commitment of the received data in the zkvm and then exposing it as a public input. | ||
| 2. The users performing the transfers have enough balance | ||
|
|
||
| After processing the transfers, the vm computes the commitment of the post state, which is exposed as a public input. The smart contract then updates the on-chain state root. If a user later wants to retrieve their state, the application must return it along with a Merkle proof, so they can verify it against the contract’s state root. | ||
|
|
||
| Notice a lot of checks that a real L2 should have are missing, since the focus are on the integration of Aligned. | ||
|
|
||
| The code can be viewed at `examples/l2`. | ||
|
|
||
| ## L2 workflow overview | ||
|
|
||
| This Layer 2 (L2) system operates in two main steps: | ||
|
|
||
| - Off-chain execution and proof generation + verification with Aligned Verification Layer (a.k.a Fast Mode). | ||
| - On-chain state update via proof verification with Aligned Aggregation Mode. | ||
|
|
||
| In Step 1, we execute user transfers and generate a zkVM-based proof of the state transition, which is submitted to Aligned’s verification layer. | ||
|
|
||
| In Step 2, once the proof is aggregated (every 24 hours), it is verified on-chain to update the global state. | ||
|
|
||
| ### Step 1: Off-Chain Execution + Proof Generation | ||
|
|
||
| 1. Initialize State: Load or initialize the current system state. | ||
| 2. Load Transfers: Retrieve or receive the user transfer data for this batch. | ||
| 3. Execute in zkVM: Run the zkVM with the loaded transfers to compute the new state. | ||
| 4. Generate Proof: Produce a zk-proof for the executed state transition committing the commitment of the received + the commitment of the new state. | ||
| 5. Submit Proof to Aligned: Send the proof to Aligned Verification Layer | ||
| 6. Save the binary proof locally for later on-chain verification. | ||
|
|
||
| ### Step 2: Proof Verification + On-Chain State Update | ||
|
|
||
| 7. Load the proof binary: Retrieve the saved proof binary from disk. | ||
| 8. Update On-Chain State: Call the smart contract method `updateStateTransition`, which: | ||
|
|
||
| - Internally calls `verifyProofInclusion` on AlignedProofAggregationService which: | ||
| 1. Computes the proof commitment from the proof `public_inputs` and `program_id`. | ||
| 2. Uses the Merkle proof to reconstruct and validate the Merkle root. | ||
| 3. Confirms whether there exists and aggregated proof with that root. | ||
| - Validates that the `initial_state_root` proof public input matches the on-chain state. | ||
| - If valid, updates the on-chain state root to the `post_state_root`. | ||
|
|
||
| # Usage | ||
|
|
||
| ### Requirements | ||
|
|
||
| 1. [Rust](https://www.rust-lang.org/tools/install): we have tested in v1.85.1 | ||
| 2. [Foundry](https://book.getfoundry.sh/getting-started/installation) | ||
| 3. [Docker](https://docs.docker.com/engine/): for SP1 prover | ||
|
|
||
| Submodules of the repo should be imported by running on the root folder: | ||
|
|
||
| ```shell | ||
| make submodules | ||
| ``` | ||
|
|
||
| You can run the example on: | ||
|
|
||
| - [Holesky](#setup-holeksy) | ||
| - [Localnet](#setup-localnet) | ||
|
|
||
| ## Setup Holeksy | ||
|
|
||
| ### 1. Create keystore | ||
|
|
||
| You can use cast to create a local keystore. If you already have one you can skip this step. | ||
|
|
||
| ```bash | ||
| cast wallet new-mnemonic | ||
| ``` | ||
|
|
||
| Then you can import your created keystore using: | ||
|
|
||
| ```bash | ||
| cast wallet import --interactive <path_to_keystore.json> | ||
| ``` | ||
|
|
||
| Then you need to obtain some funds to pay for gas and proof verification. | ||
| You can do this by using this [faucet](https://cloud.google.com/application/web3/faucet/ethereum/holesky) | ||
|
|
||
| _This same wallet is used to send the proof via aligned, so you'll also need to fund it on aligned. Follow this [guide](https://docs.alignedlayer.com/guides/0_submitting_proofs#id-2.-send-funds-to-aligned)._ | ||
|
|
||
| ### 2. Deploy the contract | ||
|
|
||
| - Generate the base `.env`: | ||
|
|
||
| ```shell | ||
| make gen_env_contract_holesky | ||
| ``` | ||
|
|
||
| - Get the program ID of the l2 program you are proving: | ||
|
|
||
| ```shell | ||
| make generate_program_id | ||
| ``` | ||
|
|
||
| - Complete the following fields `contracts/.env` file: | ||
|
|
||
| - `PROGRAM_ID=` (use the previously generated ID, you can re check with a `cat ./crates/l2/programs_ids.json` ) | ||
| - `PRIVATE_KEY`: the private key used for the deployment, it needs to have some funds to pay for the deployment. | ||
| - `OWNER_ADDRESS`: you have to provide the _address of the wallet created in step `1.`_. | ||
|
|
||
| - Deploy the contracts with: | ||
|
|
||
| ```shell | ||
| make deploy_contract | ||
| ``` | ||
|
|
||
| _Save the output contract address._ | ||
|
|
||
| ### 3. Setup the L2 | ||
|
|
||
| - Generate the base `.env` run: | ||
|
|
||
| ```shell | ||
| make gen_env_l2_holesky | ||
| ``` | ||
|
|
||
| - Complete the missing fields on the `.env`: | ||
|
|
||
| - `PRIVATE_KEY_STORE_PATH`: The path to the keystore created in `1.`. | ||
| - `PRIVATE_KEY_STORE_PASSWORD`: The password of the keystore crated in step `1.`. | ||
| - `STATE_TRANSITION_CONTRACT_ADDRESS`: The address of the contract deployed in step `2.` | ||
|
|
||
| Finally [run the l2](#running-the-l2). | ||
|
|
||
| ## Setup Localnet | ||
|
|
||
| You can also run this example on a local devnet. To get started, navigate to the root of the Aligned repository | ||
|
|
||
| - Start Ethereum package and the Batcher | ||
|
|
||
| ```shell | ||
| # This will start the local net | ||
| make ethereum_package_start | ||
| # Start the batcher | ||
| make batcher_start_ethereum_package | ||
| ``` | ||
|
|
||
| - Navigate back to the example directory: | ||
|
|
||
| ```shell | ||
| cd examples/l2 | ||
| ``` | ||
|
|
||
| - Generate the `.env` files for the contracts and L2: | ||
|
|
||
| ```shell | ||
| make gen_env_contract_devnet | ||
| make gen_env_l2_devnet | ||
| ``` | ||
|
|
||
| - Generate a pre funded wallet (or create one as specified [previously here](#1-create-keystore)): | ||
|
|
||
| ```shell | ||
| # This will generate the keystore and fund it on aligned | ||
| make gen_devnet_owner_wallet | ||
| ``` | ||
|
|
||
| - Generate the program ID of the program that is going to be proven: | ||
|
|
||
| ```shell | ||
| make generate_program_id | ||
| ``` | ||
|
|
||
| - Set the generated program ID on `contracts/.env`. | ||
|
|
||
| - Deploy the contract | ||
|
|
||
| ```shell | ||
| make deploy_contract | ||
| ``` | ||
|
|
||
| - Set the output address of the contract in `.env` | ||
|
|
||
| - [run the l2](#running-the-l2) | ||
|
|
||
| ## Running the L2 | ||
|
|
||
| - Set up the initial State | ||
|
|
||
| ```shell | ||
| make init_state | ||
| ``` | ||
|
|
||
| - Perform the L2 account updates and prove them in the zkvm: | ||
|
|
||
| ```shell | ||
| make prove_state_transition | ||
| ``` | ||
|
|
||
| - Wait 24 hs for the proof to be aggregated, or if running locally, run the aggregator with either: | ||
|
|
||
| ```make start_proof_aggregator_ethereum_package AGGREGATOR=sp1``` | ||
|
|
||
| or with cuda: | ||
| `make start_proof_aggregator_gpu_ethereum_package AGGREGATOR=sp1` | ||
|
|
||
| - Update state transition on chain: | ||
|
|
||
| ```shell | ||
| make update_state_on_chain | ||
| ``` | ||
|
|
||
| You should see a transaction receipt in the console and after the stateRoot updated on-chain. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.