Lumio L2 on Solana uses program-derived accounts (PDA) together with efficient data compression and commitment schemes to ensure robust data availability at the lowest possible cost. In this final part of our DA guide, we’ll delve into these advanced concepts to understand how DA on Solana works. 

Intro: the structure of Pontem’s guide to DA

If you haven’t read the first three parts of our epic guide to data availability, we suggest that you do that - otherwise it will be difficult to understand what we are talking about in this blog. 

In Part 1, we talked about the meaning of data availability (DA) for various types of blockchain players, including full nodes, end users, and rollup sequencers.

In Part 2, we explored the leading DA solutions in the market, including Celestia, EigenDA, NEAR, and Polygon Avail. Though they can support chains built on different blockchain virtual machines and different settlement layers, these DA layers are mostly aimed at Ethereum rollups. 

In Part 3, we looked at the L2 landscape on Solana, including Pontem-built Lumio, ZX by Zeta Markets, Eclipse, and ephemeral rollups by Magic Block. This topic is relevant to our discussion, as any potential 2 on Solana will need to solve the data availability problem. 

Finally, in Part 4, we turn to how Lumio L2 solves the DA issue on Solana by using a combination of PDA addresses, commitments, and data compression. We’ll see how much it costs to store rollup data on Solana mainnet and how this amount can be minimized. It’s a pretty technical subject, but we will try to explain everything in the clearest way possible.  

Reminder: what is Lumio on Solana?

Lumio is part of the Lumio L2 federation of rollups, which also includes SuperLumio, an EVM implementation based on Optimism’s OP Stack, and the current Lumio testnet that supports both EVM and Move VM. 

Lumio on Solana technically supports three VMs: SVM, EMV, and Move VM. Once we have made sure that the devnet is stable, we will make Lumio on Solana the default testnet for the whole Lumio framework.

You can register to test Lumio on Solana in devnet here. 

 How Lumio L2 on Solana handles data availability (DA) 

As we have discussed in Part 3, the maximum size for a Solana transaction is 1 kb (or, rather, 1,232 bytes, compared to 128 kb on Ethereum with the new blob transactions. 1 kilobyte is hardly enough for a whole batch of rollup transactions, which creates a whole new dimension for the DA problem on Solana: you have to use a different way to send and store rollup transactions to the mainnet. 

Luckily, an alternative exists: PDAs, or program-derived accounts. They can store up to 10 MB of data (see the “Intro to PDAs” section of this article). Let’s see how they work. 

Intro to PDAs

First of all, we need to distinguish between two types of accounts on Solana:

1. Keypair accounts: regular accounts that have a private key and a public key;

2. Accounts based on program-derived addresses (PDAs) - special addresses created by a smart contract (program) that don’t have their own private key.

To create a PDA, a program can use a string of numbers or some other predefined input. The address is derived in a deterministic way based on that input, the program’s ID, and an additional value called a bump seed. 

The next step is to create an actual account with that PDA. The program is the only entity that can generate valid transaction signatures for the PDAs it created, so it maintains full control of them. In other words, there is no other user or program that could generate a valid signature for a specific PDA.

By the way, another name for a program on Solana is an executable account - because it executes code when it interacts with another account. Meanwhile, a regular user account or a PDA are called non-executable accounts, since they simply store data but don’t run any code. 

Main reasons why Solana programs need PDAs

1. Storing user-related variables and data: addresses, funds, NFTs, or even - as in the case of Lumio on Solana - rollup transaction data. Unlike smart contracts on Ethereum, Solana programs don’t have built-in data storage space, so they have to use PDAs instead. The maximum that a single PDA can store is 10 MB.

2. Allowing programs to sign transactions. A Solana program doesn’t have a private key. And even if it did, it wouldn’t be able to sign transactions with it, because it would expose the key on-chain and allow anyone to exploit the program and steal all the assets that it controls. A PDA is a workaround, because it has the authority to sign on behalf of the program. 

3. Cross-program invocations. When one program has to call a function in another program, a PDA is often involved. A good example is escrow: user A places tokens in escrow and signs with their private key, after which the program needs to transfer the tokens to User B (the recipient). This requires signing a transaction, which can only be done with a PDA.

4. Using PDAs instead of hashmaps. 

PDA storage costs

A Solana program can create as many PDAs as it needs - as long as it can pay for them. Data storage costs money, as we all know - and while Google Drive may give you 10 GB of space for free, a decentralized network like Solana cannot. Instead, account owners have to lock up funds in order to use storage space, which is provided by validators. This is known as rent. 

The rent amount is expressed in lamports - the smallest units of SOL (1 SOL = 1 billion lamports). The unit is named after Leslie Lamport, a famous computer scientist and pioneer in the study of distributed systems. 

When you place enough lamports into a PDA account, it becomes rent-exempt: there is no need to pay regular rent, but the amount needs to remain locked there as long as the PDA is needed. The rent exemption threshold depends on how much data you need to store. When creating a new PDA, a developer has to indicate the expected storage amount and deposit enough lamports to ensure rent exemption.

When the developer closes the PDA, the lamports can be reclaimed. This is done by transferring the whole balance to another account (as there is no way to take out only part of the deposit). 

Commitment schemes: the basics of KZG

A rollup like Lumio groups transactions into batches, compresses them, and posts them to the L1 (Solana in our case). But how can full nodes on Solana verify that Lumio’s batches are valid? We don’t want to make nodes decompress and read the whole block, as that would be expensive and a waste of resources. Instead, we need to generate a commitment (a kind of a proof) and attach it to a rollup block. 

A commitment scheme in cryptography is a mechanism (or, rather, a primitive) that allows a party (a sender) to guarantee that a message or statement is true without having to reveal it straight away. In the commit phase of the scheme, the sender commits to the message; the commitment is binding, meaning that the verifier cannot change the statement once the commitment has been published. 

The second phase of a commitment scheme is the reveal phase, when the sender discloses the statement and the receiver verifies it. It’s very important that there is only one commitment message or value that can be derived from each statement, and that there is no way to reverse-engineer the original statement from the commit message. 

Commitment schemes differ in terms of how the underlying statement is constructed. One scheme that has emerged as particularly important in the crypto world is called polynomial: the message that a sender commits to is a polynomial - a mathematical expression that features variables (often raised to non-negative integer powers) and coefficients, for example: 

p(x) = 7x³ - 3x² + 2x - 5

In a polynomial commitment scheme, the sender needs to be able to show that for a specific value of X the polynomial takes the value Y without having to reveal the whole expression. The receiver (also called verifier) can ask the sender to disclose the polynomial’s value for various values of variables and then the coefficients from the polynomial (in our example, they are 7, -3, 2, and -5) and then check if the results of these samples are compatible. In the context of data availability, this mechanism is the basis of DAS (data availability sampling). 

What makes polynomial commitment schemes so valuable to rollups is that the size of the commitment message can be very small compared to the size of the polynomial. Thus, given that a polynomial is derived from the rollup’s transaction computations:

1. the commitment acts as sufficient proof  that those computations are valid, and

2. the commitment message is much cheaper to write to the L1 than the original computations or the polynomials based on them. 

The most popular model of polynomial commitment scheme in crypto is called KZG, which stands for Kate-Zaverucha-Goldberg. In particular, it’s used in Ethereum’s proto-danksharding framework. 

Here we won’t discuss how rollup transactions are transformed into polynomials or all the steps of the commitment and verification process (check out this excellent article for more info). Instead, let’s see how the KZG commitment scheme is used in Lumio on Solana in combination with PDA accounts to achieve secure data availability. 

How Lumio ensures data availability on Solana mainnet

1. Lumio block data is compressed to form blobs;

2. A KZG commitment is generated for compressed data;

3. Lumio’s DA program derives a PDA from the data ID and creates a PDA account for it;

4. The commitment is written to the PDA;

5. Lumio sends a series of transactions with chunks of the data (apart from the ones stored in the PDA for the commitment to be verified on the mainnet;

6. Once the commitment is proven correct, the data chunk is stored on the L1 with respect to its offset in the compressed blob of data;

7. Lumio block data and the commitment remain available in the PDA account for the time necessary for an interested party to generate a fraud proof, should they want to challenge a transaction (around 24-48 hours);

8. Lumio executes instructions to finalize the block, while the DA program on Solana marks the data in the block as available.

9. Once the fraud proof window is over, the commitment in the first PDA is overwritten with a new one, or the old PDA is deleted and a new one is created, so that the cycle can be repeated for a new block. 

It’s important to stress that the number of PDA accounts used by Lumio is elastic: it will fluctuate depending on the network load and the amount of L1 storage that Lumio requires. This, together with the high level of compression, ensures that the costs of running dApps of Lumio are minimized. 

Data from old Lumio blocks is stored on L2 archival nodes, so that data is always available. We are also considering integrating Lumio on Solana with one of the major DA layer providers in the future, just like we are going to do with SuperLumio (for which we have chosen EigenDA as our DA partner). 

Lumio DA security

By using the commitment scheme and the PDA account framework, we make sure each of the accounts that we have constructed respects the commitments generated by Lumio L2. A chunk of data won’t be stored in a PDA account unless it is included in the commitment. This means that a Lumio block cannot be finalized on Solana unless all of its data chunks match the commitment. 

This is a secure way to ensure data availability, demonstrate the validity of the rollup block data, and prove that the rollup doesn’t withhold any data. 

Forecasting Lumio costs: PDA storage and running costs

The costs of running a rollup depend on several factors:

1. Network load (the number of transactions that the rollup generates in 24 hours);

2. Compression (how much the rollup manages to compress the data that it writes to the L1);

3. Duration of storage (for how long the rollup wants the data to remain available on the L1). 

Calculating PDA storage fees

How much does it cost to run a rollup like Lumio using PDAs? Let’s first assume that we start with a single PDA (10 MB), though it will take time to fill it to capacity (depending on how much the rollup is used). There is a handy function in Solana to calculate the rent exemption deposit for 10 MB of storage space: 

72.98 SOL is around $11,400 at the time of writing. That’s quite a lot of money, but remember that the money is a one-time deposit and that it will be eventually covered through the fees that dApps pay to use the rollup. 

How much storage space will Lumio need once it’s scaled to capacity and has a big ecosystem of dApps? To calculate this, we need to evaluate the rollup’s load, or throughput. 

On average, Solana processes 2,800 transactions per second, though only 13-15% of that is user-generated transactions; the rest are validator votes. That would make around 420 user transactions per second, though actually it’s more correct to calculate network load per slot - a fixed amount of time that a validator has to generate a block. Currently the slot time is set to 0.4 seconds. 

A recent average throughput per slot on Solana is 600 kb, which comes down to 300 kb after compression. Lumio’s load will be much lower, however, and its compression much better. According to Pontem’s development team,  we can estimate the average throughput of Lumio at 50 kb per slot (0.4s), or 125 kb per second (50/0.4). 

Now we can calculate the total network load of Lumio in 24 hours: 

125 kb/s*60s*60*24=10,800,000 kb = 10.299 gb. 

Around 10.3 GB in 24 hours - that’s how much Lumio will need to handle at full capacity. That’s a bit over 1,000 fully filled PDA accounts. We have already seen that the rent exemption threshold for one 10 MB account is around 73 SOL, so for 10,000 accounts we get… that’s right, 73,000 SOL, or $11.3 million. 

Once again, this number represents a theoretical case when Lumio has a throughput just 2 or 3 times lower than Solana itself, which is an ambitious goal (though we have always been ambitious!). At the beginning, we will need only a minimal amount of storage space, so that Lumio on Solana will always be cost-efficient to run. 

Meanwhile, Lumio’s PDA storage will grow as needed, and if network activity temporarily slows down, excess lamport deposits can always be reclaimed and some PDA accounts closed. We are working on a fee model for the L2 that will make sure that the fixed costs of PDA storage are gradually recovered while at the same time keeping Lumio attractive fee-wise for dApps and end users. 

Lumio running costs

A typical transaction on Solana requires 1 signature, and each signature costs 5,000 lamports (remember, a lamport is one-billionth of 1 SOL). Many consider this base fee structure inefficient, as it doesn’t change depending on fluctuating demand for blockspace, but we just have to assume that the fee will remain fixed at 5,000 lamports. 

For this fee, you get 5,000 compute units, or CU (used to measure the amount of computational effort needed to complete an operation, similar to gas in Ethereum). That’s plenty, because it takes just 1 CU to hash 1 byte of transaction data. Now, as you remember, one transaction in Solana cannot exceed 1,232 bytes, so with 5,000 CU we could hash it almost 5 times over. 

As we have calculated before, at full capacity Lumio on Solana would handle around 10.3 GB of data every 24 hours. So if 1 kb of data costs us 5,000 lamports, we would need:

(10.3 GB/1kb)*5,000 lamports = 50,484,544,000 lamports or 50,48 SOL (around $7,900). 

This is actually much lower than the transaction fees for the same amount of transaction data on Solana 1 (that is, regular transactions not using Lumio’s compression). 

What’s next for Lumio on Solana? 

We firmly believe that dApp developers on Solana will gain a lot from the opportunity to access other ecosystems that Lumio L2 opens for them - and vice versa, that projects in other ecosystems will benefit from this unique chance to connect to the Solana ecosystem. 

At Pontem, we take data availability and rollup security very seriously. We have constructed Lumio’s DA scheme very carefully and taking into account the potential future scaling of the ecosystem. No matter how many dApps run on Lumio, block data will always be made available on the L1 for the time required and at the lowest possible cost. 

Want to be among the first to test Lumio on Solana? Then apply to get whitelisted for public devnet access here. And of course, keep following us on X and in Telegram so that you don’t miss any Lumio news!

Leave a comment

Share

The upcoming Lumio on Solana will use a very interesting mechanism for ensuring data availability. But before we delve into it, we have to explore why Solana needs rollups in the first place and which projects are making the most headway in this area, including Lumio, Eclipse, and Zeta X. 

Intro: how Pontem’s guide to DA is structured

In Part 1 of our epic guide to data availability, we discussed what data availability (DA) means for different types of blockchain users, from full nodes to rollup sequencers and end users.

In Part 2, we looked at the biggest DA solutions in the market, including Celestia, EigenDA, NEAR, and Polygon Avail. They are mostly geared at Ethereum rollups, as the EVM L2 ecosystem is dominant at the moment, though technically these DA solutions should be able to serve non-EVM chains, too. 

In Part 3 and 4, we are turning to a different and very interesting topic: data availability for rollups on Solana. There are few such projects at the moment - the most prominent being Pontem’s Lumio - but the L2 ecosystem on Solana is likely to keep growing, and any such chain will need a DA solution.

However, before we turn to the technical complexities of enabling data availability on Solana, we need to examine the whole spectrum of L2s that are currently being built on Solana or that use SVM for execution. This is what we’ll do in this article, while in the final part of the guide, Part 4, we’ll talk about advanced stuff like KZG commitments and data storage costs. 

The rise of L2s on Solana

Why does Solana need rollups?

Solana is a fast blockchain with low transaction fees, so why would it need Layer-2 chains? There are a few potential reasons.

1. Connecting Solana to other ecosystems

Right now Solana’s ecosystem is isolated from the rest of the blockchain space due to the fundamental differences in its architecture: there is no way to port an EVM dApp to Solana. Lumio L2 solves this issue: as the first VM abstraction framework, Lumio allows developers to deploy on different VMs without rewriting their dApp code. 

Imagine taking an Ethereum app and deploying it on Lumio to settle on Solana, for example. It would support both ETH and SOL as native assets, as well as bridging from both chains. Apps on Lumio can also benefit from the security advantages of different VMs.

Lumio on Solana is already SVM-equivalent, meaning that you can deploy any Solana dApp on it with no code changes. In the future, it will also support EVM and Move VM, providing the first-ever link between the ecosystems of Aptos and Sui and that of Solana. 

2. Congestion

Solana tends to have episodes of congestion when transactions grind down almost to a halt or fail altogether. Well-known examples include the surge of memecoin launches on Pump.fun and the JUP airdrop. At certain points in Q2 2024, over 70% of transactions on Solana ended in failure.

The current infrastructure of Solana’s mainnet doesn’t provide a good solution, but the problem can be resolved through L2s, because a rollup can batch thousands of L2 transactions into a single transaction settled on Solana. All dApps hosted on a rollup will keep working normally even during congestion episodes on the mainnet.

3. Competition for blockspace

This problem is related to the issue of congestion, but it’s not the same. When a single app or memecoin that goes viral starts hogging up a lot of the blockspace, other apps like DEXes and games face a sudden bottleneck and rising fees as mainnet blockspace suddenly becomes more expensive. The solution is to host high-volume apps (for example, a memecoin DEX or a game) on separate application-specific chains to make sure that the apps on the mainnet are not affected. 

4. High infra fees for dApps

As pointed out by @vibhu, creator of the NFT platform DRiP, Solana dApps can end up spending thousands of dollars a week in L1 usage costs - more than it earns in app fees. In other words, Solana may be cheap for end users, but at the same time very expensive for dApps. An L2 like Lumio solves this problem for the most part: apps will still need to pay L2 usage fees related to the use of the mainnet storage space by the rollup, but it will be very little compared to the present L1 costs. 

However, upon further analysis, only the first reason - linking Solana to other ecosystems and VMs - may be really important in the long term. Why? Solana Labs Anatoly Yakovenko explains it well: normal fees on Solana are already very low, and it will be difficult for any rollup on Solana to be cheaper, since it still needs to use the L1’s storage space to write its block data (as we will see). Plus, Solana devs are working on fixes for the issues like outages and fee spikes. 

Here at Pontem, we’ve thought long and hard about this - and a multi-VM or VM-agnostic L2 framework like Lumio is the only one that makes sense. Solana’s ecosystem is isolated, and no internal L1 fix can change this. Only a solution like Lumio can help Integrating it into the EVM ecosystem, which would open up huge advantages for everyone - from trading Ethereum-based memes with Solana’s low fees to Solana-native protocols like Raydium and Marinade being able to support ETH assets.

Leading L2 projects supporting Solana and SVM

Lumio

Part of the Lumio L2 federation of rollups, Lumio on Solana is currently in the public devnet stage. The devnet actually supports three VMs: SVM, EMV, and Move VM. It follows SuperLumio, a mainnet EVM implementation based on Optimism’s OP Stack, and the current Lumio testnet that settles on Ethereum. Once the devnet is properly battle-tested and stable, we will replace the current testnet with it, so that Lumio testnet will settle on Solana. 

You can sign up to test Lumio on Solana in devnet here. 

Eclipse 

The concept of Eclipse is to connect Solana’s speed and performance with Ethereum’s unparalleled liquidity. It executes on SVM but settles on Ethereum, the idea being that SVM is the only environment that can provide true scaling but Ethereum offers better security than any other settlement layer. In other words, it’s an Ethereum L2 based on SVM, rather than an L2 for Solana.

Note the differences between Eclipse and Lumio: the current version of “Lumio on Solana” both executes on SVM and settles on Solana (and not Ethereum), but in the near future it will also support EVM and MoveVM. In other words, Eclipse and Lumio use the same VM for execution but different settlement layers. 

Also, unlike Lumio, Eclipse is designed to support only EVM and SVM and no additional virtual machines. By contrast, Lumio is VM-agnostic: you can whip up a version of it that uses any combination of a virtual machine and settlement layer. 

Eclipse uses Celestia as its DA layer (see Part 2 of our DA guide for more on Celestia). The project launched on mainnet in November 2024. 

Zeta X

Zeta X, or simply ZX, is an upcoming L2 solution for DeFi on Solana, developed by Zeta Markets. It should deliver a fast and efficient trading experience above all, with latency comparable to Binance. 

ZX will use Solana L1 for settlement and data availability - same as Lumio. The big difference is execution: according to the documents, ZX uses an off-chain order matching engine that is compatible with zkVM (zero-knowledge VM), though it doesn’t say which (as there are quite a few zkVMs on the market now). Settlement will work as in a rollup, with batches of transactions sent to the L1 optimistically (but without using ZK proofs).

Credit: Zeta Docs

ZX is an application-specific L2, designed for trading, rather than a general-purpose rollup. Most of its transactions are expected to be trading orders. This makes Zeta X different from Lumio on Solana, which can host all types of dApps, from decentralized perp trading to games. 

Ephemeral rollups by MagicBlock

This solution by MagicBlock, primarily geared at gaming projects, allows developers to offload part of their transactions to SVM instances created on-demand and powered by so-called ephemeral validators. This enables much lower latency than on Solana mainnet, which can be crucial for real-time games with hundreds of players. 

Security is guaranteed through a fraud proof system, as well as a special set of light clients that make sure that an ephemeral validator follows the rules. Once an ephemeral rollup isn’t needed anymore, the validator commits the blockchain state back to the mainnet, subject to a prior check by the security committee.  

The data availability problem on Solana

We have already discussed the basics of the so-called DA problem in Part 1 of this guide. Here we'll talk about how the issue of rollup data availability on Solana differs from the similar issue on Ethereum - in particular, the limitations that Solana specifically imposes on L2s that make DA implementation a bit more complex. 

Why Lumio on Solana needs DA

Lumio is a rollup on Solana: it rolls up L2 transactions into batches and commits them to the Solana mainnet. Validators on Solana need to have access to rollup block data in order to make sure that each new block submitted by Lumio contains only valid transactions. However, publishing all of Lumio’s block data on the L1 would be too expensive. Therefore, we have to post a compressed version of the data while at the same time making sure that validators can always access it and verify it - this is the data availability problem in a nutshell. 

Limitations posed by Solana architecture on L2s

Rollups based on Solana have to contend with a few limitations - nothing severe compared to Ethereum, but still important when building an L1. 

100 MB block size limit

One of the advantages of rollups is that they can stuff lots of transactions into a single block - but the summarized version submitted to the L1 must be within the limits imposed by the L1. In other words, a rollup block needs to be way smaller than the maximum L1 block size. 

On Solana, the maximum block size is 100 MB - very generous compared to the roughly 7 MB that you can fit into a block on Ethereum (where the limit is actually 30 million gas rather than a set number of megabytes). With Lumio on SVM, we decided to go with 10 MB, which is more than enough.

1 Kb transaction limit

A single transaction on Solana cannot be larger than 1,232 bytes, which is a problem for rollups that need to send chunky batches of transactions. For comparison: in Ethereum, the maximum size of a blob transaction (which is what most rollups use now) is 128 Kb, which is more than enough for a compressed rollup block. By contrast, a Solana-based rollup like Lumio can fit only about 10 transactions into a single transaction that it sends to the L1.

Luckily, there is a different way to write rollup transaction data to Solana: PDAs, or program-derived accounts. They can store up to 10 MB of data (see the “Intro to PDAs” section of this article). 

This is as far as we’ll go in this article, however. In Part 4, we will delve not only into PDAs but also into another pretty advanced technological concept: commitment schemes. All that technological groundwork will finally allow us to understand how Lumio L2 ensures data availability on Solana. 

Want to be among the first to test Lumio on Solana? Then apply to get whitelisted for public devnet access here. And of course, keep following us on X and in Telegram so that you don’t miss any Lumio news!

Subscribe now

Share

Leave a comment

Where blockchains cannot solve the data availability (DA) problem, specialized DA layers swoop in to provide rollups with a cheap and secure space to store their transaction data. In this Part 2 of our DA guide, we’ll take a detailed look at the key solutions like Celestia and EigenDA. 

DA 101: A recap

In the first part of our guide to DA, we have learned that: 

• Data availability (DA) is all about making blockchain transaction data available for verification. Ethereum mainnet doesn’t face the DA problem, because each of its full nodes processes all the transactions in a new block, so it knows that they are legit. 

• The DA problem arises once light nodes enter the picture, because they don’t personally process all the transactions. Nevertheless, they can rely on Ethereum’s sync committee of 512 randomly selected full nodes for data on any transaction, so we can consider the DA problem solved.

• DA is a crucial problem for Ethereum rollups, however. They need a way to make transaction data accessible for verification by nodes on the L1 (normally Ethereum), but at the same time not spend too much on writing that data to the L1.

• In the past, rollups used calldata to record transaction data on Ethereum mainnet - a very expensive solution. Now, after the Cancun upgrade, rollup data is posted in compressed blobs that full nodes can’t read. So the DA problem becomes: how do Ethereum nodes make sure all the rollup transactions are legit?

• The solution (sort of) for optimistic rollups is a challenge window (usually 7 days) - a period when any interested validator can challenge a transaction. ZK rollups use zero-knowledge proofs instead of a challenge window.

• Another aspect of the problem is ensuring a rollup sequencer makes ALL of the rollup transaction data available (and doesn’t withhold any) - without having to post it all on the L1. This can be done with data availability sampling (DAS), where nodes download small chunks of data and check if they  match.

In Part 2, we will delve into DA layers - special solutions (which can function as separate blockchains or not) that provide rollups with a place to store their data where it is always accessible to all interested parties, from L1 nodes to end users. 

DA layers and rollups: an attractive solution to the DA problem

To make sure that all transaction data is always available to all interested parties without spending a fortune on Ethereum storage, rollups can use external DA layers - blockchains that specialize in storing and rollup data. This solution offers numerous advantages:

• Low cost: up to thousands of times cheaper than storing calldata on Ethereum L1;

• Support for innovative features, such as data availability sampling (DAS), Namespaced Merkle Trees, and fraud proofs;

• Easier rollup launch: DA layers simplify the task of building a new rollup, as you don’t have to worry about the data availability side of things - there are now even Rollup-as-a-Service solutions;

• Plug-and-Play: a rollup can switch between alternative DA layers. 

• Additional perks: a DA layer can offer benefits to the rollups that use it, such as XXX. 

Note that not just about any rollup can use external DA layers. In fact, a rollup has to be modular - that is, support separate layers for execution, settlement, consensus, and DA. Any rollup built with the OP Stack - including Pontem’s own Lumio - satisfies these requirements, because this stack is modular by design. Other modular stack alternatives include Arbitrum Orbit and zkStack. 

By contrast, an existing rollup like Arbitrum One cannot be integrated with an external DA layer, as it doesn’t support the separation between data availability and other functions by design. 

Another requirement is that a DA layer needs to provide an interface for a specific rollup stack. For example, Celestia’s integrations for Arbitrum Orbit chains and OP chains are only available in testnet for now.  So, for instance, our SuperLumio rollup that settles on Ethereum mainnet can technically support independent DA layers, but for now we use the good old calldata on Ethereum as we wait to be able to integrate EigenDA, a solution by our partner Eigen Labs. 

Let’s now explore the four major DA layer providers: Celestia, EigenDA, Polygon Avail DA, and NEAR DA. 

EigenDA

EigenDA by Eigen Labs is Pontem’s chosen data availability partner for Lumio implementations on EVM chains. There are ample reasons for this choice: advanced architecture, production readiness, and especially the opportunities to integrate Lumio into the wider EigenLayer ecosystem. 

EigenLayer is a restaking layer: it allows users to reuse their staked ETH and liquid staking tokens based on ETH, delegating them to Operators and earning incentives. In turn, operators are recruited by projects to provide so-called Actively Validated Services (AVS) in return for rewards. These AVS projects benefit from the pooled security of staked ETH, so everybody wins: stakers, operators, and AVS protocols.

EigenDA is the first such AVS: it onboards operators to act as nodes, storing and verifying blobs of rollup data. In a nutshell, a rollup’s sequencer node sends batches of transactions to EigenDA and receives a DA certificate, which it posts to Ethereum as calldata. The certificate guarantees that rollup data is available for verification on the DA layer. 

DA certificates are issued by a Data Availability Committee, or DAC - a centralized off-chain entity that attests (via a cryptographic signature) that data is available on the DA. In case anything goes wrong with EigenDA, the system will automatically fall back to Ethereum mainnet (via posting calldata). 

EigenDA data blobs can now be as large as 16 MB - up from 2 MB previously, which is a huge improvement in terms of rollup gas costs, because rollups can now go longer between certificates (and calldata is expensive, as we know). According to EigenDA, large blobs can save up to $2000 a day!

To check that rollup blocks have been coded correctly, EigenDA uses validity proofs. We’ll talk in detail about validity proofs and KZG commitments in Part 2 of this article; suffice it to say that it’s a definitive way to show that transactions in a block are valid, so a rollup protocol that uses validity proofs doesn’t need a challenge window, and it’s more trustless than an optimistic rollup. However, such proofs are also more expensive to calculate: as always, there’s a tradeoff.

EigenDA is still in testnet, but rollup developers can already use it with the main rollup stacks: OP Stack and Arbitrum Nova (zkStack is coming soon). Importantly, both of these forks now support fault proofs - a permissionless way to challenge rollup transactions (though it doesn’t eliminate the 7-day challenge window). 

Pricing for projects depends on the required data bandwidth or storage space: fees start from 0.015 ETH/Gb (around $35) or 70 ETH a year ($165,000) for a reserved 256 Kb/s channel. Yes, data availability doesn’t come cheap! 

EigenDA is integrated with major Rollup-as-a-Service providers (platforms where you can launch a customizable rollup in a few clicks), such as Conduit (which Pontem used to launch SuperLumio), Caldera, and AltLayer. 

Celestia ($TIA)

Celestia is perhaps the best-known DA layer. It’s built with the Cosmos SDK stack, so it’s technically part of the Cosmos family of chains, even though it can serve rollups from different ecosystems. Celestia itself is a proper independent blockchain, with Proof-of-Stake consensus, a native token (TIA), staking rewards, etc. 

A rollup that settles on an EVM chain (like Ethereum) and uses Celestia for DA is called a Blobstream rollup. To prove that rollup data has been published on the DA layer, Celestia validators transfer so-called data commitments to the EVM chain. In case there is any doubt about the availability of a certain rollup block on Celestia, or if a transaction appears to be invalid, a full node on the rollup can issue a fraud proof, similarly to how it happens during a dispute on an optimistic rollup. 

This is a very cost-efficient approach, but it forces Celestia to have a disputation window - just like on an optimistic rollup! - during which light nodes have to wait before they can consider a block valid. This is a major difference from EigenDA, which uses validity proofs.

Rollups that want to use Celestia need $TIA tokens to pay for data storage space, or blobspace. $TIA also plays an important security role: as Celestia validators have to stake $TIA, their stake can be slashed if they break the rules (withhold data). 

Celestia supports data availability sampling (DAS): light nodes can download a number of random pieces from a block instead of having to download the full block. The bigger the blog, the more light nodes are required for DAS for each rollup. This is a more decentralized approach than the DA committee used by EigenDA.

Celestia’s mainnet is live, and it’s currently possible to deploy on Arbitrum Orbit (on mainnet with fault proof support), as well as on OP Stack in testnet. 

Celestia is also integrated with Astria, a solution for deploying modular rollup that share the same network of decentralized sequencers, as well as with Rollkit, a framework for building sovereign rollups on various VMs. 

Polygon Avail

Polygon’s family of products is getting bigger and bigger: there’s Polygon PoS mainnet, Polygon zkEVM, Polygon CDK rollup stack, Polygon Miden privacy rollup, and now also Polygon Avail, a whole ecosystem of solutions: 

• Avail DA, a direct competitor of Celestia and EigenDA;

• Avail Nexus, a cross-chain composability protocol for rollups;

• Avail Fusion, a security layer. 

Avail DA is the first of these to become available on mainnet; it even has a token, AVAIL. Like EigenDA, Avail uses validity proofs rather than fraud proofs; and like Celestia, it features data sampling (rather than a committee).

What sets Avail apart from the competition is that it allows rollup nodes to sample data from the huge network of light nodes on Avail, whereas in other DA layers only full DA nodes can provide such samples. The advantage is that, if DA full nodes are down or start lagging for some reason, the DA layer will keep working instead of having to fall back on Ethereum mainnet. 

Avail DA is already working with OP Stack, Polygon CDK, and Arbitrum Orbit. A zkSync integration is coming very soon, as well as one for StarkWare. The ecosystem already includes almost 10 application-specific chains built using Avail DA, including Fuse, Arcana, and Darwinia

Credit: Avail blog

NEAR DA

NEAR is a very interesting L1: extremely fast and cheap, non-EVM, and with sharding already implemented. The new NEAR DA layer is billed as being up to 30 times cheaper than Celestia. 

The most interesting feature of NEAR DA is how it leverages NEAR’s sharding algorithm, essentially creating sharded data availability. On NEAR, different shards produce chunks of a block, adding a receipt to each chunk, and consensus focuses on the receipts rather than on checking the data in the chunks. The chunks together with the receipts are joined into complete blocks, and after around 60 hours the receipts are pruned (removed), though special archival nodes retain a record of the data. 

NEAR DA uses an off-chain light client to verify that rollup data has been published. As blobs of rollup data are submitted to NEAR, they are also sharded and receipts are generated - and after a few days they are pruned. The light client provides users with blob inclusion proofs, while NEAR validators don’t get slowed down by the stream of rollup blobs. 

This approach makes consensus on NEAR really fast and frees the DA layer from storing unnecessary data. 60 hours should be enough to query or challenge the rollup data stored on the DA layer, and the data throughput is much higher. 

In the future, NEAR plans to add more shards, to the point where each protocol using NEAR DA will be able to have its own DA shard and won’t even need data availability sampling anymore. 

While NEAR L1 is non-EVM compatible, NEAR DA will work perfectly with EVM-centric rollup stacks like OP Stack, Arbitrum Nitro, and Polygon CDK.  For now, though, these integrations are at the Proof-of-Concept stage. It will be interesting to see if NEAR DA will be able to compete with Celestia, which, while apparently more expensive to use, is much better known. 

Credit: NEAR blog

DA layers like EigenDA and Celestia are becoming very important players in the rollup space, while new solutions, including Polygon Avail and NEAR DA, are promising to make this niche even more interesting and competitive. However, our exploration of DA isn’t complete: in Part 3 of the guide, we’ll delve into technicalities like KZG commitments and DA storage costs, as well as see how Lumio on Solana solves the DA problem. Stay tuned! 

Data availability, or DA, is a hot topic in Web3, but it can be tricky for regular users to understand. We’ll delve deep into the definition of DA and the challenges that it poses, before exploring the key solutions like Celestia and EigenDA in Part 2. 

Data availability definition on Ethereum - wide sense and narrow sense

As rollups generate ever more transaction data, they need a safe way to store it so that they can continue scaling without creating congestion. This service is provided by Data Availability Layers - services that allow rollups to store large bodies of transaction data cheaply and make it permanently accessible to all interested parties, from full nodes on the L1 to end users.  

However, before we get to the data availability (DA) problem in the context of rollups, we have to understand what data availability even means. Different sources propose very different definitions for what sort of data should be made available - and to whom.

In this guide to DA, we will move from the narrowest context to the widest, as if through circles on the water. We will do a deep dive into DA in Ethereum, data availability sampling (DAS), and key DA solutions like EigenDA, Celestia and Avail DA. In Part 2, we’ll focus on DA on Solana and how Pontem’s own upcoming DA solution for Lumio and Solana. 

Data availability and full nodes

A full node in a network like Bitcoin and Ethereum is one that stores a complete and up-to-date copy of the whole blockchain state and executes all the transactions in a new block before approving it. 

This mechanism gives rise to one of the biggest advantages of blockchains: trustlessness. A full node doesn’t need to trust the block producer or a central authority that a certain transaction is legit. Instead, when it validates a new block, it knows that all the transactions in it are valid, because it has personally processed and verified each of them. 

Block producers (validators) need to broadcast block data to all full nodes so that they can download it and re-execute the transactions, essentially recreating the block. Ethereum mainnet doesn’t have any data availability problem in this narrow sense - but in the process, scaling issues arise:

a) as the full blockchain size keeps getting bigger, you need more resources to run a full node (1.45 terabytes on Ethereum in January 2024). This limits the potential number of full nodes.

b) as all the nodes have to process all the transactions, more time and resources are needed as the number of nodes grows.

Ethereum blockchain size in GB. Credit: Ycharts

Data availability and Ethereum light nodes

This is the second circle from the center in our analogy: we add light nodes to the mix. Such light clients don’t download the full blockchain state - only the header of each new block. Therefore, they can’t look inside and verify all the transactions. Instead, light Ethereum clients rely on a sync committee, composed of 512 full node validators who are randomly selected on a daily basis; the committee guarantees that all the data in the block header is correct. 

This isn’t full data availability, but it’s good enough. And if a light node wants information on a specific transaction, it can always send a request to a full node. 

Data availability vs. retrievability: new vs. historical data

The official Ethereum page makes an interesting distinction between data availability and data retrievability. Data availability is about nodes having access to the data they need to verify the current block, while retrievability refers to historical data. You can have data stored on the mainnet for a limited time and then deleted, and you’ll still have data availability, even if older data isn’t retrievable. 

Note that major DA solutions like Celestia store both current and historical data, so they take care of data retrievability, too. 

Data availability and Ethereum rollups

To understand the importance of rollups, consider these daily transaction numbers for September 11, 2024:

• Base: 4,7 million transactions;

• Arbitrum: 1.8 million transactions; 

• Optimism: 0.65 million transactions; 

• Ethereum mainnet: 1.2 million transactions.

Together, the three largest optimistic rollups processed over 7.1 million transactions in a single day, which is almost 6 times more than on Ethereum itself. 

Credit: BaseScan.org

Rollups help solve Ethereum’s scalability issue by executing transactions that would otherwise have to be executed by full Ethereum nodes. Relative to Ethereum mainnet, those transactions are processed off-chain - to be precise, by a powerful node called a sequencer.  The sequencer then posts block data in compressed form to the L1.

It’s with the appearance of rollups like Arbitrum and Optimism that data availability issues on Ethereum became obvious. A huge batch with hundreds of rollup transactions can result in just one transaction sent to the mainnet. So how do you know that the data posted by a rollup is correct? 

Let’s see how the definition of the data availability problem changes when we add rollups into the mix. 

Challenge no.1: making sure full L1 nodes have access to rollup block data

Full nodes on the mainnet need a way to know that the block data published by the sequencer is valid. At the same time, that data has to be kept compressed, otherwise it would be too expensive to post it on the mainnet - and in any case, Ethereum nodes can’t possibly go through all those millions of rollup transactions and check them one by one. 

Here’s another way to phrase the rollup data availability problem: rollups must make their block data verifiable by full L1 nodes without having to publish all of it on the L1.

In optimistic rollups like Arbitrum, Optimism, and Lumio, this is achieved in two ways: 

1. Up until the Dencun upgrade, rollups published transaction information on Ethereum in the form of calldata and allowed anyone to challenge it within the initial 7-day window. After that, you couldn’t challenge the data anymore, but the calldata would still be there on Ethereum. Calldata remains on the L1 forever, so that data is permanently available, but it’s very expensive to post: calldata used to account for 95% of rollup costs. 

2. The Cancun upgrade introduced a new data object type called a blob, which is a much cheaper way to store rollup data. 

Blob data is stored on the L1 for only 18 days, after which is deleted. From the point of view of L1 data availability, it’s not a problem, because Ethereum nodes need to verify only new incoming blocks, and 18 days are more than enough. But if rollups want their data to remain retrievable beyond that window, rollups need to use solutions like DA layers. 

ZK (zero-knowledge rollups) solve the data availability problem differently. The ZK proofs that they compute and post to the L1 are in themselves proof enough that their block data is valid, so full Ethereum nodes don’t need to see what’s inside.

Challenge no.2: making sure sequencers make ALL data available

As Yuan Han Li points out in his excellent article “WTF is data availability”, it’s not enough to know that the data made available by a rollup sequencer is valid. What if the sequencer withholds part of the information, though? 

This leads us to reformulate our rollup data availability problem: ensuring that rollup sequencers make all their data available to L1 nodes for verification without having to publish all of it to the L1

This can be achieved through data availability sampling (DAS), also known as data availability proofs (DAP). A node can verify the validity of sequencer data by downloading small samples of it - and once it has taken several samples from the same block, it can be pretty sure that the sequencer has been honest. 

Who does the sampling, though? In the model introduced by Celestia, it’s light nodes that sample sequencer data, while full nodes use the samples to reconstruct the block. In Ethereum, DAS is still on the level of a proposal but should be introduced in a future upgrade. 

Challenge no.3: giving all interested parties access to new and historical rollup data

In the widest possible sense, data availability means accessibility and verifiability of L1 and L2 transaction data for all interested parties: 

• dApps;

• Rollups;

• Light nodes;

• End users.

This applies to both data in newly proposed blocks and those already included in the blockchain (historical data). 

ZK rollups are a very good example why data availability in this wide sense is important. Their zero-knowledge proofs may be good enough for full Ethereum nodes, but for everyone else they are insufficient, because you can’t reconstruct rollup transactions (or state transitions) from these proofs. Users can’t even know their account balances for sure. 

Why have we spent so much time on all the various meanings of the data availability problem? So that you can see that different sources interpret the issue differently, and that the definition of DA itself is complicated. When we talk about DA layers in the next article, we will refer to data availability in the widest possible sense, because DA layers like Celestia or the way our own Lumio on Solana deals with DA cater to all the groups of participants and not just full nodes - and cover both availability and retrievability. 

Hopefully now you can see that data availability is an extremely important issue for the rollup space and for Ethereum scaling in general. In Part 2, we’ll see how projects like EigenDA, Celestia, and Polygon Avail are solving the problem of data availability on Ethereum. Stay tuned!

Share

Subscribe now

Lumio on Solana is now live on public devnet! With support for SVM, Lumio offers a high-performance Layer 2 solution on Solana.

On Layer 1 blockchains without developed L2 ecosystems, like Solana, deploying Lumio as a Layer 2 is crucial for several reasons:

• It expands the capabilities of the Solana blockchain by introducing Layer 2 scalability solutions. This allows for increased transaction throughput and reduced latency. A layer 2 also addresses the issues of congestion and outages that Solana has been known for. 

• Deploying Lumio as a Layer 2 on Solana enables the utilization of advanced virtual machine technologies such as the SVM, parallelized EVM and Move VM (coming soon). This opens up new opportunities for developers to build and deploy decentralized applications (dApps) with enhanced features and functionalities. Unique applications can be built on Solana L2s, such as perp dexes that separate order book execution from liquidations.

• Having Lumio as a Layer 2 on Solana promotes blockchain interoperability. It allows seamless interaction between Solana-based dApps and other blockchain networks, fostering a more connected and interoperable blockchain ecosystem.

This is a major step forward for Lumio, reflecting the improved stability of our Solana build.

What is Lumio?

Lumio is a rollup technology suite that enables developers to build with any VM on any chain.

With Lumio rollups, developers can deploy applications with SVM, Move VM, and parallelized EVM that settle to Solana, Ethereum and other L1s. This allows applications to use native L1 assets like ETH, SOL and TON with minimized trust assumptions secured by zk and fraud proofs. Lumio supports VM equivalence for 100% compatibility so developers can deploy code with no code changes.

Lumio will be compatible with existing federations of rollups such as the Optimism Superchain and Arbitrum Orbits. Developers can also deploy applications on alternative L1s like Solana. Together, these separate instances of Lumio will be called the Lumio Chain Collective.

Lumio on Solana is the latest stage of our Lumio build and enables developers to use native SOL and settle with Solana in their applications.

Access to Lumio Testnet on Solana

Access to the testing isn't public. The Lumio team will add all interested parties using the data from the website, then use a raffle to determine the queue. Please leave your details on the lumio.io website if you want to test Lumio on Solana, as shown in the screenshots below.

Tap on the 'Join Whitelist' button, then ‘Solana address’ to supply your SOL address. You can use the same form to get whitelisted for the Lumio Ethereum testnet.

The first group of Lumio on Solana testers will be announced soon. Stay tuned and keep in touch with us:

Follow Lumio on X.Follow Lumio on Substack.

Learn more about Lumio at lumio.io. 

Subscribe now

Lumio is launching a strategic partnership with Dfiance, an upcoming PVP strategy card game. To celebrate the occasion, we’ll be starting things off with our very own game tournament!

Dfiance is a head-to-head fantasy card game that takes place in an invaded world torn by war. Players chose to fight for one of three factions: the powerful Ephemerals, the ruthless Infernals, or the cunning Celestials. Start with a completely free deck, then improve it as you play. Win battles, earn rewards and use those to upgrade your cards, unlock features, and even wager against opponents. Either way, Dfiance is completely free to play.

Dfiance is currently in beta stage, with the production version planned for later this year.

While Dfiance does not emphasize its on-chain capabilities in order to remove entry barriers, its game assets are in fact limited-edition collectibles. That’s where Lumio comes in. Lumio’s rollup technology lets developers build with any VM on any chain.

That means developers can easily bring their apps to new environments, and you can use assets like ETH, SOL, and TON across ecosystems. 

For more updates on Lumio and our Dfiance partnership, follow Lumio on X and join our Telegram and Discord. Dfiance can also be found on X and on Discord.

If you’re a gaming project looking to deploy on Lumio, get started with our Docs.

Share

Leave a comment

Subscribe now

Today, one of the main challenges with blockchain technology is scalability. The growing demand for blockchain applications has led to increased transaction latency, slowing down the system and making it less efficient, thereby hindering widespread adoption and utility.

There are two distinct approaches to scaling a blockchain ecosystem: Layer-1 (L1) scaling and Layer-2 (L2) scaling. Layer-1 scaling involves creating a new blockchain with enhanced block data, improved consensus algorithms, sharding, or directed acyclic graphs (DAGs). Layer-2 scaling, on the other hand, optimizes the usage of the existing blockchain. Instead of performing all activities on-chain, users can carry out most of their operations off-chain within a Layer-2 protocol.

Layer-2 solutions include channels, sidechains, plasma rollups, and hybrid solutions. The latter two offer low transaction fees, high throughput, and robust security.

In this article, we will explore the challenges associated with blockchain scalability, examine potential solutions, delve into the architecture and key components of Layer-2, including outdated plasma and state channels, and focus on the more promising rollup solutions, providing more details about Lumio.

Blockchain Scalability Challenge

So, to start with, what is blockchain scalability? In the context of blockchain technology, scalability refers to the ability of the network to process transactions quickly. It involves the capacity of a blockchain network to handle an increasing volume of transactions, store data, and add more nodes efficiently and promptly, without compromising core features such as security, decentralization, and consensus.When a network cannot handle the transaction demand or requirements, it leads to slow transaction processing times, high fees, and a poor user experience. These slow transaction times and high fees can hinder the usability and practicality of blockchain networks, especially for applications that require high transaction volumes, such as decentralized finance (DeFi), supply chain management, and others. Therefore, scalability is critical for the future growth of blockchain technology.

Traditional blockchains, such as Bitcoin and Ethereum, face inherent scalability limitations due to their design. These networks typically use consensus mechanisms that require every participant to validate and store all transactions. As the number of nodes and transactions increases, scalability issues become more pronounced. Although this approach ensures decentralization and security, it limits transaction throughput. Consequently, these blockchains often experience congestion, leading to delays in transaction confirmation and higher transaction fees.

Layer-1 and Layer-2 Scaling

There are two main approaches to scaling a blockchain ecosystem: Layer-1 (L1) and Layer-2 (L2) scaling. Layer-1 scaling involves creating a new blockchain with improved block data, enhanced consensus algorithms, sharding, or directed acyclic graphs (DAGs).

Layer-2 scaling, on the other hand, optimizes the use of the existing blockchain. Instead of performing all activities on-chain, users can conduct most operations off-chain within a Layer-2 protocol.

Layer-2 blockchain solutions are essential for addressing the scalability, speed, and efficiency challenges of Layer-1 blockchains. By processing transactions off-chain and leveraging innovative technologies, Layer-2 solutions can significantly enhance performance, reduce load and costs, and improve the user experience of blockchain-based systems without compromising the security and decentralization of the underlying blockchain.

L2 Solutions

1. State Channels

State Channels are solutions that enable numerous off-chain transactions using smart contracts and a deposited fund, where only two on-chain transaction fees are required on the main network.

State Channel Architecture

Initially, participants lock their funds in a multi-signature smart contract on the main network to establish the initial state. Subsequently, participants can conduct numerous fast and economical transactions off-chain. The final state and balances are conveyed to the main network through smart contracts. The state channel is closed once all participants confirm the final state.

Despite offering high transactions per second and low transaction fees, this solution has drawbacks. These include the time and cost involved in creating a smart contract, the inability to execute EVM (Ethereum Virtual Machine) operations, and the need for participants to monitor the network. Additionally, participants must be predetermined, limiting open participation as external parties cannot join the channel without permission.

However, state channels are highly suitable for payment scenarios, particularly for subscriptions involving regular payments. The inability to execute EVM operations and the restricted open participation prevent it from being a universally applicable solution.

2. Sidechains

Sidechains are independent blockchains that run alongside the main blockchain, operating with a two-way peg mechanism to facilitate off-chain processing while maintaining security.

They function as separate chains running parallel to the main blockchain, handling transactions in batches. This allows for more efficient transaction processing off the main chain, resulting in faster confirmation times and lower fees. Sidechains are not constrained by the transaction throughput limitations of the main chain, thereby reducing network congestion and enhancing overall scalability.

In contrast to the main chain, sidechains employ different consensus mechanisms and can implement unique rules and functionalities tailored to specific use cases. By diverting certain types of transactions to the sidechain, where transactions can occur faster and at lower costs, overall transaction throughput can be increased.

After transactions are executed on the sidechain, the final state is securely settled on the mainchain through the two-way peg mechanism. This integration ensures that security is maintained while leveraging the efficiency benefits of sidechain operations.

3. Plasma

Plasma consists of chains known as child chains, which are smaller versions of the Ethereum mainnet. Initially referred to as shadow chains, they later adopted the name child chains.

Plasma Architecture

Imagine the Ethereum main network at the top, with an unlimited number of child chains that can be added beneath it. Furthermore, each child chain can have its own set of child chains. Transactions conducted on these child chains are recorded and then transmitted to the parent chain, eventually reaching the Ethereum mainnet. All transmitted data is assumed to be accurate within the system's structure.

Once data is sent to a parent chain, a 7-day challenge period begins. During this period, if anyone suspects that the data is incorrect or fraudulent, they can submit a fraud-proof to challenge the data. This is why there is a 7-day waiting period for withdrawals in plasma solutions.

4. Hybrid solutions

Hybrid solutions combine features from various L2 technologies for robust scaling.

5. And Rollups 

Rollups are a technology that bundles transactions, hashes key information, and sends these bundles to the Layer-1 (L1) blockchain for execution using a virtual machine known as the execution layer.

ZK Rollups

ZK Rollups derive their name from zero-knowledge proofs, which enable proving the possession of knowledge without revealing specific details. In ZK Rollups, transactions are initially collected and ordered by an actor called the sequencer, who soft-confirms them by execution within blocks.

Next, a prover uses zero-knowledge technology to mathematically prove the correctness of these transactions through a validity proof. This proof, along with compressed transaction data or state differences, is sent to a verifier contract on the main network for final confirmation. This process ensures that transactions achieve their ultimate confirmation.

Optimistic Rollups

Optimistic Rollups are simpler compared to ZK Rollups in architecture. They are compatible with the Ethereum Virtual Machine (EVM) and are easier to develop on. Similar to ZK Rollups, transactions in Optimistic Rollups are gathered, ordered, and executed by a sequencer in blocks.

However, the distinguishing feature lies in the type of proof used. Optimistic Rollups utilize fraud-proofs. Here, the transaction data is compressed and published to the main network, where it is considered correct unless challenged. There is a 7-day window during which anyone can contest the validity of the data by submitting a fraud-proof. If unchallenged, transactions are finalized after this period.

In summary, while ZK Rollups rely on zero-knowledge proofs for transaction verification, Optimistic Rollups assume validity unless proven otherwise through fraud-proofs, affecting the speed and security of transaction finality on the main network.

Introducing Lumio

And this is where Lumio steps in – a modular L2 project that supports any VM on any chain. It serves as an umbrella framework for future L2 chains with various implementations using different virtual machines (VMs). 

Lumio's AltVM abstraction can settle on any L1 blockchain, using any VM as its execution layer, allowing developers to choose their preferred blockchain VM (e.g., EVM, Solana VM, Move VM) and deploy on Lumio without compromising security and performance. Lumio supports parallelized execution by deploying on VMs that facilitate it, introducing an innovative concept to the L2 space.

For more details, please visit https://lumio.io/ 

Lumio Implementations

Lumio on Optimism (the canary mainnet) is the first Lumio instance to settle on the Ethereum mainnet, built using the OP Stack from the Optimism Collective. This stack powers the Optimism mainnet and will serve as the base for the future Optimism Superchain, a network of interconnected L2s.

Lumio on Optimism (testnet) – a production-grade sandbox with EVM and MoveVM versions, currently settling on Ethereum’s Sepolia testnet and moving to the mainnet. Lumio on Optimism uses a Rust implementation of the OP Stack, including the Magi client by a16z and Reth (Rust Ethereum) by Paradigm, with Move VM for faster and safer execution.

Next Lumio Implementation will settle on Solana, supporting SVM (Solana VM), EVM, Move VM, and other VMs. It will be the first Solana rollup with multiple execution layer choices, allowing developers to integrate their products into the Solana ecosystem while maintaining their preferred VM benefits.

Excited? We are. Join us to shape the future of the blockchain together.

Leave a comment

Because it lets them save development costs and resources without compromising on security. Imagine that you have a dApp on Optimism and you’d like to enter the ecosystem of Solana.

Before, you had to rewrite the code completely to move from Solidity to Rust. Since the same dev rarely knows both languages, you’d have to hire a new one—and Web3 devs aren’t cheap. Lumio allows you to avoid the extra costs of entering a new ecosystem.

Another benefit is security. Lumio will share security with many L1s and L2s across ecosystems like Superchain, Eigen Layer, and Arbitrum Orbit. dApps on Lumio will benefit from these extra guarantees, too.

Finally, it’s freedom from vendor lock-in. Some VMs are better for specific applications than others, like SVM, which is great for high-load apps like games.

But you don’t necessarily want your dApp to settle on the corresponding L1. Deploying on Lumio means that you can pick and match a VM and a settlement layer.

Pontem is on a mission to change the crypto industry... Not just with clickbait and low effort forks. Today is the dawn of crypto 2.0. With the advent of sub 100ms latency for DeFi, the industry will be revolutionized. If you are not on the right side of history, you might get ftXd.

With Lumio vertically integrated into Liquidswap and Pontem Wallet, we will deliver the best user experience in crypto across the entire tech stack: interface, smart contract, execution, and settlement. If Amazon did it, so can we... If we are right then soon Uniswap (+ uniswap wallet) and Backpack (+ backpack exchange) will also launch L2s as second movers. For now we have a competitive advantage.

To usher the world into crypto 2.0, Pontem is building Lumio to be vertically integrated with Liquidswap and Pontem Wallet. The killer app for crypto is a vertically integrated tech stack that can compete with centralized applications. When you can offer parity on latency, costs, users only care about security. P2P self custodial DeFi is the killer app that has been staring us in the face. Lumio finally makes it a reality. Not your keys, not your crypto...

We've already shipped Liquidswap, the top DEX on the Aptos network. Now, we're working on Lumio, a modular altVM L2. Lumio on testnet today is built to handle a lot of transactions really quickly and supports the Move VM language. Our aim? To give people a DeFi experience that feels as good and fast as what they'd get on big-name exchanges like Coinbase or Binance, but all happening on the blockchain.

But, to get to why this is a big deal, we need to look back at how trading on the blockchain usually works and what we need to do to make it better. Most of the time, trading directly on the blockchain can be slow, expensive and clunky. We want to fix that.

With Lumio, we're building something that was never before possible in crypto: profitable on-chain market making. We're making sure users and institutions can trade quickly and safely, without giving up control of their assets or compromising on security.

Why does speed matter? Just ask A16z and Columbia researchers: Faster chains = less profits for arbitragers... effectively killing toxic order flow AKA sniping AKA Loss Versus Rebalancing LVR.

https://arxiv.org/pdf/2208.06046.pdf#:~:text=We define loss-versus- rebalancing,model of Budish et al.

The crucial point is in 1/1 √2λγ/σ)) in the equation for cumulative arbitrage profits, this term can be seen as the probability of a profitable trade, or Ptrade. As λ increases (which means interblock time decreases, or the chain is faster), the denominator of this term gets larger, and thus Ptrade gets smaller. This directly scales down the expected arbitrage profits, as the arbitrage profits in this context are a result of trades that occur during these blocks.

https://arxiv.org/pdf/2305.14604.pdf

History of Irrational Markets: Information & Speed

In the realm of trading, the essence of success lies in possessing timely information and the agility to act swiftly in the market.

The tale of Nathan Mayer Rothschild and his alleged financial maneuvers following the Battle of Waterloo serves as a classic example. He reportedly leveraged a rapid courier network to gain early intelligence on Napoleon's defeat, a move that, according to legend, allowed him to manipulate the bond market to his significant benefit. By selling his bonds to incite panic and then repurchasing them at a lower price before the broader public became aware of the victory, Rothschild is said to have amassed a considerable fortune. While this narrative is intriguing and widely cited, its factual accuracy remains a topic of historical debate.

This principle of leveraging information and speed has not only been relevant in the past but continues to hold true today. In the digital age, the need for physical couriers has been replaced by the instantaneous transmission of data via the internet. The key to success now hinges on accessing timely information and employing the fastest means to act upon it in the marketplace.

A modern illustration of this concept is the phenomenon of high-frequency trading HFT. This was epitomized by an event in the early history of the internet, where a group of businessmen established a direct, dedicated internet connection between the Chicago Stock Exchange and the New York Stock Exchange. This setup was designed to arbitrage trade orders at unprecedented speeds, a strategy highlighted in the notable case of FlashBoys. This initiative underscored the critical advantage of speed in executing trades, proving once again that in the competitive arena of financial markets, information and the speed of response are indispensable elements for achieving success.

Modern Day and Blockchain Protocols

The emergence of blockchain technology and decentralized exchange DEX protocols has transformed the trading landscape, though the quest for optimal latency remains a constant across both centralized CEX and decentralized exchanges. Traders utilizing CEXs strive for the lowest possible latency, often relocating their trading infrastructure—such as bots and servers—closer to the CEX data centers. This strategic positioning is done to minimize ping and response times, thereby maximizing trading efficiency and profits.

The situation becomes more nuanced when dealing with decentralized protocols on blockchains like Ethereum and Solana. Here, we've seen the rise of Miner Extractable Value MEV bots. Initially, on Proof of Work PoW chains like Ethereum, traders could prioritize their transactions by paying higher fees, enticing miners to select their transactions for quicker processing. With the transition to Proof of Stake PoS systems, similar dynamics persist, but we also see innovations like Flashbots. These developments have essentially created a marketplace for validators and MEV bots, facilitating agreements to expedite transaction execution.

However, Ethereum's inherent characteristics, such as its 12-second block confirmation time and susceptibility to forks, introduce additional complexities. During the time it takes for a transaction to be confirmed on Ethereum, significant price movements can occur on CEXs, affecting the potential for on-chain actions and strategies. This discrepancy underscores the challenges and considerations unique to trading in decentralized environments, where the blockchain's technical limitations can impact the speed and efficiency of trade execution compared to traditional centralized platforms.

Ethereum's inherent traits, such as its 12-second block confirmation time and the potential for forks, present unique challenges. The delay in transaction confirmation can lead to missed opportunities due to significant price movements on centralized exchanges CEXs). This gap highlights the unique difficulties of trading in decentralized settings, where blockchain's inherent constraints can affect the pace and efficiency of executing trades, in stark contrast to the more immediate transactions seen on traditional CEXs. Additionally, the possibility of block reorganizations (reorgs) on Ethereum adds another layer of complexity, necessitating a waiting period for transaction confirmations to ensure finality.

The Role of Layer 2 Solutions (L2s)

The introduction of Layer 2 solutions (L2s) marks a pivotal shift. Many popular L2s operate with a degree of centralization; for instance, their sequencers— responsible for block production—determine the order of transactions. In environments like Arbitrum or Optimism, the transaction pool (mempool) is not publicly accessible, effectively making it a private mempool. This setup can resemble trading on a CEX, but with notably higher latency, averaging around 2 seconds for Optimism and approximately 0.25 seconds for Arbitrum. Despite the seemingly centralized aspect of these L2s, they offer mechanisms for verification: Optimistic Rollups use a dispute resolution system, while ZK Rollups rely on zero- knowledge validity proofs for validation.

The evolution of L2 solutions is towards decentralized sequencing, akin to Proof of Stake PoS chains, where sequencer nodes would organize transactions. This would open up mempools to broader access, potentially through various methodologies—prioritizing higher fee transactions or implementing time-based solutions. However, it's important to recognize that such advancements in decentralization may not necessarily reduce latency. In fact, as decentralization typically introduces more nodes into the consensus process, latency might increase due to the added complexity and coordination required. Additionally, while L2s like Optimism are designed to minimize the likelihood of reorgs, they are not entirely immune, particularly if their underlying L1 (in this case, Ethereum) undergoes a reorg, although the probability remains low.

Towards a Solution

In navigating these technological landscapes, the goal is to enhance the efficiency and reliability of decentralized trading platforms. The ongoing developments in L2 technology and the exploration of decentralized sequencing represent significant steps forward. However, the challenge remains to balance the benefits of decentralization with the need for low latency and high throughput, essential for competitive trading environments. As the ecosystem evolves, the focus will be on creating solutions that not only mitigate the limitations of current blockchain infrastructures but also unlock new possibilities for traders and investors alike.

Project ‘Yʼ Lumio (Y the F are we doing this?)

In the quest to replicate the seamless experience of centralized exchanges (CEXs) on the blockchain, Super Ultra Important Top Secret SUITS Project ‘Yʼ Lumio identifies several critical attributes that are essential for success:

1.  High Transaction Throughput TPS and Frequency To match the real-time processing capabilities of CEXs, it's crucial to handle a large number of operations swiftly. This not only increases the volume of transactions but also enhances the overall efficiency of the blockchain, moving it closer to a "chain- less" or ‘Web2ʼ experience.

2.  Minimal Confirmation Time with No Forks The system should aim for rapid transaction confirmations. Ideally, the architecture should be designed in such a way that forks are either extremely unlikely or impossible, ensuring consistency and reliability.

3.  Time-Protected or Hidden Mempool Similar to the operational model of current CEXs, where the order book's mempool is not visible to all, starting with a temporarily hidden mempool could be a strategic move. This approach mimics the opaque nature of order handling in CEXs, providing a familiar trading environment for users.

4.  Low Transaction Costs Given the nature of high-frequency trading, where the number of transactions can be vast, keeping transaction fees low is imperative. Affordable costs will enable a higher volume of trades, facilitating more dynamic and accessible trading activities.

5.  Robust DeFi Platforms with Liquidity The ecosystem must include decentralized exchanges DEXes), order books, credit markets, and other DeFi platforms that are rich in liquidity. This ensures that traders have a conducive environment for executing trades efficiently.

In light of these considerations, our research began with a focus on the Move VM, particularly its implementation in Aptos, given our expertise and ongoing development of Lumio. While we're open to exploring similar innovations on other platforms like Sui Move, Solana VM and WASM, our preliminary efforts have concentrated on Aptos due to our familiarity and the existing integration of Move VM with Reth in Lumio, despite the slower speeds due to EVM dependencies.

Aptos has demonstrated impressive performance, notably achieving a throughput of 3050k TPS, with current latency around 500600ms. However, ongoing research indicates potential for even lower latency.

It's important to acknowledge the inherent trade-off between decentralization and latency; as a system becomes more decentralized, latency tends to increase due to factors like network delay and computational overhead.

In our recent advancements with the Aptos node, we've laid crucial groundwork for ongoing modifications aimed at enhancing our Layer 2 L2 framework. Notably, we've transitioned away from the conventional consensus mechanism to adopt a sequencer approach, which systematically processes transactions from the mempool, executes them, and documents the results. Concurrently, we've suspended Peer-to-Peer P2P connections between nodes and removed the logic for transaction distribution from the mempool, streamlining operations. Additionally, we've instituted a new protocol for generating and storing payloads— comprising transactions within a block and their execution hashes—in a dedicated database for future dispatch to L2, marking a significant step forward in our implementation strategy. Also, we separated the move executor from the state commit and mempool process. This significantly reduces the time the executor spends idle waiting for transactions and commits.

Latency measurements focus on intra-machine performance, setting aside network latency for later optimization. For minimized network delays, trading bots should ideally be located close to the sequencer with a strong connection.

Machine Configuration for Testing

Important notice: in the initial version of this article, drafted approximately a month ago, we commenced testing and enhancing the Aptos L2 node. After several iterations, we have observed significant improvements in performance for both the Aptos L2 and the Aptos L1 node, which operates in performance mode. These iterative developments have substantially enhanced the results for both implementations.

The configuration of the machine used for our testing, hosted on Alibaba, is detailed below. It is important to note that the resources were not fully utilized, potentially giving the impression of an overestimation of performance. This setup is suitable given that the node benefits from a dedicated virtual CPU (vCPU, which is crucial for the types of tests conducted. Additionally, full utilization of all CPUs and threads was unnecessary due to the nature of these tests and the workings of BlockSTM.

CPU model: Intel Xeon Platinum 8269CY
vCPU: 104 vCPUs
Memory: 192 GiB
Network speed: 25 Gbit/s

Tests

Understanding the testing of the Move VM and nodes is crucial, and it can be approached in two distinct ways. The first method involves testing the executor's performance in isolation—specifically, the Move VM without the node's framework. In this setup, the Move VM demonstrates impressive performance, achieving transaction processing speeds of 3060k TPS.

However, in our case, the L2 node operates as a standalone unit, which requires the node to be functionally independent and configured for standalone operation. End-to-end E2E testing is paramount as it acts as the definitive verifier of our systemʼs integrity. Although similar to the operational dynamics of an L2 node, the standalone node primarily processes incoming transactions through Remote Procedure Call RPC. This setup positions it as a reliable indicator of our system's functional validity.

Here are the outcomes we've achieved to date, comparing End-to-End E2E latency and Transactions Per Second TPS between Pontem L2 and Aptos L1.

A more comprehensive table is available at this link.

The tests reveal an increase in the Transactions Per Second TPS of the Pontem L2 node, achieving three to four times reduced latency compared to the Aptos node: 3,000 TPS with latency under 100 milliseconds, which ranks among the best results in the market. Maximum in TPS achieved for Pontem L2 node is 11001 TPS. vs 7784 max TPS for Aptos L1. Moreover, these results offer the potential to significantly boost TPS to 20,00030,000, as indicated by some of our tests, particularly with further optimizations and enhanced testing tools.

The tests were conducted using a transaction emitter for Aptos L1. For Pontem L2, we forked the transaction emitter to maximize its performance at this stage. We also benchmarked it using metrics obtained directly from the Pontem L2 node.

Next steps

The figures we've reported represent average metrics, rather than peak values, suggesting the possibility of achieving even lower latencies and higher Transaction Per Second TPS rates under optimal conditions.

This indicates significant potential for growth through targeted optimizations, aiming to elevate TPS rates and diminish latency.

Following changes can still be made:

• Implement specific native caches for the most used applications, such as Liquidswap. Many layers have only one or two frequently used apps. Keep them always in memory and asynchronously commit to the database.

• We have an idea to move the epilogue and prologue of transaction processing into Rust and make it native.

• We are looking into the possibility of improving the process of state transition between sessions of the executor.

• Our testing environment can be enhanced so we can conduct more varied tests with more interesting results.

• And more.

For those interested in conducting benchmarks on L2, given its closed-source nature, we invite you to contact us for further discussion.

Besides this, the next step is Pontem Lumio v2, stay tuned for ultra perfomance AltVM layer.

Leave a comment

Share

Subscribe now

Lumio is a modular L2 project that supports any VM on any chain.

The main advantage of Lumio is that it can support just about any blockchain virtual machine: EVM, parallelized EVM, Move VM, Solana VM, etc. This means developers can deploy on the VM they are used to and choose on which chain transactions will settle.

While developers are vendor-locked on one chain because of native exclusive VMs, Lumio allows you to redeploy your applications on many chains, access liquidity there, and bring in a new community.

For now, there are two working instances of Lumio: Lumio Optimism (mainnet & testnet) and Lumio Solana (devnet; testnet coming soon).

Lumio Optimism (mainnet): It functions as a testnet-in-production, similar to Kusama for Polkadot, supporting engagement, TVL growth, and project launches.

Lumio Optimism (testnet): Deployed on Ethereum's Sepolia testnet, it provides a sandbox for experimenting with EVM and Move VM, supporting versatile dApp development and testing.

Lumio Solana (devnet): Supports SVM, EVM, and Move VM. Once stable, it will replace the current testnet (Lumio Optimism) and launch with settlement on Solana.

Subscribe now