Solana Gulf Stream: transaction forwarding protocol

Intro to Mempool Management

When a transaction occurs on Ethereum or Bitcoin, it is signed, and then the transaction is forwarded to a pool where unconfirmed transactions are communicated to nodes using a ‘gossip’ protocol. Meaning the entire network will be informed of the upcoming transactions. Further, these nodes may also filter transactions based on node-specified criteria, such as low fees, and exclude them. Because nodes can filter transactions, they have to be propagated multiple times throughout the network. This process is both inefficient and costly. Furthermore, if a large number of transactions are excluded because of filtering, the mempool must maintain all these transactions until they are processed, leading to stale transactions. While it may provide an economic price for block space and limit only those transactions that make economic sense at a point in time, it could leave many transactions in limbo.

This solution not only leads to surge pricing for all transactions as they all compete to be confirmed, but it also places a constraint on the overall system, creating congestion. To use an analogy, it is like a highway that has its onramps filled with cars, with each onramp checking what the car is doing or where it is going before letting it onto the highway. That would take an extreme amount of time to manage the oncoming traffic. The solution at first may seem to be to set up more ramps and maybe make cars of a certain type and let them all run at once in batches, or to just keep increasing the size of the ramps. But what if you could direct those cars to different onramps at different points and create a schedule of the order in which ramps will be open to allow cars on the highway. This means that drivers can self-organize, and there’s less waiting to get onto the highway. Cars will go to the next onramp until they find a ramp with the fewest cars, or they will just wait until there’s less traffic.

This analogy, while simplistic, can represent what Solana has implemented with the Solana Gulf Stream. The traffic is pushed out to the outer limits of the “highway” and "ramps." Because the schedule is known ahead of time, transactions are directed to the next leader. Allowing for fast switching and reducing the amount of infrastructure needed to hold all the waiting demand. This can only happen when the leaders are known and the most recent events can be referenced and valid for a known time so that they can remain valid for a set time.

So how does Solana Gulf Stream work?

Solana is a continuous block-producing chain, meaning multiple blocks are being constructed at a time. A block is proposed for consensus roughly every 800 ms. Once a block is confirmed, i.e., consensus is reached, a transaction cannot be rolled back; approximately 31 blocks have been created. Using this as a default, we can assume a block is, at worst, 32 blocks old. A transaction can reference a recently fully confirmed block-hash and maintain validity as a child of the referenced block for a set time. In simple terms, if a block-hash is 32 blocks old and block times are 800 ms, the transaction has roughly 25.6 seconds of validity, at which point it is confirmed or it expires. Transactions having an expiration date means they don’t spend time in limbo. In other words, there is no mempool, no transactions are maintained in memory, and there is no double propagation of the same transaction.

Once a transaction is created, it will be forwarded directly to an upcoming leader to be included. At which time the validator or leader will confirm the validity of the transaction, arrange it in order, and forward it on to Turbine to be propagated as partial blocks to other validators and reach consensus. The clients can subscribe to receive updates from the validators, informing them of confirmations if, after the time elapses, it can be assumed to have failed. This allows clients to sign and submit transactions and guarantee that they are either failed or executed. Meaning all failed transactions are dropped, reducing the amount of storage required. Additionally, since a transaction is sent to leaders ahead of time, they can execute it before it is their turn. Further priority can be placed on those validators with greater stake weight, assuming they are less likely to submit bad transactions because of greater financial risk.

This architecture means less memory demands and faster transaction processing, though the transactions that fail will be dropped. Under large traffic, the network will drop greater numbers of transactions because more transactions mean more transactions expiring or conflicting in an effort to keep the cache clearer. While this has the benefit of not needing to maintain the memory of unconfirmed transactions, it could present a bad user experience where transactions that an individual has submitted have apparently vanished because a user may not know that once the transaction is signed, it is effectively proposed and not final until it is confirmed or dropped. It is essential that developers communicate this to users.

Key features of Solana Gulf Stream

• Transactions are only valid for as long as the block hash is valid, approximately 25.6 seconds.

• As soon as a transaction is signed, it is sent to a leader, and it can only be confirmed or dropped, meaning it can never be queued.

• Because a transaction is only valid for a set amount of time, it can get dropped, leading to users wondering what happened and being unable to locate transactions.

• Transactions are not known by the network as a whole until the leader creates the block and it is propagated via Turbine.

• Since there is no mempool, transactions cannot be observed ahead of time. Making them effectively private proves to be effective in reducing MEV or sandwich attacks when trading, although there are ways around this.

Concluding Thoughts

To sum it all up, Solana manages unconfirmed transactions without a mempool, which is often a choke point for many blockchains. It achieves this by using what is known as Gulf Stream. A protocol that forwards transactions directly to leaders because of a known schedule. A transaction contains the block-hash of a recently fully confirmed block and is valid for roughly 25.6 seconds. This means a transaction is either confirmed or expires, effectively eliminating the backlog of transactions to be processed. Since only transactions that have been signed and delivered to the leader ahead of time will be processed by a validator, there is no need for additional bloat, and the same transaction is never propagated more than once. However, it isn’t all roses; it can sometimes lead to a bad user experience.