The Path to Decentralized Nasdaq

Solana was originally founded to build a blockchain that is so fast and so cheap that you can put a working central limit order book on top of it. Solana mainnet beta launched in March of 2020 – we are now five years in and, despite everything we have achieved, it’s becoming increasingly clear that we have not succeeded in that goal.

The fundamental building blocks of blockchains as they exist today are not designed for trading. If we want to achieve Solana’s original mission, we must go back to the drawing board and fundamentally redesign consensus from first principles, with the ultimate goal of producing a decentralized network that has the necessary features to compete with the New York Stock Exchange.

When we say compete with the New York Stock Exchange we mean exchanges on Solana need to be able to provide better prices than their centralized counterparts. In the world of markets, we define prices according to the spread: the difference between the highest price somebody is willing to buy the asset for and the lowest price somebody is willing to sell the asset for.

The tighter spreads are, the better the price a trader will get and the more efficient the market will be. 

The formula for spread is simple. The spread is set such that the expected revenue for market makers from trading against uninformed traders is equal to their expected losses from trading against informed traders. When market makers have more information than their counterparties they make money and when they have less information they lose money. Market makers typically make a little bit of money on each trade against retail but lose a lot of money after (hopefully infrequent) price jumps when they are caught offsides. This is the origin of the phrase: “market makers pick up pennies and lose dollars”.

What Determines Adverse Selection Costs?

To understand adverse selection better, we need to understand the game that market makers are playing. Market makers have this idea of a fair price (fair) that changes stochastically over time. When the fair is inside the bid ask spread, market makers' quotes are safe because takers cannot make a profit by crossing the spread to pick them off. As soon as the fair moves outside of the bid ask spread, a race begins. The market maker tries to cancel their order as fast as possible and takers try to snipe the stale order before the market maker can cancel it. A successful taker earns the difference between the fair price and the stale quote in expectation. A large part of reducing friction from adverse selection is ensuring that the market maker wins this race as often as possible.

Data from a centralized exchange shows that, after a jump, market makers win the race to cancel just 13% of the time.

Market makers on centralized exchanges don’t win the race to cancel that often but market makers on Solana win the cancel race even less of the time. The Jito auction, which is a symptom of having a single proposer who controls access to the state for an extended period makes it nearly impossible for market makers to win the race to cancel their quotes. Even if the market maker is faster, what really matters for who wins the cancel race is who pays the highest bid in the Jito auction. This puts market makers between a rock and a hard place. They can either pay a lot of money to cancel their quote or they can let somebody else pay a lot of money to snipe them. Either way they are losing money so they have to widen out.

In practice today’s on-chain market microstructure is equivalent to giving takers a head start in the race, at least when it comes to the adverse selection imposed on makers. To fix this we need to give apps much more flexibility over how to order transactions. Apps need to be able to give makers a head start in the cancel race if we want to reduce the spread. One way to accomplish this is by introducing a “cancels before takes” ordering policy. We look at the block and we process all the cancels before all of the takes.

We could institute a policy like this today on Solana simply by changing the replay ordering from what the leader decided to a different policy that prioritizes cancels. But this wouldn’t solve the problem. If we still have a single leader that leader can decide to ignore cancelations and we would be right back where we started with a substantial disadvantage to makers in the cancelation race.

The only way to avoid this is to introduce multiple leaders. That way if one leader is censoring cancels you can always submit to a different leader. 

Implementation - Ordering

The biggest question people seem to have about multiple concurrent leaders is how can we merge the blocks of transactions together when there are conflicts. The answer is surprisingly simple. We will have two types of fees: inclusion and ordering. Inclusion fees are paid to the validator who includes the transaction and ordering fees are paid to the protocol (burned). When we want to merge the blocks from the different leaders together we simply take the union of all the transactions in all of the blocks in a particular slot, order them according to their ordering fees and execute them. 

Alone this isn’t enough to give all of the properties we want. What we are after is giving more flexibility to apps to control their ordering. One more ingredient: syscall get_transaction_metadata, allows programs to read the ordering fee of transactions that interact with them, gives apps a powerful tool to control ordering (Robinson and White 2024). 

Implementation - Consensus

We want a few properties from our consensus implementation:

1. Binding & Blinding: Concurrent Leaders cannot include information from other leaders’ blocks in their own block (e.g. sandwiching a private transaction) or cancel their block based on the contents of another leader’s block (e.g. cancel a bid when you observe the other bidder’s bid).

2. Wallclock Fairness: concurrent leaders must commit to their blocks at approximately the same wallclock time. 

This outline presents the most effective scheme we’ve developed so far, in collaboration with Pranav Garimidi and Joachim Neu from a16z Research.

1. Each leader converts their block into erasure coded shreds. Once enough shreds (more than the coding rate) have been recovered, the block can be recovered. Partial recovery is not possible. 

2. The leader sends their shreds to relays in the first layer of the turbine tree. Each leader sends their first shred to relay 1, their second shred to relay 2, … and so on. If everything goes right, each relay receives a shred from each of the leaders.

3. After a timeout, the relays send signed IHAVE messages to a single consensus leader containing information about which shreds they received.

4. The consensus leader then constructs a block with these IHAVE messages; he must include a large enough fraction of the IHAVE messages otherwise the block is invalid. 

5. The consensus leader broadcasts this block to the validators who begin the process of coming to consensus on the block.

This scheme satisfies the blinding & binding properties (with high probability). It has good wallclock fairness properties although it may be possible to come up with a scheme with even better wallclock fairness eventually.

Conclusion

Solana was created to outcompete Nasdaq. To outcompete with Nasdaq we need to offer better prices than Nasdaq. To offer better prices than Nasdaq we need to give applications more flexibility to sequence cancellations before takes. To give applications that flexibility we need to ensure that leaders don’t have the power to unilaterally censor orders. And to ensure that leaders do not have that power we need to ship multiple concurrent leaders.