Is measuring blockchain transactions per second (TPS) stupid in 2024? Big questions

For years, TPS, or transactions per second, has been a metric used by blockchain developers to hype up their fraudulent new networks.

Compared to Bitcoin's maximum of seven transactions per second (typically around four) for years, Ripple XRP claimed to be able to handle 1,500 TPS of Visa transactions per second, although Chief Technology Officer David Schwartz admitted last year that it had never actually hit that figure.

Solana transactions are said to be 65,000 TPS per second in benchmark testing, although in the real world they are closer to 3,000 TPS – and the “real TPS” measurement is again much lower. Even new blockchain networks can process up to 297,000 transactions per second – “theoretically” of course.

But how much stock should one place in the scale?

Various blockchain industry leaders today argue that the advent of transaction packs makes TPS a pretty flawless measure of blockchain performance, and that TPS can be — and often is — a game.

But the reason it's still the most common measure may be because there's no better option.

Blockchain transactions increase and limit per second measurement

In the early days of crypto, TPS was the only metric that mattered, as blockchains like Bitcoin and Litecoin mostly sent transactions from one address to another — like when Laszlo Hanyek spent 10,000 BTC to buy two pizzas in 2010.

At the time, TPS helped users understand how different block size choices or cryptographic algorithms from potential forks or chains could affect the speed at which their transactions could be processed. “Scaling” was mainly a debate about how much TPS the blockchain could handle, which was important if crypto was to become the world's digital currency.

When Ethereum entered the scene in July 2015 with 13 or more TPS and programmable smart contracts, many operations became more computationally intensive than simple transactions.

Linkage of account draft and blockchain transactions

Years later, account aggregation allowed users to group multiple actions (called UserOps) into a transaction package, further expanding TPS's growing blind spot.

Offchain Labs co-founder Steven Goldfeder explains that focusing on the raw TPS number is “counting the number of bills in your wallet but ignoring that some are singles, some are twenties, and some are hundreds.”

Some transactions may have higher computational value than others, although under TPS, they are all treated as one and the same.

Offchain Labs is the developer of Arbitrum One, an Ethereum layer-2 scaling solution that aims to provide a scalable environment for decentralized applications (DApps) and smart contracts. The native account summarizes and averages about 9.95 TPS – although it claims to be capable of 40,000 TPS.

Solana is struggling with the ever-increasing complexity of marketing, said Austin Federa, chief strategy officer.

“Solana is probably at least five times faster than when I joined. But you won't see because […] The complexity of the transaction has increased significantly.

A simple transaction like sending a consensus vote or SOL to another user has low computational cost, while creating an arbitrage or NFT can be “100 times more computationally intensive.” Both are considered the same under TPS, Federa explained.

“So Solana is processing more complex transactions today than it was in 2021 – even if the number of transactions per second has not increased astronomically.

But according to their marketing, Solana's TPS is already astronomical, causing controversy.

Solana Transactions Per Second Claims: Exaggerated or Misunderstood?

Solana's marketed TPS has drawn industry criticism, with detractors taking issue with Solana's claimed 65,000 TPS in “benchmark testing” while the white paper shows a theoretical 710,000 TPS on a 1-gigabit per second network connection.

The solana.com website currently has a live transaction feed of nearly 3,000 per second. Critics argue that 80%-90% of transactions are made by non-users.

Federa argued that the criticisms were unwarranted. “Voices paying their way are the real deal on Solana,” he says.

“Some people say, ‘I don't want to count votes. Votes in other networks will not be counted.' Ok, I take your point. Let's see the actual TPS number then.

Staking tool provider Solana Compass currently lists 704 “true TPS” for its network – consensus does not include voice transactions.

“The exact number of TPS reported by Solana Compass is completely designed to destroy it, and this number is still 10x what the nearest blockchain can do,” he said.

“Then if they look at that and say, ‘Oh, well, it's all spam from X or Y or Z,' right then, we're not dealing with serious people anymore.”

Blockchain Transactions: How to Fill TPS

American author and humorist Mark Twain once famously said, “There are three kinds of lies: lies, damned lies, and statistics.”

Neil Davis, System Performance Scientist at Input Output – the firm behind Cardano – this is indeed the case with the use of TPS in the industry today.

“People like to grasp ‘easy' ideas,” Davis explains.

“Comparators are only valuable if the amount they report is a good proxy for the performance characteristic of their actual interest.”

Unfortunately, he argued, “bragging rights” appear to be the most popular driver of development.

Davis also criticized the chains for considering it as part of their TPS measure.

“Such ‘transactions' are not a proxy for any end-user activity – they seem to be trying to do more ‘virtue' than themselves.”

Offchain Labs' Goldfeder agrees:

“Oh, TPS, they say my chain can do 1,000 TPS, then there is an amazing sign to say […] They did very basic transactions like no-op. It could be a transaction that does literally nothing, or a transfer of the original asset.

I'm having a crypto mid-life crisis.

Benchmarks for comparing blockchains are terrible.

TVL, voice, TPS, users – all easy to play

Latency, fees, number of nodes, number of startups seems ok, but not perfect.

As a result, you should only use quality measures

aka vibes

– because | helius.dev (@0xMert_) November 23, 2023

“Every metric and blockchain and everything plays a certain role,” Solana Federa said. So be skeptical of high numbers.

“Solana is such a mature network that no one is sitting there sending transactions to make the numbers look good. That's not the case with other networks.

But if TPS is played often and isn't an example of capacity, are there better options?

Alternatives to blockchain TPS: UserOps in sec

Anthony Rose, senior vice president of technology at Matter Labs – the developers of zkSync – believes that UserOps per second can be a “more meaningful metric” but the community admits that it is a “not very good” metric.

The Ethereum token standard, known as ERC-4337, introduced “pseudo-transaction” objects called user operations. They act as instructions to tell the smart tag what action to take on behalf of the user.

TPS is a very inaccurate metric for measuring blockchain flow. It will only get worse in the future. One possible option is to measure UPS – User Ops per second. But what are they, how do they relate to AA, and is this really an alternative? 🧵👇 pic.twitter.com/wTx16AJra7

— bartek.eth (@bkiepuszewski) October 31, 2023

For example, a simple exchange from one token to another on a decentralized exchange normally requires two separate transactions. In accounting summaries, these UserOps are grouped together. Unfortunately, TPS still sees them all as one and the same transaction.

“As we see more adoption for accounting and this improved UX, TPS will become more and more of a useful metric,” says Rose.

However, there's a big flaw in Users' Second: The scale doesn't really work outside of the Ethereum ecosystem. Federa is not a fan, it rewards overly complicated products.

“It's like counting the number of visitors to a website by looking at how many HTTP requests there are,” he says.

Typically, a well-optimized website makes fewer HTTP requests, allowing the site to load faster.

“Like, YouTube is pulling 60 HTTP requests, while Wikipedia is like two or three.”

“And so, that's where the user's operations actually break down on a per-second scale. Rewards complex products.

Gas per second (GPS) as a blockchain transfer per second replacement

“We all move to the same gas every second,” said Avihu Levy, Chief Product Officer of Starkware.

Users pay gas to run smart contracts on the Ethereum network, so this metric is a way to indicate the computational performance of a system.

Gas per second or GPS blocks time to measure the volume and type of transactions as well as consumption. Levy argues that it is the closest measure to what a network can do in terms of computations or resources used per second.

That is only valid for EVM chains having the same gas schedule.

— bartek.eth (@bkiepuszewski) November 1, 2023

“So if I know that my network can do the average computation for one second, I can try to find comparisons between other networks that can do different levels of computation.”

The downside, Levy admits, is that networks measure computation differently.

“So if you're talking about EVMs using gas per second, it's a very good metric because they're all measuring the computational complexity in exactly the same way.”

Starknet, however, measures computing by Cairo standards. Solana uses “compute units” while Aptos, another layer-1 blockchain, uses gas units.

Levy explains that for all of these non-Ethereum ecosystem blockchains, the challenge is to create a “canonical benchmark” to accurately scale various computational parameters.

The difficulty of making standardized comparisons between different, complex blockchains and projects is perhaps the reason why TPS remains the top dog even with all its shortcomings.

Also read

Main characteristics

Beyond in-game assets: Blockchain Gaming, DAOs, Guilds and Ragequitting

Art week

Connecting the Dots: Collection and Collaboration in the World of Crypto Art

The future of blockchain transaction performance metrics like TPS

While he's not a fan, he says TPS looks like it'll be around for a while because most of the market isn't good enough to understand more sensible metrics.

“It's definitely a flawed measure. […] “I don't think there's any strong opposition,” Rose said, though he admitted that TPS is still “a place where a lot of people are incarcerated.”

“It's the way people understand it. [unlike] Significantly more meaningful parameters […]. People do not yet have a deep understanding of them.

“There is a trade-off between explanation, comprehensibility and then the quality of the measurement,” he added.

Rose says Mather Labs still uses the benchmark to see how design changes affect performance in internal test environments.

However, with the adoption of account aggregation on Ethereum, Rose said a move to UOPS “makes perfect sense.”

Meanwhile, Davis said TPS will remain active until players begin using blockchain as more than simple “stores of value.”

“If you're guessing[in blockchains]TPS is only worth as much as it ‘shills' your investment,” he says.

However, Federa thinks that once the technology is introduced quickly enough, TPS may become irrelevant for most users — just like the case with phones and computers today.

“The success of the personal computer industry is that you don't have to worry so much about what computer is in front of you.”

“We don't have a blockchain yet, but I hope we'll get to a point where all blockchains work quickly enough that the only people who need to worry about this are developers,” Federa said.

Subscribe

A very engaging read in Blockchain. It is given once a week.

Felix Ng

Felix Ng first started writing about the blockchain industry through the lens of a gambling industry journalist and editor in 2015. Since then, he has moved on to cover the blockchain space full-time. He is very interested in blockchain technology, an innovation aimed at solving real-world challenges.