Solana recently faced an outage, which raises questions about client diversity and beta status, marking its first downtime in nearly a year and about half a dozen in the last two years. However, despite this, the blockchain is flourishing with network activity and experiencing a surge in token prices.
Solana Network Outage
The SOL network encountered a substantial disruption on Feb. 6 at 10:22 am UTC, with block production paused for over five hours before validators restarted the network.
This isn’t the inaugural instance of SOL facing downtime. Since January 2022, Solana network has witnessed about half a dozen significant outages and 15 partial or major outage days.
The severity of these incidents varied, ranging from partial to complete network outages, with causes encompassing software vulnerabilities, denial-of-service, and resource exhaustion attacks.
The most recent outage was attributed to a glitch resulting in an infinite loop error, causing validators to stall on a specific block. An infinite loop error causes a program to get trapped in an endless cycle, which, in the case of Solana, hinders validators from verifying transactions as they were stuck attempting to process the same block.
Post-mortem Report
Anza, a Solana-focused software development firm, issued a postmortem report of the recent outage on Feb. 9, detailing its cause and how validators revived the network. The report revealed that SOL’s Just-in-Time (JIT) compilation cache, responsible for compiling all programs before executing a transaction, encountered the glitch.
Initially, the JIT cache was implemented using “ExecutorsCache”; however, with the v1.16 release, ExecutorsCache was replaced by a new implementation called “LoadedPrograms.” The new loader program utilized accounting information stored in a program’s on-chain account to look up its most recent deployment slot and calculate the effective slot height.
However, this loader program encountered a legacy loader issue resulting in the JIT cache encountering a loop with older programs, causing it to recompile them repeatedly instead of efficiently accessing and executing their instructions. This glitch resulted in a network stall, as the system became bogged down in an infinite loop, unable to process transactions or move forward.
Solana’s Response
Head of strategy at the Solana Foundation, Austin Federa, mentioned that there was a planned process to replace the old loader system with the new system, but it was scheduled to be disabled with an updated version. “It looks like someone intentionally called that old instruction set, which hadn’t been used in quite a long time but was still around in the codebase. The JIT compiler ran into issues where it couldn’t find what it was looking for. And that’s what created the infinite loop,” Federa said.
SOL engineers swiftly discovered and fixed the issue, updated the system’s processes, and identified these older programs to prevent future recompilation cycles. This solution entailed reducing the deployment of problematic legacy loaders to ensure that all program instructions could be accessed accurately and effectively without falling into the recompilation trap.
Critics and Effects
Some critics have suggested that a significant factor contributing to the network outages is the lack of client diversity, leading to potential central points of failure. Kadan Stadelmann, chief technology officer of open source decentralized exchange platform Komodo, mentioned that developers and validators prioritize shipping code quickly over maintaining flawless network uptime, adding: “The key reason is centralization and lack of validator client options. SOL’s ambition for high throughput and low transaction times puts immense pressure on its infrastructure, making it susceptible to performance degradation and outages.”
Solana’s Mainnet “Beta” Saga
The Solana blockchain, launched in March 2020, has made significant strides in the decentralized finance (DeFi) ecosystem over the following four years. Solana ranks as the fifth-largest DeFi chain in total value locked, with over $1.7 billion in capital allocated to various on-chain protocols. Its native Solana SOL $107 token has a market valuation of around $49 billion, making it the fifth-largest cryptocurrency.
Despite a remarkable track record, Solana has never officially emerged from its beta phase. The blockchain is still denoted as being in mainnet beta. Beta is a term that refers to nearly finished software being tested by a select group of users before an official launch.
Nevertheless, the Solana blockchain remains accessible to all, hosting numerous decentralized applications utilized for diverse product launches. Within its ecosystem, a thriving nonfungible token (NFT) community adds vibrancy.
| Note: Solana underwent brief phases of both private and public beta testing before introducing v1.0 four years ago. Presently, the network handles a higher transaction volume than most other chains. |
Despite the ongoing evolution of its code, Solana has transcended the realm of what could be deemed beta software. The enduring misconception arises from confusion between pre-launch testing and post-launch software maturity, compounded by misunderstandings surrounding Solana’s messaging.
