Solana, one of the largest blockchain networks by market value, came dangerously close to a full halt on Wednesday after a routing glitch at a major data center provider knocked nearly 29% of its staked tokens offline, according to staking platform Marinade Finance. The incident triggered fresh concerns about the network's resilience and its reliance on a small number of infrastructure providers.
Because Solana stops finalizing transactions if more than one-third of staked tokens go offline, the network was left within roughly 20 million tokens of a freeze similar to the five-hour outage it suffered in February 2024. That earlier outage, caused by a bug in the transaction scheduling logic, froze block production and forced validators to coordinate a manual restart. Wednesday's close call was different in origin but equally alarming to many in the crypto community.
What Went Wrong
The glitch was traced to a bad internet route originating at Teraswitch's Miami facility and spreading to data centers in Europe and Asia, according to Marinade. Teraswitch, a relatively low-profile connectivity provider, controlled more than a quarter of all staked tokens on Solana at the time of the incident. That concentration meant a single routing error at one facility could disrupt a substantial portion of the network's validator set, pushing it toward the critical threshold for a chain halt.
Solana validators rely on low-latency connections to receive and broadcast messages, including votes on the state of the ledger. When a large number of validators are cut off from the network, their votes are not counted, and the chain may not achieve the supermajority needed to confirm new blocks. In this case, the network did not stop, but it came close enough that Marinade, a leading liquid staking protocol, felt compelled to issue a public alert.
Foundation Defends Network Stability
Solana Foundation executives responded by stressing that blocks and transactions never stopped during the incident. They argued that the network held up under stress, and that the design of Solana's consensus mechanism, which uses a proof-of-stake system with a tower of votes, was resilient enough to absorb the loss of a significant fraction of the validator set without a full freeze. The Foundation also pointed out that the validation process continued, and that no user funds were at risk.
Still, the Foundation acknowledged that the incident highlighted the importance of validator diversification. In a statement, it encouraged stake holders to spread their delegations across a wider range of validators and data centers, reducing the risk that a single point of failure could again bring the network to the edge of a halt. The Foundation also said it was working with infrastructure providers to improve routing redundancy and monitor for similar issues.
Understanding Solana's Freeze Threshold
Solana's consensus algorithm requires at least two-thirds of the staked tokens to be online and participating in consensus to finalize new blocks. If more than one-third of the staked tokens go offline, the network cannot achieve the required supermajority, and block finalization stops. This is not an automatic chain halt; rather, the network continues to produce blocks, but those blocks are not finalized, and the chain may become vulnerable to reorganizations if validators disagree on the canonical state.
In severe cases, the network may need to be restarted, as happened in February 2024. That outage, which lasted about five hours, was caused by a bug in the validator client that caused some validators to produce conflicting blocks, leading to a loss of consensus. The community ultimately coordinated a restart using snapshot data, and the network recovered.
Wednesday's incident was different because it involved a network-level failure rather than a software bug. But the near-miss raised questions about whether Solana's infrastructure has become too concentrated. According to Marinade, Teraswitch controlled roughly 29% of all staked SOL at the time of the glitch, a level of concentration that many observers described as alarming. Other large data center providers, such as Hetzner and OVHcloud, also host significant portions of the validator set, creating similar risks.
Backlash and Calls for Decentralization
The incident reignited debates about Solana's decentralization, a topic that has followed the network since its launch. Critics argue that Solana's high-performance design, which requires substantial hardware and low-latency internet connections, naturally leads to a concentration of validators in a few professional data centers. That concentration makes the network more efficient but also more vulnerable to outages at a single provider.
In response, some community members have called for the Solana Foundation and other stakeholders to incentivize a more geographically distributed validator set. Proposals have included adjusting staking rewards to favor smaller validators, supporting data centers in underserved regions, and implementing protocol-level changes that reduce the need for ultralow latency. The Foundation has said it is exploring these options, but there is no consensus yet on how to achieve greater decentralization without sacrificing performance.
Looking at the Broader Context
Wednesday's near-freeze is not an isolated event in the crypto industry. Other blockchain networks have faced similar challenges, particularly during periods of high market volatility or when infrastructure providers experience outages. Ethereum, for example, has had its own issues with network stability, though its 2.0 upgrade and the rise of liquid staking protocols have helped distribute validators more broadly. Meanwhile, newer networks like Aptos and Sui have explicitly designed for high throughput with in mind, but they have not yet faced the same level of network stress as Solana.
Solana has also been under the spotlight for its past outages. In addition to the February 2024 freeze, the network suffered several major disruptions in 2021 and 2022, sometimes triggered by bot attacks or transaction flooding. Each time, the Foundation and the community managed to restart the chain, but each incident chipped away at confidence in Solana's reliability. Despite those challenges, Solana has continued to grow, with a thriving ecosystem of DeFi protocols, NFT marketplaces, and other applications.
The latest incident was detected early enough to avoid a full halt, and the network resumed normal operation shortly after the affected validators came back online. Marinade reported that the percentage of staked tokens offline gradually decreased as the routing issue was fixed. Still, the incident served as a stark reminder that even a highly performant blockchain can be brought to the brink by a single infrastructure failure.
Implications for Solana's Future
The near-freeze could have implications for Solana's reputation among institutional investors and developers. In recent years, Solana has attracted significant attention as a faster, cheaper alternative to Ethereum, and a number of major projects have built on the platform. But institutional investors, in particular, tend to be risk-averse, and a network that nearly halts due to a routing glitch may raise red flags.
On the other hand, the fact that the network did not halt, and that the Foundation remained transparent about the incident, might help build trust in the long run. The Foundation's quick response, including public statements on social media and coordination with validators, demonstrated that the team can manage stress events effectively. Some analysts also noted that the incident could accelerate efforts to decentralize Solana's validator set, which would make the network more robust in the future.
What Validator and Stakers Should Do
For validators and SOL holders, the incident underscores the importance of diversification. Validators are encouraged to run nodes across multiple data centers and internet service providers, and to ensure that their operations can withstand a single point of failure. Stakers, meanwhile, can choose to delegate their tokens to validators that are geographically dispersed, using services that provide transparency about where their nodes are hosted.
Liquid staking protocols like Marinade also offer a way for stakers to diversify their exposure, as they pool funds across many validators. Marinade, in particular, has been vocal about the need for decentralization, and its alert on Wednesday was part of that effort. The protocol's dashboard allows users to see the distribution of stake across validators, and it has implemented policies that limit the amount of stake any single validator can receive.
In the wake of the incident, some validators have already taken steps to move their nodes to different providers, or to set up backup routes in case of another routing failure. The Solana Foundation has also published a best-practice guide for validators, recommending that they use multiple upstream internet connections and avoid relying on a single data center.
A Familiar Pattern
Observers of the crypto industry might find the pattern familiar. Many blockchains have encountered infrastructure-related close calls that prompted soul-searching about decentralization. In 2021, a major outage at AWS affected several crypto services, and in 2023, a Cloudflare issue caused problems for some Web3 platforms. Each time, the industry has acknowledged the risks and vowed to improve, but progress has often been slow.
Solana's situation is particularly notable because of the network's emphasis on speed and performance. To achieve throughput of tens of thousands of transactions per second, Solana requires validators to be closely synchronized, which favors large, professionally managed data centers. This creates an inherent tension between decentralization and performance that the network may never fully resolve.
Nevertheless, Wednesday's near-freeze shows that the network can withstand a significant shock. The fact that it came within 20 million tokens of the threshold, but did not cross it, suggests that the system's safety margins are thin but present. It also suggests that further concentration of stake, or a larger infrastructure failure, could have a very different outcome.
Reactions from the Community
The crypto community reacted to the news with a mixture of concern and relief. On social media, many users praised Marinade for its quick alert, while others criticized the Foundation for downplaying the severity of the incident. The Foundation's claim that the network held up under stress was met with some skepticism, given that 29% of staked tokens were offline. However, others pointed out that the network had not halted and that transactions continued to be processed, which is evidence of resilience.
Some validators also shared their own experiences, describing how they were affected by the routing issue. Some reported losing connectivity for several minutes, while others were able to continue participating through backup routes. The incident highlighted the technical challenges that validators face in maintaining reliable connections, and it may prompt more validators to invest in redundant infrastructure.
Marinade, for its part, has been a leading voice in pushing for greater decentralization on Solana. The protocol's founders have repeatedly called on the Foundation to take a stronger stance on validator distribution, and Wednesday's incident gave new ammunition to their arguments. In its alert, Marinade specifically urged stakers to re-delegate their tokens away from concentrated data centers, and to support validators that promote network health.
What Happens Next
The immediate crisis is over, but the broader questions remain. Will Solana's validators become more dispersed? Will the Foundation take concrete steps to reduce reliance on single data center providers? And will the network's performance suffer if it becomes more decentralized? These are open questions, and the answers will shape Solana's future.
For now, the network is operating normally, and the price of SOL has shown little reaction to the incident, suggesting that the market has largely shrugged off the near-miss. But for those who follow the blockchain infrastructure space, the event is a cautionary tale about the fragility of even the most advanced networks. It also serves as a reminder that the health of a blockchain is not just about code, but about the physical and networking layers that support it.
As Solana continues to grow, it will face more tests. The network's high performance and low fees have made it popular, but those same attributes require infrastructure that can be susceptible to failures. The lessons from Wednesday's close call will likely be studied by engineers and protocol designers for some time, and they may lead to changes in how Solana, and other blockchains, approach network resilience.
In the meantime, validators and stakers would be wise to take note of the risks that Marinade highlighted. Diversification is not just a buzzword; it is a practical way to reduce the likelihood of a network-wide failure. And as the old adage goes, have backups in place before you need them, not after.
The Solana network has survived many challenges in its relatively short life, from market crashes to validator bugs. Wednesday's routing glitch was another close call, but it may ultimately serve as a catalyst for positive change. If the incident pushes the community to address concentration risks, then the network could come out stronger in the long run. But if those warnings are ignored, the next near-freeze might not be a near-miss.
Source: Coindesk News