Ethereum 2.0's Medalla Testnet Back Online After Multi-Day Outage
Validator nodes are up and running after Ethereum 2.0's testnet went down for three days.
Key Takeaways
- Ethereum 2.0's Medalla testnet is back online following a synchronization error that brought down the network.
- Testnet node participation rates are back to roughly 70% of pre-crash levels.
- Developers are designing a more detailed strategy in case the network must respond to future issues.
Share this article
Ethereum 2.0’s testnet is back online following a multi-day outage, according to current network statistics.
Medalla is saved!
INFO blockchain: Synced new block block=0x68a46008… epoch=3428 finalizedEpoch=3424 finalizedRoot=0xf0e6b2fe… slot=109700
— prestonvanloon.eth (@preston_vanloon) August 19, 2020
Outage Comes to an End
On Aug. 14, Ethereum 2.0’s Medalla testnet crashed due to a Cloudflare outage that affected node synchronization. This issue prevented nodes from validating blocks and processing transactions.
One of the crucial issues was the dominance of the network clients called Prysm and Lighthouse. A diversification between clients is necessary to keep the network intact if some client breaks.
The issue wasn’t addressed, which led to almost complete loss of the network’s participants during August 15-16. The nodes running the popular Prysm clients jumped four hours forward and started to make blocks for the chain that didn’t exist.
It turned out Prysm had a design flaw. The client relied on the third party, Cloudflare, to fetch the time. Meanwhile, Cloudflare’s time provision service errored and messed up clocks for a large part of Medalla’s nodes.
The problem was compounded when developers introduced an emergency fix that conflicted with Cloudflare re-enabling its time service. The testnet “could have recovered on its own if we had not acted at all,” Ethereum developer Raul Jordan explains.
This outage caused participation to drop from roughly 80% to 5% over the course of a few days.
As of Aug. 20, the testnet is nearly back to full participation according to 24-hour stats, with just under 75% of nodes online.
What Does This Mean for Ethereum 2.0?
Though some see the outage as a bad sign for Ethereum 2.0’s integrity, Jordan argues that testnets are meant to catch issues at an early stage before they affect real users and funds.
“It would have been really terrifying if this bug occurred with real money at stake once eth2 launched,” Jordan notes. Presumably, the outage and subsequent bug could have delayed transactions and driven down ETH prices. Jordan says that this event has led the Ethereum team to develop a more detailed strategy.
Others are concerned that Ethereum 2.0’s node software relies on a centralized time service (Cloudflare’s Roughtime) for synchronization. However, other methods do not necessarily prevent outages: Steemit, IOTA, and Chainlink have experienced recent downtime despite relatively distributed time systems.
In any case, the fact that Ethereum 2.0’s mainnet is just months away is cause for concern. If there are future issues, Ethereum 2.0 may be delayed further to ensure a safe launch. Standard ETH transactions were not affected by this week’s outage.
Share this article
