Earlier today, the Filecoin network was brought to a halt due to a bug, one which led to most nodes being stuck at height 336459 with mismatched state compute errors due to indeterminate behavior. The bug in question was quickly identified, however, with the team able to get the network back and running in just four hours.
The fix was an update to the chain called Lotus v1.4.0 which Filecoin quickly released and that miners were required to update to. The network had restarted and progressed to the expected block height 337120 at 3:20 pm PT.
In an incident report, Filecoin revealed that some degraded chain quality was to be expected since miners were still upgrading and power had not been fully recovered yet. At the time of writing, only 34% of miners had upgraded and BaseFee was volatile on the network.
Reportedly, a conservative estimate suggested that a full recovery of the chain would be completed at epoch 338520 on 20 December at 11:00 UTC.
While Filecoin did state that the chain is expected to be fully functional before this time, it still advised exchanges, custodians, and wallets to disable transfers until such time, adding,
“Our recommendation for you would be to keep transfers out disabled until that time or until you gain comfort with chain status — at least for larger amounts. Your own internal transfers, trading, and other fully off-chain activity should be fine.”
Following these developments, KuCoin, HBTC, and several other exchanges announced that $FIL deposits and withdrawals were temporarily closed.
Today the #Filecoin network fell apart due to some dormant bug. Took the team about 4 hours to get the network back and running. Obviously will be screwy until enough nodes update and some people do get penalized. Nonetheless, that's an amazing turnaround time. Congrats!
— Péter Szilágyi (@peter_szilagyi) December 19, 2020
While the network is still recovering, many, including Ethereum developer Péter Szilágyi, praised Filecoin for the quick response time with respect to the identification and resolution of this issue.
