OVH Data Centre Fire Shows Why Disaster Recovery Plans Are Essential
March 15th, 2021 by Kris Price
The largest European web hosting provider, OVH, suffered a catastrophic fire on Wednesday 10th March destroying one of its data centres in France.
OVH is based in Strasbourg with four data centres in France SBG1, 2, 3 and 4. SBG2 was destroyed by the fire which broke out in the morning of Wednesday 10th March. The fire also impacted SBG1 with reports stating that four rooms in that DC have been destroyed as well. Thankfully, it seems that no one was hurt by the fire. The OVH incident report site states that power supplies to SBG1 and SBG4 will not return until Monday 15th March, whilst SBG3 could be down until Friday 19th. OVH are encouraging affected customers to implement their Disaster Recovery plan.
The Impact
It is estimated that some 3.5 million websites were offline due to the fire, impacting government bodies, banks, and the famous Pompidou Centre in Paris amongst others. Popular online multiplayer game Rust lost 25 servers due to the fire and admitted the data will never be recovered.
Update:
We've confirmed a total loss of the affected EU servers during the OVH data centre fire. We're now exploring replacing the affected servers.Data will be unable to be restored.
— Rust (@playrust) March 10, 2021
OVH are one of the largest hosting companies in the world with state-of-the-art data centres, this fire shows even they can suffer the unthinkable!
Why You Need A Disaster Recovery Plan
OVH’s incident report recommended that customers activate their Disaster Recovery Plan, and Chairman, Octave Klaba also Tweeted the same advice.
We have a major incident on SBG2. The fire declared in the building. Firefighters were immediately on the scene but could not control the fire in SBG2. The whole site has been isolated which impacts all services in SGB1-4. We recommend to activate your Disaster Recovery Plan.
— Octave Klaba (@olesovhcom) March 10, 2021
I wonder how many of those organisations affected actually:
- Had fully tested their Disaster Recovery Plan (DRP)
- Even had a DRP in the first place (looking at the Tweet from Rust I assume they didn’t even have a backup of the data!)
Whilst what happened at OVH is a rare event, it highlights that just because you are utilising cloud-based infrastructure, you still have a responsibility to protect YOUR data. Never assume that because your infrastructure is hosted somewhere else that backups are covered as well. Not all hosting companies provide backup or replication as standard, often it’s an additional extra you must pay for depending on what level of Recovery Point (RPO) and Recovery Time Objectives (RTO) you require.
In the modern world most businesses cannot operate without their IT systems, and the longer the company can’t function, the more money it will lose. A well thought out Disaster Recovery Plan sets a realistic expectation to the leaders of an organisation on how long systems will take to recover in a disaster. And just like testing your smoke alarms, regular DR testing makes sure that everything will work when you need it to.
Backup vs Disaster Recovery
There is a significant difference between data backup and disaster recovery. Backup protects the company data from loss and corruption, whilst Disaster Recovery presents your failed systems (from the last Recovery Point) back to the business. So, for the best protection you need both working in tandem to get your business back up and running with minimum downtime.
Backup & Disaster Recovery as a Service
BrightCloud offer fully tailored Backup and Disaster Recovery as a Service (DRaaS) solutions powered by Veeam, one of the global leaders in data backup and disaster recovery solutions (as recognised by Gartner last year).
Our Backup and Disaster Recovery experts will design and deliver a solution to meet your organisations requirements around RPO and RTO.
By utilising Veeam’s Cloud Connect feature your hosted or on-premise servers (either physical or virtual) can be regularly replicated to one of our three UK data centres. In the event of a disaster, simply failover to your hosted DR infrastructure within one of our data centres and continue working from there. There is no limit to how long your business works from our infrastructure, relieving the pressure on your IT team to resolve the issue that led to the disaster in the first place.
There are two options upon invocation of your DR plan with BrightCloud:
- Rebuild your infrastructure and when ready we’ll help you fail back out of our hosted infrastructure. We keep all your servers up to date whilst they are hosted in our environment. You don’t have to fail back all your servers at once if you want a more gradual transition.
- Remain within BrightCloud’s data centre and sign up to our Infrastructure as a Service (IaaS) offering.
As part of signing up to our DRaaS solution you have the added peace of mind that we conduct regular DR tests and send the reports to you.
I can only imagine the reaction of IT managers across Europe after realising that their data was in SBG2 last week. If you were the IT manager of one of those companies, could you hand on heart say your DR plan would have saved you? If not, perhaps it’s time to talk to us.
