An unusually wet morning in April 2017 proved a nightmare event for ARL. Just before 8am as the business was gearing up for a busy day ahead, a torrential downpour of hail and rain resulted in a ceiling collapse, causing water to pour into the server room and douse critical infrastructure.
Four racks housing the firewall, file servers, support for 17 virtual environments, telecoms equipment, and two critical production and development servers went down, despite redundant hardware. Electricity was quickly cut to the site but the damage was done. Essential computing equipment would not refire and storage arrays were deemed unrecoverable.
There is never a good time for an event like this to occur, but only a few days out from the end of the month and with a large number of critical financial and marketing transactions pending for the following day, it was potentially devastating.
“Our suppliers are some of the heaviest hitting brands in the world and we have commitments to pay them on a certain day. We also have to ensure that the goods ordered during catalogue drives by our members are delivered and ready for sale as advertised. These are service level agreements for the business, so this event really put a lot on the line,” said Kevin Simionato, Information Systems Manager for ARL.
As the saying goes, it pays to plan for a rainy day and in this situation, ARL was able to put its pre-existing Disaster Recovery plan into action, calling on trusted partners Truis to move at speed. The call went out in the morning and within an hour, teams at ARL and Truis began deploying countermeasures to ensure the business would have IT back up and running vital transactions as quickly as possible.
