Ensuring website resilience: A comprehensive Disaster Recovery approach for financial institutions

In the wake of the recent network outage at Data Action, which impacted several Australian banks, the imperative for robust disaster recovery (DR) processes within financial institutions comes sharply into focus. In this article, we explore the intricacies of Sitback’s Disaster Recovery process and share insights into best practices for safeguarding websites against unforeseen disruptions.

The Data Action outage

The recent disruption experienced by banks, including Ubank, Bank Australia, Defence Bank, and Beyond Bank, due to a network outage at Data Action, underscores the vulnerability of relying on external service providers. Customers of the affected banks faced challenges with online transfers, mobile banking apps, and the use of Osko – a secure payment service.

This naturally resulted in considerable damage to brand reputation. Not only were customers left unable to manage their finances, in some cases they faced penalties from other companies whose products and services they subscribe to when payments were unable to be processed.

The incident raises critical questions about the resilience of online banking services and prompts financial institutions to assess their own disaster recovery strategies in the face of unforeseen challenges.

Disaster Recovery best practices

Understanding the nuances of a robust Disaster Recovery process is paramount for financial institutions looking to fortify their online presence. At Sitback, we offer a tiered approach to DR, addressing varying needs and priorities.

• Hot-DR: Suited for high-profile clients requiring a 99.99% SLA, this level employs automated failover and the blue-green deployment method for geolocation switching. The emphasis is on continuous availability, acknowledging the reality that achieving 100% uptime is an elusive goal.
• Warm-DR: With an automated failover to a static webpage and a 4-hour Recovery Time Objective (RTO), this level ensures fast recovery. Infrastructure-as-code facilitates the provisioning of a new environment, providing flexibility and efficiency, without the cost implications of maintaining two identical production-ready hosting environments.
• Cold-DR: Offering a 24-hour RTO, Cold-DR involves rebuilding and deploying the last working Recovery Point Objective (RPO) data. This level is geared towards scenarios requiring a comprehensive recovery approach, but where budget is limited, the website content doesn’t change often, or the website isn’t business-critical.

Best practices for effective Disaster Recovery include the integration of automated monitoring tools such as New Relic and Azure Alerts which notify you immediately should your website become unresponsive. Regular testing of DR processes during production environment code deployments or on staging environments ensures readiness and identifies potential gaps.

Insights from Lifeline Australia’s experience

A case study involving Lifeline Australia sheds light on the real-world application of a Hot-DR solution. Lifeline, a provider of life-critical support services, faced a nightmare situation when Azure, Microsoft 365 and Power Platform services in Sydney (where their website was hosted) went offline for up to 46 hours due to several consecutive issues stemming from insufficient data centre staffing levels.

Multiple well-known brands across Australia, including BoQ and Jetstar, were also hosted within this region and were unprepared for this. Consequently, it resulted in website and app downtime for many of these brands, causing significant negative impact for their users.

Sitback had architected a Hot-DR solution for Lifeline, featuring multi-region failover and auto-scaling set up within load-balanced environments. As soon as the outage occurred, we were notified of the issue through our monitoring tools, and the automated process kicked in, routing all traffic to a second instance of the website hosted in the Azure West region which was unaffected by the outage.

This strategic approach ensured Lifeline’s website remained accessible when the Azure East region stack went down as a result of cooling and power failure, showcasing the practical efficacy of a well-designed Disaster Recovery process and ensuring that Lifeline’s vital support services remained online.

Addressing concerns and key considerations

For financial institutions navigating the aftermath of the Data Action outage, understanding preparedness for future disruptions is a key concern. It is crucial to learn from these episodes and to assess and refine existing DR strategies in line with the evolving landscape.

For banking organisations looking to update their existent website with a more robust and resilient solution, we have created a website accelerator platform to optimise the process.

Banking website accelerator platform

Sitback’s Banking website accelerator platform is a game-changer for financial institutions, offering a streamlined SaaS solution specifically designed for banks, credit unions, and mutuals. Key features include:

• Seamless integration: The platform integrates effortlessly with financial calculators, third-party apps, and CRM systems, providing a modern online branch for the brand.
• Speed to market: Avoid building from scratch; the platform offers out-of-the-box features like web forms, rates management, product-centric Information Architecture, and atomic disclaimers.
• Low-risk SaaS: Immediate updates, low upfront investment, and flexible subscription models make it an ideal choice for budget-conscious organisations.
• Focus on customers: Optimised for the banking sector, the platform ensures a secure, compliant, and user-friendly online experience, allowing teams to focus on customer-centric features.

Most relevant here though, is that the platform has been designed to maintain as high an up-time as possible. Either Sitback can provision infrastructure on your behalf according to your Disaster Recovery requirements, or the website platform can equally be hosted within you own corporate infrastructure – we will happily advise on the recommended set up to best accommodate the platform based on agreed RPO and RTO limits.

Closing thought

In the ever-evolving digital landscape, the recent disruptions underscore the need for financial institutions to reevaluate and fortify their Disaster Recovery processes. As organisations strive for continuity, customer trust, and brand integrity, a well-thought-out Disaster Recovery strategy emerges as a fundamental pillar for sustained success in the digital age.

If you’re interested to learn more about DR strategies and the security precautions that can be employed to mitigate the risk of website failures, read more in How to start a Disaster Recovery Plan for your website. Good luck, and as always – reach out if you need a hand.

Featured photo by Christina Morillo.