How to design software to fail

According to an Opengear study, 38% of U.S. firms lose more than $1 million due to network outages every year. During 2020, a year where the operational efficiency of remote work technologies was more critical than ever, major companies like Google, Zoom, Slack, and Microsoft all saw major outages across their platforms.

To avoid downtime that can cost a significant amount of money, hurt relationships with users, and tarnish brand reputation, companies need to design their software to fail. 

Designing software to fail means designing to automate software restoration and solutions. These safeguards help avoid massive service interruptions—and keep design teams in an agile, solution-oriented mindset. Let’s walk through some strategies for designing software to fail. 

Why is it important to design your software to fail? 

Regardless of your design process, there are components outside of our control that will fail. That’s why designers and companies need to prepare to avoid downtime and effectively manage it when it does happen.

Think about the last major outage you experienced with your favorite software. From email providers to workflow tools, or even messaging applications, these outages occur all the time—and when they do, we’re always surprised at how massively disruptive they are to our day-to-day work. Users immediately head to down detection websites or to social media channels to learn more and even complain about the outage. 
In this article, we’ll walk through how to design your software to fail. 

How to design software to fail

There are five key building blocks to designing software to fail and recover quickly: 

1. Build redundant components

Redundancy is the duplication of critical components or functions in a system to increase the system’s reliability. Think of it like building a fail-safe and then building a fail-safe to that fail-safe, again and again. 

It’s crucial to never leave the major functionality of your design reliant on a single component. Instead, build redundant cloud components, ideally with minimal or no common points of failure. 

2. Set up automation 

To mitigate against major outages: Test, test, test, and test some more and automate.

By automating the software build, promotion and release processes, companies can better control software development and reliably scale software production—and leave less chance for error. To accomplish this an increasing number of companies need to invest in automation engineers to automate business, IT, and development processes. 

3. Plan for scalability  

When designing to fail, you should also plan to scale. The two principles go hand in hand. Companies scale design efforts to meet customer demand or scale hiring to meet the needs of the business; engineers also build in scalability and elasticity into the software. Building scalability into your systems allows your software to accommodate higher workloads and elasticity gives your system the ability to adjust resources to adapt to different loads dynamically usually in relation to scaling out.

In theory, each version of an app or product is a better version than the last and better able to meet the demands of its users. Scalability is essentially an increase in capacity. If your team is building modular or redundant components, then you will almost certainly have a bottleneck or issue somewhere in your product, given that fallibility is inevitable in software development.

Any shared resource in your network is a potential point of failure that will limit your scalability at best and cause a cascading set of problems at worst. When you plan for scalability, you’re also preparing for these bottlenecks to occur.

4. Focus on reliability

Knowing that software and cloud services failures are inevitable, the focus can shift to containing and recovering from those failures quickly to boost reliability. Engineering practices like fault modeling and fault injection are necessary elements of a continual release process that builds more reliable software and cloud systems.

5. Build with elasticity 

Some days will place more demands on your software, app, or cloud platform than others. By building in elasticity, you can increase or decrease the scalability or capacity of the system by adjusting the number of deployed services. 

If you’ve also set up automation as previously discussed, you can create a reactive system that adapts to changes in demand or load automatically. With this type of elasticity, flexibility, and reactivity in place, you can avoid failures due to system overloads. 

In a world that requires more flexibility and agility than ever, planning for failure is key to success. Resiliency is more valuable than perfection. Failures will happen, but the tools and systems that are built to minimize disruption will boost reliability—and increase consumer trust.