Why incident response plans fail

Following a cyber attack on critical infrastructure, emotions run high and the clock starts ticking. Suddenly what appears to be a well-structured incident response (IR) plan on paper can turn into a confusing “storming session” around who owns what.

Rather than identifying, analyzing and eradicating the threat, organizations can easily become entangled in processes hindering response time and further endangering operations.

The longer the “dwell time,” or the time an attacker remains within the system, the more damage the attacker can cause, whether it be data loss, impacts to operations or physical damage to assets. According to a recent survey done by the SANS Institute, 50 percent of organizations took two days or longer to detect breaches, and 7 percent didn’t know the length of an attacker’s dwell time.

While many industrial organizations have an IR plan in place, very few run through a routine simulation exercise of this plan. Simulated exercises reveal various incorrect assumptions made throughout the IR process and identify gaping holes where there are missing contacts or protocols that are critical for a successful IR program.

Additionally, organizations are beginning to see cybersecurity ramifications beyond damage to assets, data and reputation. Standard & Poor’s has threatened to downgrade the credit ratings of banks that have poor cybersecurity, and more state and federal laws are being passed that require organizations to implement reasonable security safeguards.

These “reasonable security safeguards” take into account IR plans and dwell time after incidents. It’s not enough for companies to have plans in place, they must demonstrate that their plans are effective in mitigating risks around cyber incidents.

Attacks are part of today’s connected environment.

In a recent IR simulation held by an oil and gas company, I was able to participate in the capacity of an industrial controls supplier. The simulation challenged assumptions on roles and responsibilities. In the oil and gas industry, the supplier ecosystem is complex. Using an offshore project as an example, a company that provides a control system for an offshore platform may first sell the system to an engineering procurement and construction company.

The control system may then be operated by a fuel services company and ultimately owned by an oil producer. The organizational complexity in upstream is massive. As a result, it’s critical for these organizations to break down any assumptions about IR, and both assign and confirm ownership as part of this process.

While every assumption should be tested in an IR exercise, there are a few top considerations that must be transparent for IR to be truly effective. These include:

Line of communication

When an incident occurs, key stakeholders want to be aware of what’s happening and how the situation is being addressed. Keeping executives in the know and managing expectations around the line of communication is an important part of an IR plan. There should be an assigned “incident captain” who can quickly alert the necessary parties and inform them of immediate next steps.

This is particularly crucial when an incident impacts IT, field teams, multiple business units, global regions and suppliers. Time is of the essence, and a simulated exercise ensures the communication plan is clear, accurate and has the necessary contact information at the ready to bring awareness to all stakeholders.


When it comes to managing suppliers in an IR plan, there are a number of questions or assumptions that should be verified during a simulated exercise. What role do your suppliers play in the event of an attack? Do they have a contractual agreement that outlines their role in IR and disclosure around cyber incidents? Do they install software that was purchased from another vendor? Do suppliers know what software you have in operation? Do they run simulated testing of software updates on machines prior to actual implementation?

It’s not enough for companies to have plans in place, they must demonstrate that their plans are effective in mitigating risks around cyber incidents.

This last question is critical for operational technology environments that don’t regularly shut down and restart for software updates. Further, according to a recent study, only one-third of companies are confident they know the exact number of vendors accessing their systems; the average company’s network is accessed by 89 different vendors every week. An IR plan should incorporate all necessary information about who has access to networks and what role suppliers will play in response to cyber attacks, including how they should be communicated with and how they can help mitigate the risk.

Operational flexibility

If a control system’s human machine interface (HMI) went down as the result of a cyber attack in an exploration and production (E&P) operation, which systems could continue to operate despite the handicapped position of the industrial automation system? When distributed control systems (DCS) are down, an organization can operate machinery from the control panel.

This isn’t a simple solution, however, and leadership must consider whether there is an operator on every shift who is qualified to operate a generator control through the panel and not through the DCS. Organizations must also consider whether there is something they might lose that requires connectivity to operate, causing disruption to operations. Running through scenarios as part of an IR exercise will help companies determine what type of operational flexibility and resiliency they have, and what steps they must take to improve upon it.

Attacks are part of today’s connected environment, so IR is not as much about the attack but rather resiliency. Cybersecurity practices need to be collaborative and open, not only within an organization but across industries. Executives should be thinking about how they inventory assets and what type of services they would require from manufacturers to deal with a cyber incident.

They must communicate a clear picture to the board of what is required and how this plan will be executed efficiently. Running through an IR exercise helps raise awareness about cybersecurity within an organization and creates a resilient business culture that is prepared for anything.