Cisco recently published their 2021 Security Outcomes Study. It is worth a close look. Not so much for the results and conclusions applicable to the enterprise, but the methodology is worth adopting for the ICS environment. It would be great if ARC or someone else would do this and publish the results.
First, I’m not a fan of almost all industry ICS security studies. They tend to greatly inflate the size of the ICS security market, the concern about ICS security (as evidenced by the actual money spent), the ICS security posture, and just about everything else. Most graphs are hockey sticks. Many are little more than marketing vehicles that point out a huge problem that the issuer’s protect is best suited to solve. (This Security Outcomes Study has a bit of this as “Proactive Tech Refresh” is the most impactful Security Practice.)
With that caveat, there is much to like about this study. The approach:
Security Program Outcomes
ICS security is not a contest to see how many good practice security controls you can implement and maintain. In broad terms it is to help the company reduce ICS cyber related risk to an acceptable level. This study measures the impact of security strategies on a set of eleven Security Program Outcomes across three categories.
- Enabling The Business examples: 1. keeping up with the demands and growth of the business 2. Obtaining buy-in from peers and other organizational units
- Managing Risk examples: 1. managing the top cyber risks to the organization 2. avoiding major security incidents and losses
- Operating Efficiently examples: 1. operating a cost-effective security program 2. minimizing unplanned work and wasted effort
In the ICS world, we could have most of the same Security Program Outcomes and maybe some new or modified Outcomes such as avoiding security incidents that cause a loss of life (or environmental damage). There would likely be more value in revising these Security Program Outcomes for the entire organization, this was a first try for this methodology, rather than having a customized ICS list.
The 4800 respondents from 25 countries were asked if their company was succeeding in each of these eleven Security Program Outcomes. The most success (47.9%) was meeting compliance regulations, and the least success was (35.5%) minimizing unplanned work. The gap of just over 12% makes this statistic of minor importance, but it is necessary for the next steps.
Security Practices
The Study then identified 25 Security Practices, developed largely from the NIST CSF. Some examples include:
- Security is important to executives
- Prompt disaster recovery
- Accurate asset inventory
Cisco asked the 4800 if they “Strongly Agree” that their company followed each practice.
The Payoff
At this point they had a perceived, not actual, status of Security Program Outcome and the Security Practices that company was confident they had implemented. It was time to correlate. In Cisco’s words:
For each practice-outcome combination, we calculated the change in probability of achieving outcomes associated with higher levels of adherence to various security practices.
With this information they could identify what Security Practices had the biggest impact on the Security Program Outcome average, and more importantly to each of the eleven Security Program Outcomes. Some companies strongly agreed they had “timely incident response”. How did this impact one or more Security Program Outcomes as compared to those that did not strongly agree they had timely incident response?
Cisco pulls out many analytic conclusions from this Study. You should read it. Just one nugget to whet your appetite.
Which function of the NIST Cybersecurity Framework contributes most to success?
The Identify function ranks #1. The Protect function ranks next to last for contributing to a program’s overall success.
Conclusion
As the need to address ICS cyber related risk has gained mindshare, there has been a leap to identify as many deficiencies from good security practice and try to grind through addressing those. This has not been effective in most cases because it requires a massive increase in work to address a loss that has not occurred. And a second leap to buy a shiny product.
A study like this that asked the same questions of just the ICS environment would be enlightening. We would learn what Security Practices provide a perception of meeting a company’s desired Security Program Outcomes from the business perspective.
As with all studies, some skepticism on the results is warranted. The perception aspect to the questions is the biggest yellow flag. However, companies do spend money based on their perception of the current situation.
So who has access to 500 or 1,000 individuals who would be able to answer these questions for the ICS environment?