I was glad to be off playing cowboy last week. I missed the noise around the Minnesota, and other states, water sector security incidents, and it gave me time to think about what I wanted to write on the subject.
First, there is not and should not be one plan to secure OT, and more importantly the ability to deliver potable water, for the US water sector. There needs to be at least two very different plans.
Plan 1: Small Systems – Community Drinking Water Systems Serving Under 100,000 People (or under 10,000 people)
According to EPA ECHO numbers 142,406 of the 142,891 water utilities (99.7%) service less than 100K people. If you lower the number served to less than 10K it is still 138,203 water utilities (96.7%).
The last thing these small, or non-large, utilities need is a long list of security controls and a time consuming regulatory program. They don’t even need money to address OT security. Here’s their plan:
- Implement a small number, less than 5, of specified low cost OT security controls. Removing OT from direct, unrestricted Internet access would be one control. A firewall between IT and OT could be another. I’d cap the annual cost at $50K.
- Maintain and periodically test their ability to operate without automation.Assume everything is either taken down or untrusted. Can they deliver potable water to their customers within an acceptable outage time (and this time is not zero) without their automation. They cannot be dependent on automation because they cannot afford or implement the required security program.
The temptation is to add in more requirements. I wrote and deleted more bullets. These utilities are struggling with larger risk issues that they, and their resources, shouldn’t be pulled away from.
The physical infrastructure is a much more likely cause of an outage. I’ve lived through this having been without potable water for months and weeks at a time over the last four years due to physical infrastructure issues.
Plan 2: Community Drinking Water Systems Serving Over 100,000 People
This represents 485 water utilities according to EPA’s Echo. Less than 1% in the number of utilities, but they serve about 50% of the US population. It is very difficult for them to operate large parts or all of their systems manually. They need automation, and they need a security program around that automation.
Much like NERC CIP in the electric sector first addressed the “bulk electric system”, the OT security focus should be on the water systems that serve large populations. We can argue about what security controls and regulatory system should be required. The key is to focus on the 1% of the water systems that service very large populations.
I believe we will find the OT security situation to be much better in these very large water systems. Not perfect. Still in need of improvements and rigor. But nothing like the water utilities that have made the news the last few years. I know many large water utilities that have been working on OT security for 10+ years and have solid programs. Admittedly there likely is some selection bias, since I generally only interact with those that care about OT security.
Possible Plan 3: Community Drinking Water Systems Serving 10,000 to 250,000 People
There could be an opt-in / opt-out plan for the large EPA category, and perhaps the smaller size in the very large EPA category. The utility decides if they want to be in Plan 1 or Plan 2. Or perhaps a defined less rigorous Plan 2. They decide if the benefits of reliance on automation is worth the cost and effort of the OT security program.
Data Link: EPA ECHO Dashboard