We know very little about what security controls and consequence reduction actions reduce the number and impact of incidents that includes an OT cyber component.
Read that again.
We have hypotheses. I have hypotheses, and wrote a book on the topic A Year In OT Security. It’s an approach of understanding your situation and prioritizing actions, not a set of recommended security controls. With every year of experience I believe more that the answer of what to do varies greatly by sector, company, and site.
What do we know based on incident data:
- Effective segmentation of OT from IT has stopped many attacks from reaching OT.
- Operation’s reliance on IT systems and data can cause OT to shut down until these systems and data become available. (frequently demonstrated by ransomware on IT)
- OT directly accessible from the Internet will be found by attackers and often exploited. Get your OT off the Internet.
- More sophisticated and directed attackers will exploit weak remote access into OT for persistent OT access. Deploy two factor authentication for remote access as a minimum.
- Malware is often walked into OT on removable media. Have an effective removable media policy.
It’s a short list. I probably missed a few. Add them in the comments and include what incident data you base this off of.
The problem is we are getting consensus on how to reduce OT cyber risk that is unsupported by data. They all sound reasonable and are unproven. OT Asset Inventory is a great example.
- What evidence do we have that an OT cyber asset inventory has reduced the frequency or impact of OT cyber incidents?
- If / when we have this evidence, what percentage of the OT cyber asset inventory is required to get what reduction? Do you need the key 20%? 100%? Different percentages based on level or criticality?
- What is the cost to get this risk reduction? Is this the best place to spend these resources? Would it be better spent on a safety or protection component? Recovery capability? Alternate source? Other security controls?
It may turn out that having a complete and rich OT cyber asset inventory does greatly reduce the number and impact of incidents. We don’t know yet, and the consensus that this should be a highly prioritized action is troubling. There are similar questions with detection, micro-segmentation, patching outside of the exposed to other zone cyber assets, … We don’t know.
Until we do know the community would be better served by multiple approaches being tried, hypotheses being tested. Or at a minimum approaching OT cyber risk management with less undeserved certainty.
AI’s Impact On Premature Consensus
Which leads me to this article’s title. A review of the OT security content even prior to ChatGPT would indicate there was a growing consensus on how to best address OT cyber risk. Unwarranted certainty. Certainty without evidence.
AI is already making this worse. Not because of the hallucinations issue, wrong answers, that is often the problem. AI makes premature consensus worse because content creation is a lot easier. If you spend time on LinkedIn you’ve seen this with OT security content. More people generating significantly more content aided by AI. It’s mostly 101-level content, conventional wisdom, stated with authority.
I don’t blame any individual for doing this. It’s typically done with good intentions and with confidence in the climb up to Mount Stupid. We’ve all climbed that mountain, and it is a very exciting time. You’re learning so much so fast about a new world.
The problem is AI uses this and creates two reinforcement loops. The first loop is human. The more you read about a certain control or approach being right, the more you are likely to believe it, even if supporting data is not available.
The second loop is AI and more dangerous. The increased content generated by AI or slightly modified AI output, is then used by AI models and further strengthens the consensus. There are many terms for this, which I looked up on Claude. I prefer “synthetic data poisoning” or “model autophagy”.
What’s the answer? I’ll suggest two actions:
- Insist on data to prove, or disprove, the effectiveness of any risk reduction project. Create your hypothesis and metric before deployment and measure after deployment.
- Experienced OT security professionals need to create more content expressing skepticism about OT security recommendations they feel are wrong or mis-prioritized and provide alternate hypotheses. Zag. Not to be contrarian, but to help those on the climb and AI models understand there is not an evidence based consensus.
This doesn’t mean we all sit still until there is a consensus. If you are an experienced OT security professional and understand the asset owner’s business and environment, then you should have a hypothesis of what to do first, second, third, prioritized by efficient risk reduction. You should consider, but don’t bow to any existing standard, guidance document, top 5 list, or talking head (including me).
Use your experience and judgment because there is no evidence based consensus on what should be done to reduce OT cyber risk beyond a small number of initial controls and actions.