Information sharing is a key component of President Obama’s Executive Order with emphasis on sharing information on threats and incidents. With the current insecure state of ICS, the value of this information is limited to the question IF we should start to do something about the problem OR NOT; are we really still answering that question?
And quite frankly the critical infrastructure ICS community is not ready for this information. ICS are lacking basic security controls, are insecure by design and quite fragile. Threat and incident information is only important and helpful when you have a functioning and effective security program — unless you are arguing that basic security controls are not urgently required in critical infrastructure ICS.
The best and quickest way the US Government, and other governments around the world, can help improve critical infrastructure ICS security are:
- A focused and very public information sharing campaign showing how insecure by design and fragile ICS are today and
- Highlighting vendor and owner/operator efforts, or lack of efforts, to upgrade or replace these systems with secure and robust ICS
The good news is this information sharing does not require an Executive Order, legislation or even cooperation by the vendors and owner/operators to implement.
It’s axiomatic that information sharing and public discussions too date have not convinced owner/operators that they should spend money and effort securing critical infrastructure ICS. It’s also self-evident that ICS vendors, with few exceptions, have not found adding basic security and robustness to their products is a good business decision. Therefore DHS and other elements of the US Government should focus information sharing on convincing all involved in allocating these resources
We have heard often, and agree, that most participants in ICS security know (the “everyone knows” response to Project Basecamp) that ICS are missing basic security features and fraught with vulns due to poor design and coding practices. Trying to convince the ICS community to spend time and money on ICS security has and will continue to fail. There are a small number of owner/operator exceptions, but our experience is this is most often instigated by a C-level executive who is concerned, not the operations group. And the number of these ICS security early adopters are not large enough to drive the product vendors to secure their offerings.
Pick One Key Security Control
The key is focus. We are not going to be able to break through to the people who can fund and drive ICS security with a complete security program. Identify a few security controls, and focus on them one at a time. Controls that are so critical that little else matters if these are not in place, and controls that will make a major difference in keeping the bad guys out or detect them. Controls that can be explained simply and powerfully to a non-technical audience.
Here are three examples that I would start with:
- All critical commands and management functions must be secured – at a minimum authenticate the source and data before accepting the request.
- Eliminate all remote connections to the ICS except for emergency situations. The ability to remotely access the ICS must be disabled by default, and all remote access connections must be explicitly approved and reviewed periodically to verify they were emergency situations.
- The ICS must have an attack detection capability that would detect new or unusual communication as well as any communication that is indicative of a cyber attack.
However long the list is, the highlight should be on one item for six to twelve months before introducing the next. Build an entire marketing campaign around that item. Yes, I wrote marketing. Not only does the information need to be shared, but it also needs to be sold.
Bring the Focus and Pain to C-Level Executives, Public Utility Commissions, Stockholders, Vendors, …
Let’s say the first item is “All critical commands and management functions must be secured – at a minimum authenticate the source and data before accepting the request.” (Side Note – most executives, Senate staffers, and anyone else outside of ops is surprised when I explain to them the ICS lack any effective security if an attacker can get to them. It truly seems impossible this is still the fact 11.5 years after 9/11.)
The basic question for any owner/operator is have you started a program to implement authentication for critical commands and when will it be completed?
The US Government should now know what ICS systems and components are used in the critical infrastructure — we could quickly create a list with 90% accuracy. Then:
- Create a public list of the devices lacking this basic security control from the critical infrastructure product list
- Create demonstrations showing how this lack of authentication allows an attacker to take complete control with a variety of products. Put some teeth and detail behind the Cyber 9/11 and Cyber Pearl Harbor terms.
- Be creative with videos, social media, presentations, etc. Show real physical damage like Aurora, show a case study of teens causing the damage with a couple of days of effort, show displays that are inaccurate, show documentation that causes the problem, show statements from the protocol committees on insecure by design, … Highlight this item in messaging at all levels of technical detail.
- Show how ineffective “the keep the bad guys out” strategy is. Document and anonymize pen tests with different approaches to getting in, including physical, cyber and blended.
- Have meetings with critical infrastructure CEO’s and discuss the risk they are passively accepting. Ask them what their plan is to authenticate critical commands? When will it be completed?
- Create a set of questions, related only to this one issue, that every Public Utility Commission should ask their regulated utilities.
- Create a set of questions that any shareholder or stock analyst should ask companies running the critical infrastructure. Meet and hold joint press conferences with the SEC on this issue. Work with the SEC to add each issue to the disclosure process.
- Encourage Congressional Committees to call critical infrastructure owner/operator and vendors who sell to them and ask them if they have started a program to address this and when it will be completed.
- Have the carrot as well. Highlight vendors who have added authentication of critical commands to their products, particularly those that have provided a simple upgrade path.
- Track progress on this issue with percentage of vendor solutions with this control and deployed system percentages by critical infrastructure sector
Reluctance to put out this information was understandable pre-Stuxnet, both for hiding offensive efforts and not wanting to start an offensive cyber weapon frenzy. These reasons are no longer valid, and governments and organizations around the world understand the value of and how to compromise insecure and fragile ICS. Not sharing this information is only harming the effort to secure these systems.
After the message from the first item is accepted by the C-level executives and other decision makers … that this is mandatory and a program is started, move on to the next item. After even one item you will have engaged decision makers in the process and subsequent items will be easier, and eventually lead to an ICS security program.
Again the key is to have something that will make a big difference and is easily understood. For example, a PUC, stockholder, CEO or anyone else could easily ask, “Do we have any remote connections to our critical infrastructure ICS?”. Develop a marketing campaign showing all the different types of remote connections, how they can be exploited, and an accountability program to remove them. (I’d argue the S4x13 spear phishing real world test and making the Digital Bond spear phishing email and malware public was better than the generic ICS-CERT warnings) Go to centralized sites that have a tremendous number of remote connections to critical infrastructure ICS, such as ICS vendors and support sites, and show them alternatives to address the questions they should be getting from multiple customers.
It’s also important to identify what ICS-CERT, DHS and the USG should stop doing. Accessibility from the Internet and Shodan are a great example. Many ICS are at risk because they can be reached from the Internet, but this is not a major issue for critical infrastructure ICS. This should not be part of the critical infrastructure messaging, but it could be appropriate for general cyber security messaging to individuals and businesses … which leads to ICS-CERT efforts today and in the future.
ICS-CERT
ICS-CERT is the wrong entity to perform vulnerability coordination. They should go back to the original plan for ICS-CERT, providing Level 2 support to US-CERT. US-CERT and CERT/CC do a fine job of vulnerability coordination. ICS-CERT should set a criteria for when the devote more time than a token phone call, such as:
- the vulnerability affects a system on their critical infrastructure list AND
- the vulnerability affects the security posture of the system
The second requirement is important and often leads to misallocation of ICS-CERT effort. Does it really matter if there is a CSRF or buffer overflow vulnerability in a device that you can connect to and take complete control using a feature?
Removing all the coordination and effort on vulns with little impact would allow ICS-CERT to dive deeper into a smaller number of vulnerabilities, not insecure by design but true vulnerabilities, that could have a major impact on the critical infrastructure. For example, a vulnerability in the PI Server protocol would jump up high on the list, as would a vulnerability in DNP3 SA stack or WirelessHART stack. They could get these systems into the lab, verify the full extent of the vulnerability, and test any proposed fixes. The resulting Advisory would be not based on trusting the vendor supplied information.
Similar filtering needs to take place in ICS-CERT’s incident handling and fly away teams. An attack on a company with an ICS doesn’t warrant ICS-CERT involvement. ICS-CERT highlighted cases they had worked related to SSH password cracking attempts on corporate network systems. This probably happens quite often to an Exxon or FPL.
Filtering needs to take place on the impact to critical infrastructure. The infamous Springfield Water Hack, that was a false alarm, shouldn’t have been an ICS-CERT issue even if it was real. A minor pump outage in a small municipal utility cannot be chased down by our elite ICSsec forces. When DHS offers the elite services at no charge, everything becomes important. Would these organizations pay for a response by hiring someone like Mandiant? This is a good measure of the true impact of the incident.
My contention is ICS-CERT has focused on these huge number of vulns and anything that can remotely qualify as an ICS incident to look like they are busy and making a difference. It has worked with the press, Congress and even many owner/operators. However, if the measure is the improvement of the ICS security posture over the last ten years it must be considered a failure.
ICS-CERT and DHS are not alone in this failure. The ICS security community including vendors, owner/operators, industry organizations, consultants, gurus and anyone else involved has failed as well. The difference is ICS-CERT, DHS and other elements of the US and other governments have the loudest and most respected voice. Their silence on actually fixing the security problem has a huge negative impact.
As much as I would like this site, my voice and the voice of other ICS security gurus to instigate a much delayed serious effort to make critical infrastructure ICS secure and robust, it is clear that the US Government is the big voice that can make a difference in the US. The average asset owner will never take action as long as the government is completely silent on an issue which is only stressed by a handful of consultants. We need DHS, FERC, DoE, DoD, and the Executive Branch to or a similar organization to actually say out loud that critical infrastructure ICS must be secured now, not decades later. I can only speculate why they have been silent or SCADA Apologists for the last 11.5 years.
Mature Information Sharing
The type of information sharing envisioned by the Executive Order could be very helpful once critical infrastructure ICS has basic security. Today it is just not that important. The information that needs to be shared now is “Your ICS systems and procedures are insanely vulnerable. Now do something about it.”
Next … The Cybersecurity Framework