Part 1 covered the need to pull and publish more useful information from the gathered ICS incident and vulnerability data. Part 2 covers “Are the numbers intentionally misleading?
245 Incidents Reported To ICS-CERT in 2014 Means What?
The big statistic picked up by the press from the front page of the latest ICS-CERT Monitor is in 2014 ICS-CERT “received and responded to 245 incidents reported by asset owners and industry partners.” This number is meaningless, but it gets the lead from DHS/ICS-CERT/INL.
In fact, this number is actually slightly less than the number of incidents reported in 2013. If the gross number has meaning, then we should saying we are succeeding in reducing ICS attacks.
Or perhaps it is evidence that asset owners, vendors and researchers had bad experiences and are choosing to no longer work with ICS-CERT.
The number of vulnerabilities handled by ICS-CERT was also down from 187 in 2013 to 159 in 2014. ICS-CERT’s method of counting “vulnerabilities” is so broken and wrong that the gross numbers have little value beyond PR for ICS-CERT. For example, some vulnerabilities in ICS common components are counted individually for each product that integrated the component. Others are counted as one vulnerability. And still others are completely ignored by ICS-CERT leading to 100’s of vulnerable products being counted as zero.
Looked at another way, when an ICS application distributed from an ICS vendor contains Havex, Conficker being found on an ICS computer, and an Internet scan of a building automation controller are each counted as one incident, the gross number of incidents has little value.
Everything ICS is “Sophisticated”
The most misleading number in the statistics from the ICS-CERT Monitor is 55%. “Of the total number of incidents reported to ICS-CERT, roughly 55 percent involved advanced persistent threats (APT) or sophisticated actors.” This is despite “In many cases, the threat actors were unknown due to a lack of attributional data.” So this means almost every incident reported to ICS-CERT is considered APT or sophisticated actor.
ICS-CERT needs to publish the criteria of what makes one a “sophisticated actor”, but based on earlier Alerts and Advisories it is a low bar.
Busy Work
The last set of numbers has Idaho National Labs and DHS/ICS-CERT increasingly competing with industry for some undefined reason. ICS-CERT performed 37 onsite assessments in the six months from Sept 14 to Feb 15. Why?
Why 37? Why not 10 or 100?
What is anyone to take from reporting this number and the sectors the assessments took place in?
Where are the statistics showing progress from this work? And what is the progress they were hoping to achieve?
Why is DHS/ICS-CERT doing this work? Why are they doing it for free for critical infrastructure companies?
There are a number of companies, including Digital Bond, that do onsite assessments. There are also companies that do quality ICSsec incident response and ICSsec training.
This work and related reported numbers feign useful activity and avoid the reality that DHS/ICS-CERT is not taking the leadership role in providing technical expertise to help in developing secure ICS protocols and standards, accurately informing government and industry, analyzing ICS attack code and performing the role that ICS-CERT was created to do.