Dale Peterson interviews the ICS Detection Challenge Winner – Claroty and the runners up – Nozomi and Security Matters. They discuss where the competitors did well, how the products are likely to improve in the future, and what the future direction of the ICS product detection category is likely to be.
The ICS Detection Challenge at S4x18 last January pitted Claroty, Gravwell, Nozomi and Security Matters in a competition to determine who could create the most complete asset inventory and who could do the best job detecting attacks through passive ICS network monitoring only. Dale Peterson and Eric Byres discuss the packets used in the test and analyze the results. What this product category can and cannot do. The last 15 minutes talking about the future of the ICS Detection product category.
The Asset Identification and Inventory results from the Challenge showed the potential of these solutions, but also the limitations due to maturity and a passive only approach. Claroty won the Asset Identification phase by identifying the largest number of assets (device type and vendor) with a score of 23, followed by Nozomi and Security Matters with a score of 20. Nozomi clearly provided the most detail in their asset inventory and was the only competitor to identify the key SCADA system. Security Matters did the best in identifying CVE’s for the asset inventory.
The Detection Phase of the ICS Challenge was won narrowly by Claroty (24) over Nozomi Networks (22) and Security Matters (22). Congratulations to the Claroty product and team. The final score was not as interesting as what was detected, what was missed, and conclusions about this product class. This post has five conclusions Dale pulled from the scoresheets and interviews, notes on the scoring methodology and deficiencies, and a chart showing the 19 incidents inserted into the packet capture.