I’ve analyzed and made a number of predictions on the ICS Detection market over the past two years. The biggest surprise to me over the last six months has been speed of the market. The winners and losers are being largely determined in 2018 and will result in 10+ no-choice exits in 2018/2019.
Three data points on speed and then thoughts on facing the IDS Detection Challenge.
1. Invitation To Proof of Concept Tests / Pilots
While there are 25+ competitors, we see three of the same four vendors being invited to come in and demonstrate their solutions. Sometimes this is a one or two day on-network demo. Other times it is a formal test plan implemented over a two week period. They then select the winning vendor from this short list of Claroty, Dragos, Nozomi and SecurityMatters.
I’m sure all of the 25+ vendors are getting some demos and pilots just through shear sales effort, relationships, and past sales for companies that are extending their product line with an ICS detection product. The leading four vendors are in 5x or more opportunities than the competition, and the gap is widening.
In fact the market leaders are struggling to service all of the requests for demos and poc tests. We have seen examples of all four leaders fail in individual asset owner tests, due to resource allocation, not product failings. The ramp up in personnel, support, marketing and every other area of the business has been dramatic in the market leaders, and this is leaving a large number of the competitors behind.
Note 1: We have minimal visibility in the European market, so there could be other vendors in the leader category in Europe or one-off countries outside of North America and parts of Asia.
Note 2: There are about five vendors in a second tier who have a handful of impressive and committed reference accounts. Large asset owners accounts that they captured before the market had this speed-based shake out or accounts they have been working on for 12+ months that have closed in 2018. If these companies watch expenses they could survive by servicing these accounts, almost a semi-custom solution, but their market share will shrink substantially.
2. Product Features and UI/UX
Another surprise … I expected to see a leveling of product capabilities since most, but not all, are approaching the technical challenge in a similar way. This is not the case. After seeing demos of the majority of solutions at trade shows and personal Internet demos in 2018, a large gap between the leaders and most of the competition is emerging. The one area this is obvious is in the UI/UX provided to the analyst.
It shouldn’t be a surprise that UI/UX is critical. Many of the 25 competitors can come in and show very detailed and useful information about the ICS after connecting their product. It’s almost always an impressive demo, but asset owners are unlikely to have the same skill and experience level assigned to the detection effort. Asset owners will only use a fraction of the capabilities at the start of a detection effort. This initial capability needs to be simple and the path to use more solution capabilities should be intuitive.
Less obvious product differentiation is the detection capabilities of the products. We see the results of numerous asset owner testing, but this typically just compares 3 of the 4 market leaders. It’s interesting that asset owners doing this testing come up with very strong, but differing results on which solution is best even while testing the same solutions.
Given the resources being assigned by the market leaders, it would be unusual if the others are not falling behind in breadth and depth of detection capabilities. The smaller / slower competitors may need to niche down to be competitive. For example, by focusing on Mitsubishi’s systems and protocols used in the manufacturing sector, the lagging competitor may be best in market technically and also be able to focus their marketing effort.
Note 3: We are also seeing solution differentiation occurring between the four market leaders, with Dragos being the most different of the four. That said, you can see the differences in approach to detection in the Claroty, Nozomi and SecurityMatters solutions as they are past building out the basics and really thinking about how the analyst interacts with the solution and what is most important to detect.
3. Mindshare and Funding
If you follow the ICS security space you are seeing the same vendor and expert names appear over and over in the press, webinars, conferences, and other marketing and awareness activities. Sometimes this is a pure purchased mindshare, and in other cases it is effective hiring and use of influencers. We could add three more companies to the leader category on the marketing side, although it is unclear why this is not leading to more poc tests. My guess, and it can’t be called more than a guess, is these companies are having success in Europe.
The leading companies are doubling, or even quadrupling, in employee size in both 2017 and 2018, and have an equally fast growing budget on marketing and demos / poc testing. Then when they win, they need staff for installs and support as effective channels are almost non-existent at this point. Funding is required for cash flow even if you believe the top line revenue claims of some of the competitors. Claroty has raised $92M; Nozomi $52.5M; Dragos at $11.2M would seem likely to be near another, much larger round. SecurityMatters had a head start, being in this field since 2008, and appears to be more modest in terms of speed/growth to date.
Facing The ICS Detection Challenge
The ICS Detection Challenge tests a solution’s ability to identify cyber assets and detect cyber incidents through passive monitoring of packet captures from actual deployed ICS.
At the S4x18 Challenge this January, Claroty, Nozomi Networks and SecurityMatters chose to compete and all finished in the top tier, with Gravwell also competing and finishing at a lower tier. It’s very interesting that these are 3 of the 4 market leaders, and it leads to chicken/egg questions.
– Did these three compete because they were confident they would do well? My answer: Yes
Many, but not all, of vendors asked and declined would not have faired well based on what I have seen of their product and what I’ve been told by asset owners who tested them.
– Are they getting some of the poc opportunities because they finished top tier? Yes.
I know of asset owner influencers who attended S4x18 that selected vendors for the poc because of the Challenge. How widespread this was and is we will never know.
S4x19 ICS Detection Challenge
We learned a lot from this year’s Challenge, and are improving the S4x19 Challenge in almost every way. Better packet captures from more ICS (400GB), an existing detailed asset inventory that didn’t need to be created by analyzing pcaps, a larger team starting months earlier to create the cyber incidents, an improved scoring approach and much more.
The goal of the S4x19 ICS Detection Challenge is to identify the Top Tier solutions in passive cyber asset identification and in passive cyber incident detection.
After inviting every vendor we know that has a passive solution: we have 4 teams at this time step up to the Challenge: Claroty, CyberBit, Dragos, and an asset owner team using ELK.
There are still a small number of vendors making a final decision, and a total of 8 spots are available to competitors. It has been fascinating and instructive to hear the reasons for not competing, including:
Lack of Resources / Don’t Want To Allocate Resources
After the earlier discussion of speed and resource strain, one could be sympathetic to this reason. Not me. There is no way to prep for the Challenge. A team of two analysts brings the product and insures they can receive the pcap feed on Monday, and then the team competes for 8 hours on Tuesday.
Many, if not most, of the competitors send their best analysts to S4 already. There are 100’s of asset owners there to see the results and if you do well you get on the S4 Main Stage to show where your solution shined. Not to mention the post event videos, articles, podcasts, etc. It is hard to imagine a more highly leveraged opportunity to gain mindshare.
It is also, at this point, the only public testing of these passive capabilities. It is not one of ten opportunities to compete. It is one of one.
It’s Not A Fair Test Of Our Product
Over half of the competitors offer specifically a passive cyber asset identification and passive cyber incident detection product. This reason makes no sense for these vendors. The Challenge was designed to test their main product features and claimed capabilities.
There are some vendors where passive asset inventory and passive cyber incident detection are only a portion of their solution. For example, they may have an active component. While I’m a huge proponent of adding active capabilities for asset identification, not having a Top Tier passive component to the solution should be a non-starter for asset owner selection of a detection solution.
We Are Worried We Won’t Have Support For The ICS In The Challenge
Some expressed concern that they don’t know the products in the Challenge pcaps, but we have reassured them that the primary ICS are extremely common protocols and products. The pcaps include some less common protocols and products to test breadth of coverage, but this is a small percentage of the assets and packets. The fact that technical resources from Rockwell Automation and Schneider Electric are on the team creating the packets for Detection portion of the Challenge provides a good hint and reassurance that the Challenge includes mainstream protocols and systems.
We Don’t Like X / We Don’t Think Y Is Fair
This feedback is great, and it has altered the S4x19 Challenge. We have been able to address most of these concerns and eliminate or reduce them. The way the Challenge is run has not been the reason for a decision not to compete to the best of my knowledge.
Not Facing The Challenge
All business have to make strategic and tactical decisions on where to spend their time and treasure. And they have to live with the results of these decisions. Raising money, developing a product, assembling a team and going to market is impressive, and this accomplishment shouldn’t be discounted. Startups are also high pressure and very emotional, and this article is going to be read by many friends in these startups as an attempt to bully them into the Challenge. There is some truth in that; I’d like to see the Challenge team give all of these products a hard shake and see which are Top Tier from a technical perspective.
The analyst in me though has a hard time understanding why any company that is not in the market leader category today would choose not to compete short of not feeling confident in being Top Tier.
Consider the case of CyberBit who has chosen to compete. They have limited mindshare and likelihood to be on a shortlist, at least in the US market in 2018. If they finish Top Tier alongside a market leader this is a huge win for them. I have no idea how competitive their product actually is, but we will find out.
There are 10+, or even 15+, vendors who are in the same situation. They are not even being considered in a large percentage of US and Asian asset owner opportunities. A Top Tier finish could change that. It would get them on the S4 Main Stage in front of 500+ influencers, and get them articles, videos, podcasts, etc. to reference when prospects ask why they should be considered.
The clock is ticking for about 20 of the vendors in the ICS Detection Market. They will either be forced to exit or be a small niche, inside the relatively small ICS niche, solution. There are a limited number of events or circumstances that will change the momentum of the market before it is narrowed to 3 – 8 competitive vendors in 2019. I believe the Challenge is one of these, and has me drawing the conclusion that most that decline to compete in the Challenge know they are not competitive solutions.
Here are details on the S4x19 ICS Detection Challenge. There are three spots remaining.
Contact s4@digitalbond.com if you are interested in competing.