Vendors will gladly provide demos. Of course, the demo attempts to show the solution in the best light. You’d never show a demo where your product is lagging or lacking. Each company selects its own advantageous environment and circumstances. This makes comparing even similar products difficult.
At S4, we’ve attempted to address this by providing similar input to a group of competitive products and analyze the results. First with two ICS Detection Challenges and this year with the SBOM Challenge. For S4x24 we will be doing this, with a twist, in the Vulnerbility Management Pavilion.
The Input / The ICS
We have engaged the ICS Village to bring down to Miami Beach a highly realistic ICS to provide the input for the event. They will provide a feed of network traffic to everyone in the Pavilion. The companies in the Pavilion will also be able to scan the ICS, if their product does this.
They will also be able to inspect the ICS and ask questions of the engineers and automation professionals who run the ICS … although their information won’t be complete or always accurate. We are trying to mimic an asset owner who would have access to the ICS, not mimic an adversary’s experience.
Companies that create SBOMs will be able to get information about what’s in the ICS in a variety of ways
While it’s still very early in the planning the ICS Village will have some surprises over the four days. The system may change. There may be some interesting traffic on the network. They may provide some hint cards to those visiting the Pavilion.
The Output
Visitors to the Pavilion will be able to see how the solutions collect, process, and present the same input, apples to apples. Personally I want to see what recommendations they make in what priority order. I also want to see how they present the data and how they are calculating risk and risk mitigation.
The Companies In The Pavilion
We wanted to have a mix of different solutions, and unfortunately we had to turn many solutions away due to the size of the Pavilion.
Scanning Companies
- Framatome
- RunZero
- Tenable
SBOM Companies
- aDolus
- Finite State
OT Detection Companies
- Forescout
- Industrial Defender
- Otorio
Some scanning companies do OT detection and vice versa. I’m also eager to see if any of them partner. Does a scanning or detection company feed info to a SBOM company who then feeds them back info on a SBOM?
The Twist
In the past we have tried to score an event like this. At S4x24 we will not. Partially because we haven’t had great success with past scoring, and partially because the solutions in the Pavilion are so different. You will need to go in and use your own judgment.