The Control System Cyber Security Self-Assessment Tool (CS2SAT) was presented at the PCSF Annual Meeting earlier this month. I had promised a review of this tool, and it takes place in two parts. The facts of the CS2SAT are in a SCADApedia entry and my comments on the CS2SAT are here in this blog entry.
Overall, the CS2SAT is a very useful tool for asset owners in a variety of situations. For an asset owner just starting a cyber security program it will provide a reasonably comprehensive list of technical and administrative controls they need. For an asset owner with a mature cyber security program, CS2SAT will identify gaps that were either not thought about or perhaps were missed during implementation.
There isn’t a lot new in this tool. Rather the value is in the compilation of this data, presentation in an online questionnaire format, and reports showing the gaps. This is no small benefit because it can be quite an effort to read and implement the variety of standards and guideline documents out in the control system security area.
At least for now, the price is right on this tool as it is free for asset owners in the US and available internationally with some agreements being completed.
I do have several comments and suggestions, most of which I passed along at the PCSF presentation.
- The questions to calculate the Security Assurance Level (SAL) should have something to calibrate the impact of financial losses to the organizations overall gross revenue or some other financial figure. A $1 million dollar loss for Digital Bond would be huge, but it would not be dire for a large utility.
- A final score with maybe some scores on functional or component sections would be helpful. One attendee at PCSF mentioned that if he ran this on 15 installations he would like a simple metric for security posture comparison.
- There are a lot of questions in this tool, and this made me think of Eric Byres MTTC talk and paper at S4. Can we approximate the security posture or risk with a smaller number of well selected questions. This is not really the CS2SAT’s purpose – – which seems to be to identify missing controls and make recommendations. However, if we had data from numerous CS2SAT assessments it would be very interesting if a small subset (maybe 10%) of questions could closely approximate a score for the entire assessment. Statistically this would be easy to determine once the data and overall scoring algorithm existed.
- If NERC CIP was selected, a series of questions related to compliance with the eight standards were added, but the lack of integrated audit support would lead me to not recommend it as a NERC CIP tool. If an asset owner went through all the trouble to answer the questions, they still would need to have some other tool to gather the information required to prove compliance. This other tool, whether it be an accordian file or sophisticated software program or appliance, would require these same questions be answered again. At a minimum the details on what is required to prove compliance via the audit tests should be included in the help files.
- The time to complete the potentially many hundreds of questions is unclear and probably will be based on the number of people in the organization that are needed to answer the questions. I believe a small organization with one or two people capable of answering all the questions could complete the tool in one day. However, a larger organization, such as a pipeline operator, could take weeks to do an intellectually honest self-assessment. I know when we audit SCADA Information Security Policies it typically takes three to five days on-site with the appropriate representatives in the asset owner’s organization. CS2SAT asks a lot more questions than the typical policy audit, but they are less specific questions.
- The drawing program, DrawNodes V1.3, for the network diagram is just ok. It had a quick and dirty feel to it, but it did work. A drawing program is not the type of program you would want to write and support for something like CS2SAT so hopefully it was integrated from some existing and supported code base.
- These acronyms are getting a bit out of control between OPSAID and CS2SAT. I think this may be one of those rare occasions where involving the marketing people would have been a good idea.
My recommendation – – contact INL and get a copy. It is worth adding to your toolkit.