After working on a few factory acceptance tests for SCADA and DCS implementations, I have some suggestions for the process as it relates to security, and particularly security configuration testing.. They can be roughly categorized as suggestions for the vendor, for the project team, and for the security team (that perhaps is brought in as an afterthought to give their stamp of approval).
For the vendors…
I may be more sympathetic than most here – I understand your need to balance bringing a product to market, flexibility, security, etc… However, working with Bandolier I have seen vendors who are balancing these issues and proactively addressing security. This means implementing internal security processes, having outside assessments of their applications, and developing security baselines that work with their product.
So, as it relates to acceptance testing, you should be providing some level of security configuration and validating it on the test floor before a customer ever sees the system. Of course, I’m biased to using Bandolier to help with this process as some vendors are now doing.
For the project team…
They say timing is everything. The key for making sure you get a system in an optimal security configuration starts before the RFP goes out the door. This seems pretty obvious but gets overlooked more often than you might think. Need a place to start? Check out the DHS Cyber Security Procurement Language for Control Systems. I recently conducted an unscientific survey of control system application vendors asking what percentage of their customers use the DHS procurement language. Let’s just say there’s room for improvement in this area.
For the security team…
Ideally you were involved from the beginning. And even more ideally, the vendor has taken care of security configuration and all you have to do is validate their work. Since this is not usually the case, my advice to you is to request a separate security test prior to the FAT and then test again at SAT. The reasoning for a separate test is simply this: it is incredibly difficult to change security configuration at FAT. Disabling services, for example, can invalidate a lot of functional testing. Again, for security configuration testing, I’m biased toward using the Bandolier audit files. The settings for these systems sometimes measure in the thousands and can be buried deep withing the OS and application. Just finding and disabling all the unnecessary services and default accounts may take longer than the couple days you have on site.
These are observations and suggestions based on my experiences. If you have a story, thoughts, or additional suggestions on the acceptance testing process and security, please share it in the comments.
For everyone involved, this is about taking steps in the right direction. Reasonable progress, not instant perfection. That said, there is one thing we can all agree on: security configuration is much less painful to do prior to the system going into production. If it’s ever going to become important to you, get as much done prior to implementation as possible.