Because NERC CIP is the regulatory force for cyber security in the electric sector, I tend to do a lot of work with clients on how to best to implement the various technical and administrative requirements. CIP-003-3 R6 requires that owners establish and document a Change Control and Configuration Management program, which is also a good practice anyway.
The problem I encounter most when developing these programs for clients is the standard complaint against NERC CIP; owners only wish to do the minimum, and not much beyond it. This is especially tragic for Change Control, because Change Control has potential to help prevent compliance violations and security problems, but is usually seen as an impediment to efficient work by personnel and compliance overhead by managers.
The quick and simple way to implement this program is a simple form, filled out by the requester, submitted to an approver. Once approved, the requester does their change, and then closes out and files the form. Important information in the form is very bare bones, usually just a name, date, description of the change, and a signature. All very easy, and most likely compliant.
But, I have a lot of trouble with the minimum approach here, for a couple of reasons. First, what has changed from the normal way of doing things, except that paper is generated? It adds zero value to the existing process, except to get a CIP-003-3 R6 check-in-the-box. Second, it doesn’t take into account the various changes, many of them required by the NERC CIP, that could be standardized in the program itself. And lastly, the responsibility for adequate performance to the NERC CIP standards is fully on the requester and the approver, and is not in the change control process itself.
One of the best reasons to have a change control program is to standardize to streamline routine changes. In NERC CIP, an obvious candidate is the CIP-007 R3 patching process. But, the idea is to not simply standardize R3, the idea is to standardize the whole change pipeline. For a little example:
1. Patching is routine, so the change request could be initiated on the periodic basis to correspond with how often the patches are assessed, tested and installed.
2. Patching is a significant change under CIP-007 R1 , so testing is a required part of patching. It makes sense to roll the patch testing into your patch change request. Ideally the test procedure itself would be a standard one, and include some standard documentation of the results.
3. The assessment of patches is a standard and repeatable process, this assessment, and the documentation, could be included in the change as well, as a prerequisite to the change.
4. It’s a good practice to ensure your backups are available before performing a significant change, and to make new backups afterwards. This lets you quickly rollback from a bad change, and to keep up-to-date backups in case of disaster. Rolling a basic backup test (CIP-009-3 R5) as a prerequisite for the change and a full backup after the change might be appropriate.
So, what are the benefits of this approach? Well, all the documentation associated with a change is available in a single packet, which simplifies discussion with auditors. Some auditors actually suggest this approach, such as MRO in this document’s CIP-003 section. Personnel also don’t need to create and get multiple change requests approved, a reduction in personnel time.
There are other, more asset specific pipeline additions that could be made too for control centers, generation, and transmission. For instance, many generation sites have vendors test patches (to a certain extent) before ever admitting them into the patch program. I actually discuss many standard changes in my Generation Cyber Security Training in Rosemont on May 17th.
I wish more owners would follow this approach, as it allows you to make best use of compliance and security funds, and reduce some of the overhead and frustration. That money and time can then be spent streamlining other processes.
title image by Nanagyei