Earlier blog entries talked about the ISA Embedded Device Security Assurance Certification and the validation methods for the Functional Security Assessment part of this certification. In this entry I’ll review the as yet unpublished validation columns in the Software Development Security Assurance document. Again, ISASecure has been kind to provide these documents to me and allow me to comment publicly on them.
The second validation column in the document is titled “Validation by Independent Test Required (Yes/No)”, and this column has me baffled. There are only two Yes answers to independent test required in the 19 pages of tabular written requirements. The Functional Security Assessment document mandated independent test for more than 3/4 of the requirements, and almost all of those independent tests were document review. So why no similar type of document review or, even better, process audit here? It appears to be inconsistent to have self certification for one leg of three legged EDSA stool, and independent test for the others. I have no idea why the Ascertain Changes and Authorize Changes requirements were selected as the two requirements that require independent test.
As I mentioned in the Functional Security Assessment document review, perhaps ISASecure should consider certification rigor levels with different levels for self certification and independent test. Both have value, but the independent test may give owner/operators more confidence in the results.
The other validation column is Validation Activity, and this column is much more interesting in the Software Development Security Assessment document than the previously reviewed Functional Security Assessment document. Many of the entries would be useful for a certification entity, while others need a bit more work or a supplemental document. Here are a few examples.
SDSA-SMP-1.2 – Review of Security Management Plan: Verify the existence of review minutes with a list of action items, all of which have been closed.
This is useful Validation Activity guidance that would help a certification organization. It also is a useful process check.
SDSA-SMP-1.3 – Lifecycle Model: Verify that the lifecycle model is documented and includes all of the required phases of the security development lifecycle.
There are three references, but what are the “required phases” and does this Validation Activity really provide any helpful information?
SDSA-SMP-5.6 – Modification Audit: Pick a few modifications, and verify that the CM process documents the originator, the date and time of the changes and that a mechanism exists to determine exactly what changed.
Another example of a helpful Validation Activity. Although this is somewhat obvious it does provide information to the certification organization. But wait – – this requirement does not require independent test so the vendor will be self certifying. Leading to the question what type of records are going to be required for self certification and how will those records be reviewed by the certification organization?
SDSA-SRS-3.2 – Known or Presumed Threats: Verify known or presumed threats to the assets which protection will be required are documented. Evidence that requirements were reviewed and known/presumed threats list was included in review (e.g. meeting minutes or inclusion in completed review checklist.) Record results as: a. Pass, b. Fail, c. Not Applicable
There are a number of straight forward documentation checks as Validation Activities, but these are worthwhile. One could argue it is just a documentation exercise, and some vendors could treat it this way, but let’s have a bit of optimism and consider the enforced elements in the plan could spur thought and action and sound security engineering practices over time.
There is a lot to review and consider in validation activity portion of this document. Hopefully it will be published soon in a future draft for all to see and consider. Coming up with a certification program in enough detail for a variety of organizations, let alone a variety of certification teams, to provide a consistent certification process is not easy. This is a good start, and I can see how it can be made to work.