ISA’s ISASecure has been working on an Embedded Device Security Assurance certification. We have previously reviewed, see links at the bottom of the post, the Functional Security Assessment and Software Development Security Assessment documents that represented two legs of the three leg certification. What remains is the protocol stack testing that ISASecure has named Communication Robustness Testing [CRT].
This week two of the necessary certification infrastructure or process documents have been posted:
EDSA Chartered Lab Operations and Accreditation
EDSA Recognition Process for Communications Robustness Testing Tools
Let’s start with the one missing document, EDSA Common Requirements for Communications Robustness Testing as well as the six more detailed testing documents under this document. These documents have to be detailed enough so independent organizations can develop similar protocol testing tools that will achieve consistent results. As I’ve been saying for three years now, this is an extremely hard set of documents to write. In fact I’ve been suggesting that ISASecure would be better off using the time and effort to develop their own protocol test tools using one of the fuzzing frameworks. This is not difficult, would provide consistent results and provide ISASecure with more intellectual property.
Recognition Process Document
ISASecure, however, has been determined to complete the mission they started and took two more steps forward with these two documents. An ISASecure accredited lab must use a “recognized” CRT tool, and the Recognition Process document defines the sequential steps to achieving recognition. Steps 1 and 2 are providing general technical information and test space coverage. Step 3 requires the CRT vendor to provide test results for selected tests, and then Step 4 requires packet captures for selected tests.
This sampling approach for CRT tool recognition is a practical solution. It should prove that a CRT tool meets most of the test requirements and devices clear a minimum hurdle. Oddly enough though some recognized test tools could actually be much more rigorous than others and result in different negatives. This is something field device vendors and accreditation labs will need to think about in their selection of CRT tool. Think of it like a NERC CIP-005 assessment where owner/operators want to do just what is required to meet the regulation and not any more.
There are seven “Testing the Robustness of …” documents referenced in the Recognition Process document so this is where the gory details of the protocol testing for Ethernet, ARP, ICMP, IP, TCP and UDP are likely listed. The most important part of the Recognition Process document is Annex A that maps the requirements in the unpublished Testing documents to three of the four steps mentioned in the Recognition Process document. Step 4 is not included and likely will be provided to the tool vendor after completing Step 3.
After a quick read, this Recognition Process seems to strike a good balance between practicality in a sampling approach and verifying the rigor of a CRT tool.
Lab Accreditation
First, don’t skip over the references in Section 2. There are a number of document titles here that show the structure of the ISASecure certification program. Most of these documents are not yet published even in a draft format, but it shows ISASecure has been working through what will be required for a credible certification program.
Labs will need to be accredited to IEC 17025 and IEC 65 to qualify for ISASecure accreditation, both of which are new standards to me so no comment. In fact, this document probably will only make sense to an organization active in certification programs. There are requirements for personnel, testing, mapping to the various IEC standards and more. I’ll have to defer to others on how reasonable or workable this document is, but it is does show a fair amount of work and thought went into it.
Hopefully ISASecure will push out the remaining documents, polish them up and get this program active. Is an October announcement at ICSJWG possible?
Related Posts: