The semi-annual Industrial Control System Joint Working Group Conference is traditionally the best place to catch up with everyone in the ICS Security community. DHS puts on a solid program, and there is a certain feeling you need to be here even though there have been little non-conference results from ICSJWG or its predecessor PCSF. Make sure to follow our tweets on day two @digitalbond.

Here’s what happened on day one:

Marty Edwards, the new Director of DHS Control System Security Program, started the day with a low key discussion of the evolution of control systems. It lacked a real call for action or compelling point, but it did clearly demonstrate Marty’s experience working in ICS. I believe he is the first person in that role that comes from the ICS world.

My question for Marty and ICSJWG is how do they measure success with ICSJWG. Is it by number of people attending the semi-annual events? Some useful products coming out of the Working Groups? Measureable increase in information sharing? At PCSF, the annual conference became the main focus and measure of success, but I think DHS has bigger goals and measures for ICSJWG.

Stephan Parker from EnergySec/NESCO provided the best summary I have heard on what NESCO wants to accomplish. NESCO is a bottom up effort to Engage, Equip and Empower the energy sector owner/operations. Engage so far has been attending events, voice of the industry meetings and a variety of other outreach efforts.

Equip was the most interesting with three programs highlighted. The Repository of Open Source Security Solutions for the Energy Sector (ROS²ES) was new to me. NESCO is going to be funding this effort both to create the repository and encourage contributions. They are also focusing on Workforce Development / Training with the NESCO Academy program, which adds to a growing ICS Security training options. The last effort was information sharing that was covered in general terms.

Darren Highfill, representing Southern California Edison, spoke about Smart Grid security efforts through the UCA International User Group and ASAP-SG. ASAP-SG has had success by industry/government funding the initial draft of security profiles that are then handed off to standards and guideline groups for review, comment and revisions. This has dramatically sped up the cycle time for document development and approval. There are security profiles out or almost out for AMI, 3rd Party Data Access, Distribution Management and Synchrophasor Management.

One other note, the Embedded System Security Task Force in the UCAIUG may have some information that would be helpful for PLC/RTU/field device security. Similarly this Task Force should be leveraging the work that ISAsecure has done on embedded system security certification.

Joel Langill of SCADAhacker covered the Luigi vulnerabilities in general and a 7-Techs command execution vulnerability. He tried a live demo that sort of worked, but did a good job of showing the exploit code loading a program, how Metasploit payloads are built and run, and discussions on what an attacker can do. The combination of the demo not working and the low quality projector made this presentation much less than it could have been, but it was still well received. UPDATE: The demo was successfully completed at lunch and was well received.

The morning finished with a presentation from Patrick Beggs from NCCIC and reports from three Subgroups. The Subgroups continue to flounder, or maybe are starting to founder. The Roadmap subgroup did issue the first deliverable and is the group making progress. The Workforce Development subgroup is doing a reboot and the Vendor subgroup continues to talk about the charter. The Vendor subgroup is the most disappointing because it had momentum at PCSF. ICSJWG was complaining that it is not diverse enough with non-vendor members, but what made it effective in PCSF was vendors were able to work through common issues. In general, the ICSJWG subgroups continue to disappoint and don’t get the participation that other ICS security efforts get.


The afternoon and the remainder of ICSJWG has three presentation tracks. It’s a sign of a good agenda that there are often two talks at the same time people want to attend. I got sidetracked with some meetings – – another benefit of ICSJWG is you can meet with clients, partners, vendors … all in one place, so I was only able to attend 2.5 of the sessions.

David Sawin provided an informative, if not a bit scary, talk about DHS work on securing the Transportation sector. There are a lot of sub-sectors here, including pipeline, and many of them are just beginning to address SCADA security. They have done a number of inventory or surveying of systems and are working with some of the main vendors, but little progress in deployed systems to date.

Using rail as an example, they are focused on trying to secure Positive Train Control. David said there were three main vendors in this field, and they are talking to all three. This is a good issue to start with, but only one of many aspects that need securing in rail.

Transportation is beginning its own Roadmap. As you can see, Roadmaps are very popular now in DHS. There were a lot of interesting tidbits in the talk. For example, pilots are ditching their flight bags for electronic flight bags the size of a cell phone that plug into the plane’s information systems.

There was an update from ISAsecure, but it was mostly the same information in our podcast and blogs. We are still awaiting the first ISAsecure certified PLC. It is coming within months according to the speakers.

Logistic Bullets

  • There were about 250 people at the peak of the day. I’m not sure what DHS expected, but my guess for success was 400.
  • It was a big improvement that the Subgroup reports were limited in number and to 10 minutes. Until there are significant results they shouldn’t get much program time. The Subgroups met on Monday.
  • They really needed a stage or something that raised the speaker with the flat, ballroom style venue.
  • The projector was low quality and screen was small which made the Langill demo less effective.
  • The energy level at the beginning needed to be higher. First couple of presentations need to have big ideas powerfully presented to set the tone.
  • Image by IntangibleArts

Fail. Not ICSJWG, but me. There were a lot of great presentations today, and I only managed to see 1.5 and give my own. One of the main benefits of ICSJWG is the crowd it draws. There are clients, vendors, gov types and lots of old friends to talk to. Unfortunately that kept me from most of the presentations on Day 2.

I did have a chance to talk with a number of the presenters and see the Powerpoints so here is what I saw or learned.

Kevin Hemsley of ICS-CERT gave a presentation that was helpful and disappointing at the same time. He went into some detail on his perspective of dealings with Luigi Auriemma, and since I have had some emails from Luigi it was interesting to hear the other side. ICS-CERT and Luigi have communicated a number of times over the past six months, but there has been no meeting of the minds. Likely causes are different goals and language issues.

I finally had my chance to publicly ask ICS-CERT about their Stuxnet handling. There was no real answer, as expected, except they had learned a lot from Stuxnet. This is really the key. When the next Stuxnet type malicious exploit in the wild occurs, will ICS-CERT be ready and respond better? The handling flowcharts they presented covered a coordinated disclosure and non-coordinated/0day. ICS-CERT is doing a good job of implementing these processes, especially this year.

They did not yet have a flowchart or handling process for the case where we learn of a vuln through widespread or targeted actual exploitations. There was a good idea, not mine, that after ICS-CERT has created this third process they should run Stuxnet through it to see if the process would have worked.

I provided a presentation on Quickdraw, Portaledge and Bandolier research results. Nothing new to loyal blog readers, but we are always surprised at how many people don’t know about these projects. Outreach continues.

Look out for SPIDERS – Smart Power Infrastructure Demonstration for Energy Reliability and Security. We will blog on this when we get the presentation, but they have deployed sensors across a variety of networks and systems to collect data for defensive efforts.

Ernie Rakaczky and Paul Forney of Invensys presented on the Security Development Lifecycle for Control Systems. The presentation talked about creating a Cyber Security Culture during development, execution/FAT/SAT, and throughout the deployed system life cycle. Offline Ernie was telling me that the SDL was actually reducing elapsed development time because fewer problems were making it to QA. They were being caught earlier. Another way to look at this is the 10% additional effort for the SDL results in a more accurate development schedule. Lots of statistics and lessons learned here and probably worth a podcast segment.

We have not been shy to call out vendors when they fail, but there is good news out there with selected vendor vulnerability handling and SDL work. They tend to go hand in hand. Unfortunately the distribution of vendor actions tends to be bi-modal with some vendors making significant progress and others still in denial. Sometimes it can happen in the same company with bimodal divisional responses.

Nate Kube presented on the Achilles Practice Certification efforts on the WIB standard and certification. We have been critical of the loose structure around this effort, and the overstatement of what it was. So it was encouraging to see these documents have been submitted as an IEC work item, details to follow. They will force a level of analysis and rigor that was missing.

Thanks to DHS for continuing to put on this event.

Image by Green_Mamba