RuggedCom was first contacted by Justin Clarke in April 2011 concerning backdoor access to their switches and serial converters. Late on Friday, they announced that they would remove the account from their devices, and that the change would only take a few weeks.
From the press release (notably written by RuggedCom’s VP of Marketing), the backdoor sounds like code cruft left over from the development process. I smell something fishy, though. If the code is really just development cruft, it should be easy to remove. RuggedCom should have removed it when Justin Clarke contacted them a year ago, or should have removed it when ICS-CERT contacted them months ago. They did not, though. Take another look at Justin’s reported timeline:
Apr 2011 – Vendor notified directly
Jul 2011 – Vendor verbally acknowledges knowledge of backdoor,
and ceases communication.
Feb 11 2012 – US-CERT notified
Mar 12 2012 – Vendor responds to US-CERT.
Apr 06 2012 – Due to lack of further contact by vendor, CERT sets
public disclosure for April 13 2012
Apr 10 2012 – Vendor states they need another three weeks to alert
their customers, but not fix the vulnerability.
Apr 11 2012 – Clarification requested regarding need for additional three weeks.
Apr 23 2012 – No response from vendor.
Apr 23 2012 – This disclosure.
In medical parlance, RuggedCom addresses the symptom but not the disease in their press release. The disease, in this case, is a lack of a methodical development process that has any awareness of security. RuggedCom clearly does not include security as a part of its development lifecycle, at least not in their switch and serial converter lines. This ‘developer backdoor’ made it into release. Nobody and no process at RuggedCom stopped it, and RuggedCom has no process to address security concerns in already-released products. They were not going to fix it at all until Justin went full disclosure.
This is bad because RuggedCom’s product is not software, it is hardware and firmware. Upgrading a field-deployed device like this is expensive and can only be done at a time when entire networks of end devices (PLCs, RTUs, relays, etc) can be offline. That is not often. It is a cost that is passed on to RuggedCom’s customers in downtime and risk, and a cost that RuggedCom is no doubt seeing in overtime as its workers rush to recompile and test firmware on all hardware revisions (I certainly hope they take this step, anyway).
What I would like to see in RuggedCom’s response is something that tells me they are aware of the risk that they have placed on their customers, and that they are changing their development process to include security checks. I would like to see them start doing internal code reviews, and maybe even bring in a Justin Clarke as an external security consultant to check out their products. It is never too early in the development process to run things by a security guy or gal. Fixing a backdoor is far more expensive to do later.
In short, I would like to see all vendors of broken ICS equipment start asking why, or at least to give me some confidence that they are doing so in private. I think the root cause in this case is going to lie somewhere around the VP of Marketing making RuggedCom’s announcement — there either isn’t a ‘security’ group at RuggedCom or the head of it has no visibility and no mandate to provide information.
Image by spi516