A few months ago I was lucky enough to do a lab assessment demoing a secure control system network. One component of the lab network got my attention a bit: an embedded Network Time server that gets its time from GPS.
Its sole function in life is to get time via a GPS signal and share with the rest of the network via the NTP protocol. To do this, a typical Time Server device runs embedded Linux and includes FTP, Telnet, SSH, Web, and SNMP services, as well as the expected NTP and sometimes unexpected daytime protocols. Also, these servers seem to love carrying additional open ports: echo and other services are sometimes left enabled.
While I didn’t find any hidden features in the product on our assessment, it got me to thinking: what other NTP servers are out there that might have security issues? And how often do people pay attention to these ‘appliances’?
Time Servers are particularly interesting to me because of where they’re found: very large data centers may have them, but they seem to be most likely in either industrial or high-speed financial networks.
It turns out that many, many NTP servers run embedded Linux. On a recent weekend I went window-shopping on eBay to find manufacturers, then grabbed a half-dozen firmware images and found 3 major vendors running embedded Linux with trivial-to-exploit security issues. I wasn’t looking for ‘actual exploits,’ that would require debugging code, but rather silly mistakes like undocumented accounts that might allow command-line access and known-exploitable versions of server services.
Most embedded Linux NTP servers (and most embedded linux boxes in general) run BusyBox. If they provide SSH access, they always seem to run DropBear.
The most common misconfiguration by far is enablement of a ‘root’ account that can be used for login. One device had disabled the root username but set another (undocumented) account name to id 0. The misconfiguration is a combination of failing to restrict the root shell, as well as failure to configure the ssh server and securetty files to disallow remote root login via telnet and SSH. When in doubt, vendors ought to disable root entirely and stick to using sudo for privileged commands
Embedded products have been rising as security targets over the years. Many of them can be easily identified remotely, and many have trivial security issues. Couple this with the fact that few vendors make firmware upgrades easy to manage mean that these boxes sit around unpatched for years, waiting to be the gateway into a network. As a target, an embedded device running Linux would be appealing to a bad guy. Embedded products are often overlooked in network assessments and they make forensics far more complicated. Plus, a free Linux system makes a great platform to pivot and compromise other systems.