Currently giving Binarly a spin, I got to run analysis on a bunch of firmware, I’ve been wanting to have a look at for a long time, with one special trait, it’s automated. Upload, wait a few minutes and then dive into the results. I started comparing the amounts of vulns, the criticality, rated a few devices myself and then got stuck on the question, what significance my comparisons have. Well, none, because I asked the wrong question. The right one should have been: How well is the target device maintained?…

If you’ve used EMBA or FACT in the past, you’ll quickly start enjoying Binarly , because it actually works reliably. Create a device, update the firmware image and then wait a few minutes. Done! Easy :)

My Test Run

Having spent some time in hardware and embedded, railway, telco and looking many different more or less crazy devices. It was super easy for me to select a few devices to use for my initial test run:

I used the last firmware version for each of the devices, which I downloaded during the last 2..3 weeks. As you’ve probably noticed from just dropping the amount of vulns, neither the list of devices nor the number of vulnerabilities is the important part of this post!

Why are the Numbers so High?

Binarly isn’t a typical static vulnerability scanner. Where typical classical scanners would identify i.e. a library by its hash and then do a look up for the library to see by which CVEs it’s affected, Binarly looks a little deeper. By analyzing the included binaries and libraries it creates a full dependency tree and thus doesn’t only count a lib with a major vuln as one CVE but additionally flags the binaries using the lib as potentially vulnerable. In addition, while doing the analysis it also performs a little magic to calculate a “code based” exposure score.

My Principle

With a background in pentesting and security research, I’ve always been very critical concerning vulnerability scanners and automated testing tools, as their results often distract from the actual issues. Who hasn’t seen companies simply sell a Nessus scan as a pentest? Thus, for me, automated scanners where always a part of quality assurance. For embedded devices I’d say: Run the firmware through a tool like Binarly and if it lights up like a Christmas tree, go back, don’t use it and don’t waste money on a manual pentest. In return, if it doesn’t do thorough pentest, and see whether a creative person can find logical flaws and dynamic issues.
Looking at the list of findings above, I noticed that my “lights up like a Christmas tree” is probably highly misleading, as none of the products listed above actually looks good!

A New Perspective

Reading through the combined list of vulnerability, for many of them I can directly say that while they may be highly critical, in the typical usage scenarios, the exposure is so small, as risk acceptance is trivial.
The following shows a list of my top ten vulnerabilities

• CVE-2014-0160
• CVE-2023-44487
• CVE-2012-1823
• CVE-2024-4577
• CVE-2014-3566
• CVE-2014-3566
• CVE-2014-0224
• CVE-2015-7547
• CVE-2020-8617
• CVE-2024-2961

Just looking through the years a critical vulnerability from 2012-2015 is by far worse than the ones from 2024 and 2023. Yes, even these vulns are two years old, but with a limited exposure, they might be in acceptable risk. In return, a vuln from 2014 like CVE-2014-3566 implies that the overall base firmware is probably pretty old, because why would one not add a current SSL version to a current setup?
As an example, let’s have a look at the AVM FritzBox 5690 Pro:

So, while it looks a little red, the only underlying issue is the use uClibc from 2020-03-23. Here it’s highly possible, that the existing vulnerabilities could only be exploited locally, which is often put out of scope for embedded devices, or the relevant functions are reachable at all. As such, with a lack of current vulns, the FritzBox looks well maintained.
To carry on with the Telekom Digitalisierungsbox Basic with a firmware from 05/2023.

Looking through the other devices:

• The GIRA HomeServer has an outdated busybox version
• The Weidmueller Modbus Ethernet adapter has outdated busybox dropbear and pppd versions
• The Phoenix Contact mGuard has outdated openvpn and busybox versions
• The Wieland LTE Router has outdated openvpn and ssh versions
• The Teltonika TRB140 has an outdated copy of libnghttp2.so in addition to busybox and openvpn

Next to all of them have outdated SSL versions.

How Critical are Critical Vulnerabilities?

An outdated glibc can happen to the best Linux distributions, as it can be hard to update. busybox on embedded devices is only rarely exposed intentionally and often runs as an emergency shell. Thus, these issues could be treated as being forgivable. In return an outdated openvpn on a firewall/router with VPN access could be a massive issue. The same could apply to a router with an outdated PHP web interface, even though the interface shouldn’t be exposed to potential attackers anyways.
Thus, the question, how critical critical vulnerabilities are is “it depends”

How well is the Device Maintained?

This is the question I will try ask in future, even though I’m not sure yet, how to answer is. For now, my factors will at least be:

• How old was the base firmware when it was initially released?
  • Was it built with a current bootloader version
  • Was it built with a current Linux Kernel? (If it has one)
  • Where basic components current? (I.e. PHP, busybox)
• Are significant components kept up to date?
  • I.e. Is the openVPN version kept up to date on a VPN router?
  • Is the PHP versions kept up to date on a device with a PHP webinterface?
• Are updates just hotfixes or actually upgrades of components?
• I.e. are only vulnerable components updated?

I guess over time I will have to create a small metric for this.

And Now?

The next time you see a table with vulns or a product lighting up like a Christmas tree on a vulnerability scanner, take a step back and work out whether it’s really as bad as it looks. Ask yourself what the critical components are and whether they are affected, look at the vulnerabilities in context and well, work out if they’re a practical issue.
And if they are, well, go a different, better maintained device :)

P.S.
Also don’t forget: No matter how small the amount of vulnerabilities, it doesn’t tell you anything about potential logical flaws or bad passwords.