With connected automobiles, the stakes for getting security right have never been higher. “What’s the worst that could happen?” is a lot more serious when you’re talking about a computer that can travel 100+ MPH.
When an industry without experience in Internet security starts connecting things to the Internet, it typically makes a number of mistakes both in how it implements secure systems, and how it interacts with the security community.
My colleague Marc Rogers and I set out to audit the security of the Tesla Model S because we wanted to shine a light on a car that we hypothesized would have a strong security architecture, given the Tesla’s team’s deep software experience. Out of this research, we hoped to be start a conversation about simple and clear security best practices for the automotive industry.
That hypothesis turned out to be correct: The Tesla Model S has a very well designed security architecture, that we believe should serve as a template for others in the industry. We also found a number of vulnerabilities that allowed us to, with physical access to the vehicle, to gain root access to two of the infotainment systems: the instrument cluster (IC) above the steering wheel, and the 17-inch touchscreen center information display (CID) in the middle of the dash. This allowed us to perform a number of tasks, such as remotely opening and closing the trunk and frunk, locking and unlocking the doors, starting the car, and stopping the car.
However, this research focused on answering the question: how can we make cars more resilient to attack, assuming attackers can get into the infotainment systems. All of the exploitation performed was done with physical access and we did not demonstrate any remotely executable exploits. There is sufficient research already done that proves cars can be exploited remotely. Further, we believe it to be a relatively conservative assumption that any browser running WebKit will be exploitable to an attacker with sufficient skill or resources.
Public Service Announcement
Before going into our process for identifying these vulnerabilities, it’s worth mentioning the safety and ethical guidelines we took into account while performing this research.
- Do not interact with any drive/safety systems. Only infotainment systems/network. While performing the initial research, we did not want to inadvertently compromise anything safety-critical on the vehicle itself.
- Do not interact with any Tesla servers. When auditing a connected device, it’s sometimes difficult to tell whether you are probing the device, or the company connected to it. We set up a number of tests (including measuring latency) to ensure that any system we were testing was, in fact, local to the vehicle. We later modified this to include some of Tesla’s infrastructure subject to the permission in their bug bounty program.
- Do not make any permanent modifications to hardware or software. The Model S is not cheap, we’d rather not break it.
The infotainment components communicate via an onboard LAN. In order to communicate with the vehicle (CAN) network, all commands must flow through the “gateway” which sits on both the infotainment network and the CAN network.
We first performed an initial information gathering exploration of the Model S and identified a number of potential attack vectors. In this exploration, we analyzed the physical hardware of the car to identify potential physical attack vectors into the vehicle. From this analysis, we found the following:
- Two removable memory cards on the CID board
- 1 USB port on the CID board
- An unknown 4-pin connector
- Various test points and diagnostic interfaces
We then performed a penetration test on all of the attack surfaces we had identified:
- Browser: Vulnerability #1, found. A WebKit-based browser, which, on the most recent vehicle we tested is running version 534.34, which is several years old and has multiple known vulnerabilities which have been used to compromise other systems. We attempted to weaponize two of them, and were able to achieve reliable browser crashes but no arbitrary memory reads or writes. Without access to a debugger on the system, blind exploitation was seeming difficult.
- Bluetooth: We found nothing particularly insecure.
- USB: We were able to connect to the USB port on the CID’s board and boot the device into an NVIDIA Tegra recovery mode. The bootloader was configured to be password protected (and we didn’t have the password), so we were unable to use this interface to extract firmware.
- Memory Cards: One of the memory cards contained a file, carkeys.tar, which included the car’s OpenVPN credentials, specifically an x509 certificate, an RSA private key, and an OpenVPN static key. The car keys of tomorrow, are apparently cryptographic in nature. The other contained a large amount of mapping data.
- Wi-Fi: The car exposes no open ports; however it does attempt to reach an OpenVPN server every time it determines there is a live Internet connection. OpenVPN was well configured and was not vulnerable to man-in-the-middle attacks that can occur when the server’s certificate and the car’s certificate chain up to the same root certificate authority.
- Unknown connector: This turned out to be an ethernet connector with a non-standard interface. We were able to strip standard Cat-5 UTP wire to determine the pin configuration and communicate with the internal network.
We then explored the internal network to map the services exposed by various components in the infotainment system. We identified over 30 services accessible on the internal network.
Two of the services were out of date with known vulnerabilities.
- Insecure DNS Proxy: The Model S runs dnsmasq 2.58. (Vulnerability #2)
- Insecure HTTP Service: The Model S mini_httpd 1.19. (Vulnerability #3)
We did not investigate the ramifications of the vulnerabilities in these services, as it was not immediately obvious how they could be used to gain access to the systems running them.
We also identified that both the CID and the IC were running X11 without any form of access control (Vulnerability #4). We had a little fun with it.
Finally, we identified two services ic-updater and cid-updater running on the IC and CID respectively, which when given the “status” command, shared a lot of internal technical state information about the infotainment system.
While in the middle of this research, others on the Internet had found the Ethernet interface and Tesla issued an over-the-air update that blocked it from being able to communicate with the internal LAN.
Knowing that the IC was still the network, and that it had a similar 4-pin ethernet cable, we maintained our access by building an interface to the back of the IC’s ethernet port and connecting it to an external ethernet switch between the IC and the internal switch so that we could maintain our access to the internal LAN.
One day, we noticed that the CID updater’s status output contained a very particular url:
firmware_download_url = hxxp://firmware-bundles.vn.teslamotors.com:4567/…
We now know where we can download the car’s firmware from (Vulnerability #5).
Around the same time, we noticed that Tesla had opened their bug bounty, granting permission to audit the security of their backend systems. We then used our VPN keys (retrieved from the memory card, above) and our knowledge about the Model S OpenVPN configuration (from our man-in-the-middle attempt) to establish a VPN connection and download the car’s firmware update.
The Firmware: 646,422,592 bytes later
The firmware bundle was a SquashFS filesystem, which is readily extractable.
We made a number of observations upon analysis of the firmware:
- This car is a server: It has log rotation and other scripts that you would find in any datacenter system
- Tesla engineers are Monty Python fans: there is a script in the car which makes a request to hxxp://firmware-bundles.vn.teslamotors.com:4567/custom-packages/knights-who-say. The response body contains thousands of lines of the word “ni”
- There were no private keys in the firmware bundle
- There was an encrypted password file (shadow) for the IC
We attempted to crack the shadow file and found that a number of user accounts on the IC had very weak passwords that we were able to crack quite quickly. (Vulnerability #6).
After logging into the IC with these credentials, we immediately found that the account was a sudoer and we were able to become root.
From here, the next step was to gain access to the CID, which was the “main” infotainment system. Upon further firmware analysis, we found that the CID contained a key rotation scheme whereby the car would receive a new random security token every 24 hours from the Tesla “mothership” (actual name of the server). The CID would then set the “tesla1” account password to this token, and inform the IC of the new token. The IC would then store this token in plaintext on its filesystem.
We used this token from the IC to log into the CID as “tesla1,” which was also a sudoer. We then had root on both the CID and the IC.
From here, we were able to identify a number of additional findings:
- With local access to X11 services, it is possible to disrupt the instrument cluster’s display while driving.
- The CID contains a DSA key it uses to log into the IC via SSH.
- We can re-enable the ethernet port by sending a UDP broadcast packet using a key derived from the security token every 30 seconds. No more disassembly of the dash needed.
- With the VPN keys, we can retrieve the latest security token (changed every 24 hours) without disassembling the car and hacking the IC’s ethernet interface.
- The firmware bundle contains what seem to be drive system firmware images. We did not evaluate any of the security mechanisms of this update process and welcome further research in this area
- The Tesla Service Wi-Fi network used in service centers used a permanent, static WPA key that is contained in one of the binaries in the firmware.
Controlling the Vehicle?
In order to determine to what extent having control over the infotainment system would allow control over the vehicle as a whole, we explored how the “legitimate” vehicle control systems on the CID worked.
The Model S internal lan is relatively high traffic, with UDP broadcast messages occurring roughly 500-1000 times per second. Identify which of these messages was a meaningful vehicle control message proved challenging.
In order to make sense of this, we built a tool, tescat (open source on github) that listens for UDP broadcast packets on a particular port and identifies any time it receives a packet that it has never seen before (uniqueness calculated over a fixed window of bytes).
By baselining the traffic then running various vehicle control commands from the CID and smartphone app, we were quickly able to identify the packets of interest and the service on the CID responsible for sending them.
Some reverse engineering of this service showed that the Tesla Model S does NOT seem to send raw CAN frames from the infotainment system to the vehicle. Instead, there is a Vehicle API (VAPI) whereby the CID asks the gateway to perform any one of an “allowed” set of actions.
The existence of a gateway that operates in this manner is very good for the resilience of the car, for it means that even if the infotainment network is compromised, that does not grant an attacker the ability to inject raw CAN frames into the vehicle network. They can only send “legitimate” VAPI requests.
The assertion that infotainment compromise does not grant an attacker the ability to directly communicate with vehicle systems, of course, reduces down to the gateway’s ability to withstand attack. We did not audit the security of the gateway in depth and welcome further research in this area.
While we identified a number of “legitimate” VAPI commands that we were able to execute, perhaps the most significant was the ability to “power off” the car. In our testing, we found that we were able to issue the command while the car was driving, and the car would follow through with powering off. At low speed (below approximately 5 MPH), the car would immediately apply its brakes and lurch to a stop. At speeds above 5 MPH, the car would effectively shift into neutral and no longer accelerate, without applying the brake. Notably, the driver still retains control over steering and braking of the vehicle at this point.
From a safety standpoint, we tested all of these drive system interactions in a closed environment, never exceeding 10 MPH.
What is Tesla doing about this?
We’re proud to say that we’ve had a great working relationship with Tesla and that they were able to push an OTA update to every single Model S in a very timely manner (a matter of 1-2 weeks).
This update fixes a number of the vulnerabilities we disclosed and increases the hardening of the system as a whole, specifically by locking down which systems can communicate with the gateway and increasing the ability of the CID to withstand browser compromise.
Our interaction with the team there was extremely productive as it’s clear that they are taking the security of their vehicles extremely seriously.
Summary of findings
In order to gain access to the Model S, we had to conduct a multi-stage attack leveraging multiple vulnerabilities. Compromising this car, had more in common with an advanced enterprise threat, where the attacker needed to move laterally into multiple systems, than a simple embedded systems hack.
To simplify our findings, we produced a “kill-chain,” shown below, that details all of the methods we were able to use to move laterally within the Model S to achieve what would be a malicious attacker’s goal of disrupting the driver or the vehicle itself.
Even though we found a number of weaknesses that Tesla is addressing, we believe their team made a number of very good architecture decisions:
- Over-the-air update process: If Tesla needs to fix a security vulnerability, they can do so with a simple update which takes the driver only one push of a button to apply. No recall or USB key in the mail needed. Further, the driver does not need to subscribe to a cellular data service in order to receive these updates, they are provided free of charge.
- VPN configuration: The Model S uses a properly configured VPN which does not suffer from common misconfiguration issues.
- Account password rotation: The onboard account passwords are rotated every 24 hours.
- Isolation between vehicle systems and infotainment systems: The Model S includes a gateway system that exposes a clear API to the infotainment systems, rather than the infotainment systems connecting directly to the CAN.
There are also a number of aspects of the Model S we believe can be improved:
- “Tesla Service” Wi-Fi uses a static WPA key. Would be better to use WPA Enterprise authentication so no static key is shared between vehicles.
- Perimeter security model. The Model S has a very strong set of perimeter security, which is good; however, the internal systems do not have the same level of security. We believe the car’s security model should assume that an attacker is able to get on the infotainment network but that doesn’t give them any meaningful advantage.
- Plaintext credential storage. The VPN keys and security token are both stored in plaintext. It would be better to store these in a hardware store (e.g. TPM, TrustZone, Secure Element)
- Traffic between infotainment systems unencrypted and sometimes unauthenticated. Services communicating on the infotainment LAN do not use any form of encryption, so that an attacker is able to observe all traffic. Further, only a select few services perform authentication. As part of moving from a perimeter security model to a component-level security model, all communications between components should assume zero trust of the network itself.
Best practices for the industry as a whole
We hope that the this research will help advance the conversation around automotive cybersecurity, particularly in establishing several best practices that we believe every automobile manufacturer should follow when building connected automobiles:
- An over-the-air update process: Ideally without the owner having to subscribe to a separate service.
- Isolation of vehicle and infotainment systems: With this in place, it’s important that any gateway systems receive an extreme amount of security scrutiny.
- Hardening each individual component: A resilient automotive cybersecurity architecture should assume that attackers will compromise some component (e.g. the web browser). That single component compromise should not affect the functionality of the system as a whole.
There are a number of other projects also working on advancing the conversation, for example the Five Star Automotive Cyber Safety Program. We welcome all the help we can get in making cars more resilient to cyberattack.
Finally, we’d like to thank everyone who contributed to this project in innumerable ways: John Hering, Alissa Rogers, Vishal Verma, Meghan Kelly, Steve Regester, Michael Bentley, Alicia diVittorio, Kurt Opshal from EFF, Joe Grand, Tim Wyatt, Blaine Schanfeldt, Daniella Vallurupalli, Jeremy Linden, Tim Strazzere, Caleb Fenton, Nam Bui, Lauren Hampton, Dana Palmie, Heather MacKinnon.