Behind the curtain: Tesla’s inflated statistics and displays of AI competence

Second of two parts
Part one

As Tesla employees watched videos of FSD’s missteps, the company’s board and CEO ramped up their claims about the technology’s safety and readiness for full autonomy. For much of last year, leaders at Tesla promoted the 10-times-safer claim.

“A car on FSD being 10x safer” will drive sales, Tesla CFO Taneja said in a July earnings call. “Even at $99 a month, it’s like you’re getting a personal chauffeur for almost $3.33 a day.”

A key problem with Tesla’s methodology stems from one comparison error that inflated Tesla’s claimed level of safety by a factor of three. The automaker counted Tesla crashes with airbag deployments and compared them with federal data on all crashes in which a tow-truck removed a vehicle — a far less restrictive criterion. Crashes requiring tow trucks often aren’t severe enough to trigger airbags.

Tesla took this apples-and-oranges approach even though apples were readily available for comparison: The federal data it used included crashes where airbags deployed. This flawed methodology produced the finding that Teslas using FSD or Autopilot travel 10 times farther between crashes than the average human driver.

The more valid comparison — using airbag-involved crashes for Teslas and all other cars — finds Teslas using the driver-assistance systems travel about three times farther between crashes where airbags deployed, according to an analysis performed for Reuters by Marco Benedetti, an assistant research scientist at the University of Michigan Transportation Research Institute and a former NHTSA statistician. Two other traffic-safety researchers vetted Benedetti’s calculations and agreed with the findings.

But that doesn’t mean FSD is actually three times safer than the average driver, Benedetti said, because of several other flaws in Tesla’s methodology.

Tesla tweaked its approach in November to include only data for vehicles using FSD and exclude those with Autopilot. Including Autopilot had increased Tesla’s claimed miles between crashes because it’s a less sophisticated system intended only for highways – where cars rack up miles and crash far less frequently than on urban streets. The company, however, continues to use the flawed airbag-crash comparison on its website to claim FSD is seven times safer than the average human driver, or about 85% safer expressed as a percentage change.

Several other flawed measurements employed by Tesla cast doubt on whether FSD is any safer at all, Reuters found.

The automaker, for instance, doesn’t consider vehicle age when comparing the crash rate for its cars to the national rate. Tesla compares its vehicles – which are just 4.1 years old on average, according to S&P Global Mobility data – against all U.S. vehicles, which have an average age of 12.8 years. That skews the results, 10 safety researchers told Reuters, because almost all automakers have more recently started offering groundbreaking safety features, including blind-spot monitoring and automatic emergency braking, across their lineups.

Tesla also reduces its crash tally by only counting wrecks that happen either with FSD switched on or within five seconds of the feature being turned off. The U.S. government, by contrast, requires automakers to report crashes occurring within 30 seconds of an advanced driver-assistance system being deactivated.

Tesla says FSD could save more than 32,000 lives and prevent more than 1.9 million injuries annually. Some traffic-safety researchers called those figures meaningless because they are based on the unrealistic assumption that every US vehicle, including freight trucks and crash-prone motorcycles, would be replaced by an FSD-enabled Tesla car —and that every Tesla car is, in fact, at least seven times safer than the one it replaces.

Waymo’s more rigorous approach

The premise of Tesla’s safety statistics is also flawed because FSD isn’t a truly autonomous system, 10 traffic-safety researchers said. Tesla isn’t comparing its technology to human drivers, as executives say. Instead, the automaker is comparing the average human driver to another average human —one driving a Tesla using FSD. Tesla also fails to consider that these drivers can turn FSD on and off — and research shows motorists often avoid using advanced driver-assistance systems in complicated traffic situations where the tech feels unsafe to them. Tesla’s own data shows FSD is used mostly on highways.

Alphabet’s Waymo, by contrast, compares its fully driverless robotaxis, now deployed in 11 US metropolitan areas, to human-driven vehicles in similar conditions.

Waymo takes a more rigorous approach than Tesla, examining crash data in markets where it operates and adjusting for the types of roads and neighborhoods its robotaxis traverse. Waymo focuses on specific crash rates — such as those with airbag deployments or serious injuries — for both its cars and human-driven cars in the same markets.

“We’ve got to be really careful with the language we use,” said John Scanlon, a Waymo safety researcher. “You need very specific research questions and very specific conclusions.”

Waymo also points out shortcomings in its data and collaborates with outside researchers to publish its safety statistics in peer-reviewed journals. Tesla, by contrast, seeks no peer review and publishes only top-line statistical safety claims while keeping its underlying crash data for Tesla cars secret.

Videos of Teslas striking dogs, cats, and deer

Inside Tesla, data labelers get an unvarnished view of FSD safety. Three former employees described several videos showing Teslas failing to recognize animals and striking them at speed – without braking.

Five former employees said specific teams focused on FSD’s problems recognizing school buses. That’s a concern raised by a technology-safety group called the Dawn Project, which aired ads during the 2023 and 2024 Super Bowls showing videos of FSD-enabled Teslas failing to stop for buses with stop signs and flashing lights.

Two former employees said they saw similar video clips inside Tesla.

Five former data labelers described a harried, disjointed work environment, with priorities shifting based on directives from Musk and FSD engineers. The data-labeling unit struggled with chronic turnover because of the monotonous work and generally low pay, they said.

Tesla higher-ups often launched new projects in reaction to news reports or social media posts showing FSD making mistakes, four former employees said. One described an effort to address how sunlight could obscure cars’ exterior cameras. That was prompted by a social-media video showing how light reflecting off a passenger’s watch blinded one of the cameras, shutting down FSD. Another effort on railroad crossings followed news reports about FSD-driven Teslas failing to stop at them.

FSD clips also regularly showed speeding, five of the employees said, which engineers and others up the chain treated as a low-priority problem.

One employee said labelers saw Teslas regularly exceeding speed limits by 20 to 30 miles per hour after the automaker introduced an FSD “Mad Max” mode enabling more-aggressive driving. Another labeler reported seeing an FSD-piloted vehicle traveling 60 mph in a 25-mph zone.

Public robotaxi displays

As Tesla employees struggled to train FSD, Musk touted Tesla’s self-driving capabilities in October 2024 in a flashy robotaxi unveiling at the Warner Bros. studio lot near Los Angeles. The invite-only crowd cheered as Musk gestured to about 20 prototypes of the two-door “Cybercab,” which has no steering wheel or pedals, crawling around the studio.

“The cars are just going by, with no people,” he said.

Musk has said Tesla’s software is designed to work anywhere, navigating unfamiliar landscapes in real time. But for weeks preceding the Cybercab event, staff tested the prototypes every night from 6 p.m. until dawn, collecting video of the route the cars would follow at the launch, according to two former data-labeling employees. Labelers spent hundreds of hours annotating curbs and road markings on the video to prevent embarrassing incidents, the employees said.

Waymo performs such mapping on a large scale before launching in specific cities – an approach that Musk has repeatedly dismissed as too costly and slow. Musk in 2024 derided Waymo’s “very localized solutions” as “quite fragile.”

After the Warner Bros. event, Musk declared on a January 2025 earnings call that Tesla would be launching robotaxis in June 2025 in Austin. He touted the technology as a “generalized AI solution” that didn’t require “high-precision maps of a locality.”

For months before the Austin launch, however, Tesla extensively filmed features in a limited robotaxi zone to map the area, capturing stop lights, road signs and other features. Data labelers annotated that video to ensure the software could handle challenging scenarios, including passenger pickup and responding to emergency vehicles, according to two employees with direct knowledge of the matter.

The Utah data-labeling staff, three of the employees said, doubled in the half-year before the Austin launch to about 300 workers. The department, they said, worked primarily on projects to make the carefully controlled Austin test go smoothly.

As Tesla data labelers prepared for the rollout, the software was still erratic, two of the employees said. With each FSD update, some driving behaviors improved. Others worsened. In the Utah office, two large screens displayed statistics on miles between driver interventions for FSD – a key autonomous-driving safety metric.

“It would go up and down like the stock market” with no consistent improvement, one of the former employees said.

The vehicles hit the streets with two sets of human safety monitors available to grab control: one sitting in the front passenger seat, and others watching remotely. In Utah, labelers could see on videos when the remote monitors took over the vehicles. One former employee said the Austin routes were designed for a limited area so the cars’ software could be trained extensively on specific maneuvers on particular streets.

“It was like, ‘OK, we trained a car’” to operate in a restricted zone, the person said. “You can’t get creative outside of that.”

Four of the sources said scaling up safely could take years. In July, a month after the Austin robotaxi launch, Musk predicted the service would expand to serve half the U.S. population by the end of 2025.

In January, Musk falsely claimed Tesla operated 500 “robotaxi vehicles” in Austin and the San Francisco Bay Area, adding he expected that to “double every month” on an “exponential curve.” Musk has said Tesla operates a “robotaxi service” in the Bay Area when it in fact only operates a ride-hailing service under a state permit, typically used by chauffeurs, that requires a human driver.

In reality, nearly a year after the Austin launch, Tesla still operates only about 50 robotaxis there, according to a recent slide presentation by city officials. The vehicles traverse a limited and carefully mapped zone, three of the sources said. Some still have human safety monitors in the front passenger seat, based on recent observations by a Reuters reporter.

In April, Tesla said it was rolling out robotaxis in Dallas and Houston, alongside maps showing the serviced areas.

Reuters reporters who recently tested the service in both cities found long wait times and erratic availability. On three occasions when a reporter managed to get a ride in Dallas, the robotaxi wouldn’t drop him at his downtown destination within Tesla’s advertised service area.

Each time, it left him about a 15-minute walk away. – Rappler.com