AV ArchitectureExplainerJun 8, 2026, 3:37 AM· 4 min read· #2 of 2 in automotive

How End-to-End AI Replaced 300,000 Lines of Code in Autonomous Vehicles

The self-driving industry is undergoing a massive architectural shift, abandoning hand-coded rules in favor of neural networks that learn to drive by watching humans.

By Factlen Editorial Team

End-to-End Vision Pioneers 45%Multi-Sensor Redundancy Advocates 45%Industry Analysts 10%
End-to-End Vision Pioneers
Argue that massive neural networks trained on human driving data can solve autonomy using only cameras, eliminating the need for complex rules and expensive sensors.
Multi-Sensor Redundancy Advocates
Maintain that true safety requires a combination of AI and redundant physical sensors like LiDAR and radar to guarantee performance in all conditions.
Industry Analysts
Focus on the commercial implications of the software shift, noting how end-to-end AI lowers compute costs and accelerates global deployment.

What's not represented

  • · Traditional human drivers
  • · Public transit advocates

Why this matters

This architectural shift is what will finally allow self-driving technology to scale globally. By replacing rigid rules with adaptable AI, autonomous vehicles can now navigate unmapped cities, complex construction zones, and unpredictable human behavior, bringing safe robotaxis to the masses much faster.

Key points

  • The autonomous vehicle industry is shifting from hand-coded C++ rules to end-to-end neural networks.
  • End-to-end AI processes raw camera data and directly outputs steering and braking commands, mimicking human learning.
  • Tesla's FSD v12 deleted over 300,000 lines of code by adopting this 'photon-to-control' architecture.
  • Wayve is using foundation models to scale mapless autonomous driving to over 500 cities globally.
  • Legacy operators like Waymo maintain that multi-sensor redundancy, including LiDAR, is still required for verifiable safety.
300,000
Lines of C++ code deleted in Tesla FSD v12
500,000
Paid Waymo rides per week in 2026
500+
Cities tested by Wayve's mapless AI

The reality of self-driving in 2026 is no longer confined to closed test tracks or tightly controlled pilot programs. In cities across the United States, autonomous vehicles have become a mundane fact of the daily commute.[6][7]

Waymo, the Alphabet-owned pioneer of the industry, now operates fully driverless commercial robotaxis in ten major metropolitan areas, including San Francisco, Phoenix, Los Angeles, and Atlanta.[6]

These vehicles are currently completing over 500,000 paid rides every single week, navigating complex urban environments without a human behind the wheel.[6][7]

But beneath the surface of this massive commercial rollout, the underlying technology powering autonomous vehicles is undergoing a radical, invisible revolution.[10]

The architectural shift from modular software stacks to unified neural networks.
The architectural shift from modular software stacks to unified neural networks.

For the past decade, the industry standard was built on what engineers now refer to as "AV 1.0."[6][10]

This traditional approach relied on a highly modular software stack, dividing the monumental task of driving into discrete, manageable chunks: one system for perception, another for prediction, and a third for planning the vehicle's path.[3][5]

AV 1.0 required explicit programming. Engineers wrote hundreds of thousands of lines of C++ code to dictate exactly how a car should behave in specific scenarios—if the camera detects a red light, stop; if a pedestrian steps into the crosswalk, apply the brakes.[4][5]

The fatal flaw of this modular, rule-based approach is the "long tail" of edge cases. The real world is infinitely complex, and it is mathematically impossible to hand-code a rule for every bizarre scenario a car might encounter.[3][5]

Whether it is a person in a chicken suit chasing a dog, or a double-parked delivery truck blocking a temporary construction lane, rule-based systems often freeze or disengage when faced with situations outside their programmed logic.[5][6]

In a 'photon-to-control' architecture, raw camera data flows directly into a neural network to output driving commands.
In a 'photon-to-control' architecture, raw camera data flows directly into a neural network to output driving commands.

Enter "AV 2.0," or end-to-end AI. Instead of breaking driving down into discrete, programmed steps, end-to-end systems use a single, massive neural network that learns to drive by watching millions of hours of human driving data.[1][3]

The architecture is often described by engineers as "photon-to-control." Raw sensor data—the photons hitting the car's cameras—flows directly into the neural network, and the network outputs steering, acceleration, and braking commands without any intermediate code.[5][8]

This shift closely mirrors the revolution in natural language processing that gave the world large language models. Just as AI learned to write by ingesting vast amounts of text rather than memorizing grammar rules, end-to-end driving models learn the implicit "grammar" of driving by observing human behavior.[3][4]

Tesla became the most visible champion of this approach with the release of Full Self-Driving version 12. In a sweeping architectural overhaul, the company deleted over 300,000 lines of explicit C++ control code.[4][5]

The scale of the autonomous vehicle industry's transition in 2026.
The scale of the autonomous vehicle industry's transition in 2026.

According to Tesla's engineering team, the new system no longer contains a single line of code explicitly defining a "roundabout" or a "stop sign." Instead, the neural network understands these concepts implicitly through pattern recognition, resulting in smoother, more intuitive driving.[2][4]

The UK-based startup Wayve has taken this concept even further, pioneering what it calls "Embodied AI." Wayve's foundation models are designed to be vehicle-agnostic and mapless, meaning they do not rely on the highly detailed, pre-mapped 3D routes that traditional robotaxis require.[1][3]

Because the AI understands the fundamental principles of driving rather than memorizing a specific city's layout, Wayve has been able to rapidly scale its testing to over 500 cities globally, from London to Tokyo, using the exact same underlying model.[3][6]

This scalability has attracted major automotive and silicon partners. In early 2026, Qualcomm announced a collaboration with Wayve to integrate this end-to-end AI directly onto its Snapdragon Ride system-on-chips, paving the way for automakers to deploy advanced, mapless autonomy in consumer vehicles.[9][10]

Despite the rise of vision-only AI, companies like Waymo maintain that LiDAR is essential for verifiable safety.
Despite the rise of vision-only AI, companies like Waymo maintain that LiDAR is essential for verifiable safety.

However, the industry remains fiercely divided over whether pure end-to-end AI is safe enough for fully driverless operation. The "Great Divergence" pits the vision-only, data-driven approach against the multi-sensor, redundancy-focused philosophy of legacy players like Waymo.[5][8]

Waymo argues that while neural networks are incredibly powerful, they remain "black boxes." To ensure verifiable safety, Waymo's newly deployed 6th-generation "Driver" continues to utilize a multi-modal suite of 13 cameras, six radar sensors, and four LiDAR sensors to create a mathematically absolute 3D map of its surroundings.[6][8]

Ultimately, the transition to end-to-end AI represents a profound maturation of the autonomous vehicle industry. Whether the future belongs to pure vision-based neural networks or hybrid systems equipped with LiDAR, cars are no longer being programmed to drive—they are being taught.[4][10]

How we got here

  1. 2017

    Wayve is founded with a contrarian vision to use end-to-end deep learning instead of modular rules.

  2. 2020

    Waymo launches the first fully driverless public robotaxi service in Phoenix, utilizing a modular AV 1.0 stack.

  3. Late 2023

    Tesla releases FSD v12, replacing 300,000 lines of explicit C++ code with a single end-to-end neural network.

  4. Early 2026

    Waymo expands to 10 US cities with its 6th-generation driver, while Wayve partners with Qualcomm to put end-to-end AI on consumer chips.

Viewpoints in depth

End-to-End Vision Pioneers

The belief that AI can learn to drive purely from visual data, just as humans do.

Proponents of this approach, led by Tesla and Wayve, argue that the real world is too complex for hand-coded rules. By feeding millions of hours of human driving video into a massive neural network, the system develops an implicit understanding of the road. This 'photon-to-control' method eliminates the need for expensive LiDAR sensors and pre-mapped HD routes, allowing the technology to scale to new cities and countries almost instantly. They argue that if humans can drive using only two eyes and a brain, AI can do the same with high-definition cameras and a neural net.

Multi-Sensor Redundancy Advocates

The stance that verifiable safety requires a combination of AI and specialized hardware.

Legacy autonomous operators like Waymo maintain that while end-to-end AI is powerful, it operates as a 'black box.' If a neural network makes a mistake, it is incredibly difficult for engineers to pinpoint exactly why the AI chose to swerve or brake. To ensure public safety and satisfy regulators, these advocates argue that vehicles must have redundant physical sensors—specifically LiDAR and radar. These sensors provide absolute, mathematically verifiable distance measurements that don't rely on AI interpretation, ensuring the vehicle can 'see' even in pitch-black conditions, heavy rain, or blinding fog.

What we don't know

  • Whether regulators will eventually mandate LiDAR for all fully autonomous vehicles, or if vision-only systems will be deemed safe enough.
  • How quickly end-to-end AI can solve the remaining 'long tail' edge cases that still cause occasional system disengagements.

Key terms

End-to-End AI
A machine learning approach where a single neural network takes raw input (like camera video) and directly produces the final output (like steering commands), without intermediate programmed steps.
AV 1.0
The traditional approach to self-driving, characterized by modular software stacks, hand-coded rules, and heavy reliance on high-definition 3D maps.
LiDAR
Light Detection and Ranging; a sensor that uses pulsed laser beams to measure distances and create highly accurate 3D maps of the surrounding environment.
Foundation Model
A large, general-purpose AI model trained on a vast quantity of data that can be adapted to a wide range of downstream tasks, similar to the technology behind ChatGPT.
Edge Case
A rare, unpredictable, or extreme situation that a system might encounter, which is difficult to account for in traditional rule-based programming.

Frequently asked

What is the difference between AV 1.0 and AV 2.0?

AV 1.0 relies on hand-coded rules and separate modules for seeing, planning, and acting. AV 2.0 uses a single neural network that learns to drive holistically by observing human data.

Why did Tesla delete so much code?

By switching to an end-to-end neural network in FSD v12, Tesla no longer needed 300,000 lines of C++ code that explicitly told the car how to handle specific situations like stop signs or roundabouts.

Do self-driving cars still need LiDAR?

It is heavily debated. Companies like Tesla and Wayve believe high-resolution cameras and AI are sufficient, while Waymo insists LiDAR is essential for safety in adverse weather and complex environments.

Where can I ride in a fully autonomous vehicle today?

As of 2026, Waymo operates fully driverless commercial robotaxis in 10 US cities, including San Francisco, Phoenix, Los Angeles, Austin, and Atlanta.

Sources

Source coverage

10 outlets

3 viewpoints surfaced

End-to-End Vision Pioneers 45%Multi-Sensor Redundancy Advocates 45%Industry Analysts 10%
  1. [1]WayveEnd-to-End Vision Pioneers

    Building Embodied AI For Any Vehicle, Anywhere

    Read on Wayve
  2. [2]ElectrekEnd-to-End Vision Pioneers

    Tesla pushes end-to-end neural networks for highway driving

    Read on Electrek
  3. [3]ZenMLEnd-to-End Vision Pioneers

    Wayve: End-to-End Foundation Models for Self-Driving Vehicles at Scale

    Read on ZenML
  4. [4]FredPopeEnd-to-End Vision Pioneers

    Tesla's Neural Network Revolution: How Full Self-Driving Replaced 300,000 Lines of Code with AI

    Read on FredPope
  5. [5]SkyworkIndustry Analysts

    The Great Divergence: How Tesla's FSD v12 is Reshaping the Autonomous Driving Landscape

    Read on Skywork
  6. [6]Built InMulti-Sensor Redundancy Advocates

    Waymo Explained: Alphabet's Autonomous Vehicle Company

    Read on Built In
  7. [7]TIMEMulti-Sensor Redundancy Advocates

    Waymo's Self-Driving Future Is Here

    Read on TIME
  8. [8]WaymoMulti-Sensor Redundancy Advocates

    Beginning fully autonomous operations with the 6th-generation Waymo Driver

    Read on Waymo
  9. [9]QualcommIndustry Analysts

    Qualcomm and Wayve Advance Production-Ready End-to-End AI for ADAS and Automated Driving

    Read on Qualcomm
  10. [10]World Economic ForumIndustry Analysts

    Autonomous Vehicles: Timeline and Roadmap Ahead

    Read on World Economic Forum
Stay informed

Every angle. Every day.

Get automotive stories with full source coverage and perspective breakdowns delivered to your inbox.