AI
How Teleoperation Works Inside an Autonomous Vehicle
Zoox’s robotaxis do not drive themselves through every situation. When the AI hits a construction zone it cannot resolve, it sends a request to a human operator at a remote center. The operator reads the camera feed, reasons about the scene, and sends suggested waypoints. There is no steering wheel on their side.
“vehicle is not doing this by itself.”
- Zhisheng Hu
The AI stays in full control throughout, treating the waypoints as advice it can accept or ignore. Assisted teleoperation is the bridge between AI confidence limits and real-world unpredictability.
Why Validation Alone Is Not Enough
Zoox runs repeated validation every day: define a hard scenario, place a trained operator at the teleoperation tool, check whether the vehicle reaches the destination without incident. Pass means no collision. But that bar assumes every command the operator sends is benign.
“We also need to prepare for the worst.”
- Zhisheng Hu
Human operators make mistakes. Scaled to hundreds of daily scenarios, each with dozens of waypoint and timing variations, manual testing cannot reach the dangerous edge cases. That gap is what motivated the fuzzing approach.
Applying Fuzzing to the Teleoperation Command Layer
Hu’s team applied software fuzzing to the teleoperation command layer. Zoox vehicles collect millions of miles of data daily. The team extracts simplified snapshots from those logs: vehicle position, the teleoperation command sent, and the speed and trajectory of every other agent on the road. Those snapshots become fuzzer seeds. Teleoperation parameters then define mutation axes, shifting waypoints laterally or longitudinally to produce paths that look plausible. Each mutated scenario runs in a high-fidelity simulator. The monitor watches for near-misses, feeds results back, and the fuzzer generates tighter mutations next time.
Three Crashes the Fuzzer Found
After running hundreds of base scenarios, the fuzzer generated over 50,000 command variants. Three cases stand out. In the merging-from-parking collision (14:36), the fuzzer iterated until it found a waypoint the AV followed straight into an oncoming vehicle. In the reverse-into-intersection side collision (17:00), analysis of the driving log found a bug in the prediction module that only triggers under a precise combination of timing and agent paths.
“subtle bug might sit there all the time.”
- Zhisheng Hu
In the right-turn emergency-stop collision (19:40), a conflicting stop command issued mid-turn eventually found the exact moment that left the AV blocking the intersection.
Closing the Loop and Expanding the Attack Surface
Each collision the fuzzer finds starts a root-cause cycle: engineers pull the driving log, identify the bug, retrain the affected module, and rerun the scenario before any software ships to public roads. The fix generalizes to similar scenarios, not just the specific case found. Zoox is now applying the same logic to the perception module, injecting distorted sensor inputs to find how far object-location errors can drift before downstream decisions break, and to the planner, which gets stress-tested against rare aggressive cut-in events.
Q&A
Are you using the same fuzzing to change the perceived world — fuzzing inputs from different modules in the AV? Yes: Zoox fuzzes both the input to the perception module (raw sensor data) and the output (object detections), using real-world perception errors from their driving logs to calibrate realistic offsets. ▶ 25:32
Instead of using the fuzzer to find crashes, why not use it to find the safest route — flip it to the positive side? Hu confirmed Zoox does generate nominal test cases automatically from the same framework to train the AI on safe routes, but kept that out of scope for this Black Hat presentation. ▶ 27:08
Notable Quotes
vehicle is not doing this by itself. Zhisheng Hu · ▶ 2:08
We also need to prepare for the worst. Zhisheng Hu · ▶ 8:20
subtle bug might sit there all the time. Zhisheng Hu · ▶ 19:33
Key Takeaways
- A fuzzer seeded from real-world driving data generated over 50,000 teleoperation command variants for Zoox’s AV fleet.
- Prediction-module bugs that only fire under precise multi-agent timing combinations are invisible to manual test cases.
- Each collision found in simulation drives a root-cause fix and retrain cycle before any code ships to public roads.