The official Waymo weblog: Waymo considerably outperforms comparable human benchmarks over 7+ million miles of rider-only driving
.png "Author: The Waymo Team")
Security leads every part we do at Waymo. This yr alone, Waymo has served over 700,000 ride-hailing journeys with public riders and no human driver. We couldn’t have hit that milestone with out placing security entrance and middle, and we’re working onerous to enhance the measurement, transparency, and efficiency of our fleet.
Our new analysis discovered that Waymo Driver efficiency led to a major discount within the charges of police-reported and injury-causing crashes in comparison with human drivers within the cities the place we function.
“These experiences signify a good-faith effort by Waymo to judge how the security of its autonomous driving system compares with the security of human driving. The outcomes are encouraging and signify one step in our evolving understanding of autonomous driving security,” stated David Zuby, chief analysis officer of The Insurance coverage Institute for Freeway Security (IIHS) after reviewing the papers.
How the Waymo Driver compares to people
There’s no single metric that may present the security of autonomous driving. We use a number of totally different analysis metrics and strategies aiming to get a whole image of security. Considered one of many metrics that can be utilized to observe the rider-only service after it has launched is the variety of car crashes per mile of driving in comparison with people.
Within the efficiency research, we in contrast the Waymo Driver’s crash charges to human drivers’ on a number of totally different benchmarks. This research is likely one of the first to check general crash charges utilizing knowledge from absolutely autonomous operations solely, slightly than a mixture of absolutely autonomous driving and testing with a human behind the wheel. Not like the current research by Swiss Re that centered on crashes leading to Waymo’s legal responsibility claims, this research consists of all Waymo crashes, whatever the Waymo car’s position within the crash, and with any quantity of property harm. It additionally makes use of publicly out there knowledge, which permits different researchers to copy the outcomes.
Waymo’s knowledge was derived from crashes reported underneath NHTSA’s Standing Common Order (SGO), over 7.14 million absolutely autonomous miles pushed 24/7 by means of the tip of October 2023 throughout Phoenix, San Francisco, and Los Angeles. That knowledge was then in comparison with related human crash charges leading to police experiences, accidents, and/or property harm.
When contemplating all areas collectively, in comparison with the human benchmarks, the Waymo Driver demonstrated:
- An 85% discount or 6.8 occasions decrease crash fee involving any harm, from minor to extreme and deadly circumstances (0.41 incidence per million miles for the Waymo Driver vs 2.78 for the human benchmark)
- A 57% discount or 2.3 occasions decrease police-reported crash fee (2.1 incidence per million miles for the Waymo Driver vs. 4.85 for the human benchmark)
Which means over the 7.1 million miles Waymo drove, there have been an estimated 17 fewer accidents and 20 fewer police-reported crashes in comparison with if human drivers with the benchmark crash fee would have pushed the identical distance within the areas we function.
When in comparison with crash charges in San Francisco, Phoenix, and Los Angeles, individually, the Waymo Driver considerably outperformed native respective human benchmarks as properly (although the comparability in LA doesn’t but have sufficient mileage to be statistically important). Notably, native human benchmarks different from one metropolis to a different — for instance, San Francisco had the very best fee of crashes the place an harm was reported with 5.55 incidents per million miles, which was roughly thrice increased than the nationwide common.
P.c discount for the Waymo Driver in police-reported and any-injury-reported crashed car charges in comparison with human benchmarks
Normally, the Waymo Driver additionally demonstrated decrease property harm charges in comparison with the human benchmarks. Nonetheless, the benchmark charges themselves different significantly relying on the human knowledge supply, even throughout the similar location, so warning ought to be taken when decoding these outcomes.
Producing a legitimate comparability of human and AV efficiency
Figuring out legitimate and comparable human crash benchmarks is vital for understanding autonomous driving expertise’s efficiency. Regardless of public accessibility of each human and autonomous car (AV) crash knowledge, evaluating the 2 comes with its share of challenges. Our benchmarking paper goals to make sure a good comparability between AV and human driving by addressing the most typical errors and biases and establishing legitimate benchmarks from the cities wherein Waymo operates.
There are two essential sources of statistical bias to manage for when evaluating human and autonomous driving. The primary is human crash underreporting. Whereas the info on human crashes that result in accidents or property harm is pretty strong, numerous human low-severity crashes — like hitting some highway particles or minor “fender benders” — should not reported to police. In distinction, AV firms report even essentially the most minor crashes to be able to show the trustworthiness of autonomous driving on public roads. For instance, solely 21% of crashes that Waymo has reported to NHTSA up to now have resulted in a filed police report, whatever the get together at fault.
The second is variations in driving circumstances and/or car traits. Public human crash knowledge consists of all highway varieties, like freeways, the place the Waymo Driver at present solely operates with an autonomous specialist behind the wheel, in addition to numerous car varieties from industrial heavy automobiles to passenger and bikes.
These variations imply that changes must be made to human crash knowledge earlier than evaluating it to AV crash charges. Simply as you wouldn’t evaluate biking in a road-race to mountain biking with out accounting for variations in altitude, terrain, and different components, comparisons between human and autonomous driving have to account for variations too.
To verify the human crash knowledge is legitimate and comparable, the harm and property harm benchmarks in our 7.1 million mile research both included underreporting changes for police-reported crashes or had been derived from naturalistic driving research databases, which equip automobiles with sensors to report driving and usually tend to catch low-severity crashes. The methodology we used was based mostly on greatest practices from a literature evaluate of 12 previous research and 1 e book evaluating AV and human crash charges.
At Waymo, we imagine that worldwide standardization for legitimate evaluation of AV crash knowledge is required. We’re searching for alternatives to have interaction the neighborhood of visitors security specialists and requirements organizations to advance this work sooner or later.
Our complete strategy to security
Whereas being safer than human drivers in mixture is a vital facet of AV security, it’s not ample by itself. The evaluation of crash charges is only one amongst many strategies used to judge the Waymo Driver’s efficiency, akin to how a routine well being checkup entails analyzing numerous features of bodily well-being.
Our strategy goes past security metrics alone. Good driving behavior issues, too — driving respectfully round different highway customers in dependable, predictable methods, not inflicting pointless visitors or confusion. We’re working onerous to constantly enhance our driving conduct throughout the board.
By means of these research, our aim is to supply the newest outcomes on our security efficiency to most people, improve transparency within the AV trade, and allow the neighborhood of researchers, regulators, and lecturers finding out AV security to advance the sector.
You’ll be able to learn our two newest security analysis papers and be taught extra about our general strategy to security here.