. Tesla claims:
A 40% crash rate reduction with the autopilot
as compared to no autopilot, over an 18 month period [1].
. If this is true—and we could imagine it is (at least
partially?)—then Elon's remark to journalists would make
sense:
"It's [..] irresponsible [..] to write an article
that [..] lead people to believe that autonomy is
less safe,” [..] “Because people might actually
turn it off, and then die" [1]
* * *
But to have an opinion about the autopilot's risk statistics I would also need to know:
a) What populations (data) they compare;
b) How each population is defined (inclusion and exclusion criteria);
c) What's the sample size (18 months, and?);
d) Who makes these calculations (to clearly identify possible conflicts of interests).
- Not sure if this type of data is publicly available?
* * *
Actually the National Highway Traffic Safety Administration (NHTSA) seems to indicate[1]:
1) that the data comes from Tesla—cf. point d) => conflict of interest;
2) Autopilot on/off was NOT used for the risk statistics—although it's central (point a);
3) instead the "40%" would measure the "number of airbag deployments per million miles" which is a proxy-metric that's not directly related to car accidents.
- Hey, this is odd (it's definitively not a Science or Nature method protocol).
* * *
. "The Insurance Institute for Highway Safety suggests:
A "13%" reduction in collision claim frequency, indicating
sedans with Autopilot enabled got into fewer crashes
that resulted in collision claims to insurers."
However it's a small difference and there're possible confounders like social status (a "Tesla driver"), gender, geographical area, and usually the confounders have a large influence on experiments, so it's unlikely that this (small) 13% difference would remain if we adjust for confounders...
* * *
. Tesla claims: A 40% crash rate reduction with the autopilot as compared to no autopilot, over an 18 month period [1].
. If this is true—and we could imagine it is (at least partially?)—then Elon's remark to journalists would make sense:
"It's [..] irresponsible [..] to write an article that [..] lead people to believe that autonomy is less safe,” [..] “Because people might actually turn it off, and then die" [1]
* * *
But to have an opinion about the autopilot's risk statistics I would also need to know: a) What populations (data) they compare; b) How each population is defined (inclusion and exclusion criteria); c) What's the sample size (18 months, and?); d) Who makes these calculations (to clearly identify possible conflicts of interests).
- Not sure if this type of data is publicly available?
* * *
Actually the National Highway Traffic Safety Administration (NHTSA) seems to indicate[1]: 1) that the data comes from Tesla—cf. point d) => conflict of interest; 2) Autopilot on/off was NOT used for the risk statistics—although it's central (point a); 3) instead the "40%" would measure the "number of airbag deployments per million miles" which is a proxy-metric that's not directly related to car accidents.
- Hey, this is odd (it's definitively not a Science or Nature method protocol).
* * *
. "The Insurance Institute for Highway Safety suggests: A "13%" reduction in collision claim frequency, indicating sedans with Autopilot enabled got into fewer crashes that resulted in collision claims to insurers."
However it's a small difference and there're possible confounders like social status (a "Tesla driver"), gender, geographical area, and usually the confounders have a large influence on experiments, so it's unlikely that this (small) 13% difference would remain if we adjust for confounders...
* * *
Here's the article AARIAN MARSHALL on Wired. [1] https://www.wired.com/story/tesla-autopilot-safety-statistic...