united-states regulations
What standard of testing are self-driving cars&#39; algorithms legally held to?
<p>The reasoning behind the question...</p>

<p>One common argument for autonomous vehicles is that they are far, far safer in number of accidents per unit time per average person than having the person pilot the vehicle. Which, at first glance, feels like a good argument; if you can demonstrate that a given algorithm, under a variety of different conditions over a period of months, has fewer accidents than a normal human then it makes sense to use that algorithm instead.</p>

<p>However, the thing that gets completely glossed-over in that argument is the fact that there is a single point of failure in that entire system: effectively, all the cars are using the same driver. So, if the software is fine for a few months, everything's fine, but if the image recognition software performs very, very poorly on an overcast day and one day it's overcast, or if a software update is pushed out which contains a regression, suddenly millions of people all die in one day.</p>

<p>In other words, an AV won't necessarily improve statistics; it will simply change their distribution, because there's effectively one driver for all those cars. So, when that one driver has an off day, all the passengers have an off day.</p>

<p>This is effectively the same model the pharmaceutical industry has been dealing with for its entire history, and as a result some pretty solid testing guidelines exist to minimize risk; first the new drug / algorithm is tested for a while with a small group of volunteers in a double-blind fashion (such that everyone is told they're getting the new thing, when half of the group are not), then larger and larger groups are tested under more varied conditions to see whether uncommon but serious flaws emerge, and to determine whether the placebo is actually more helpful / less dangerous than the new treatment.</p>

<p>Is there similar legislation in place to regulate these AVs' navigational software?</p>

<p>The federal guidelines (from NHTSA) are <a href="https://www.transportation.gov/sites/dot.gov/files/docs/AV%20policy%20guidance%20PDF.pdf" rel="nofollow">here</a>: these are not currently hard regulations (but they aren't just talking points). Manufacturers must certify that their product complies with safety standards, which is true of brakes as well as self-driving systems. Federal vehicle safety regulations are generally framed in terms of a functional requirement, like "doesn't, by design, crash".</p>

<p>They list (p. 28) a set of specific things that an HAV should be able to do, such as "merge onto the freeway", "deal with a roundabout", taken from California, with a suggestion that this is a minimum set, to be expanded. The HAV guidelines indicate that the future regulations will have a very finely tuned set of designations ranging from zero to full vehicle intervention in the driving process, and requirements for recording and reporting all sorts of events of concern. The latter is important in establishing that some system (for whatever reason) is in fact unsafe. Regulation starts by articulating a desired end and a means of determining whether desires have been met, and later when there turns out to be a really big problem, Congress may decide to limit what manufacturers can do, e.g. they could pass a law forbidding the use of Windows ME as the OS underlying an HAV, which would authorize a specific regulation from NHTSA. </p>

<p>At common law, there is strict liability for a defective product, which can be either a defect in the manufacturing of a particular product, or a defect in the design of the product which can be demonstrated by showing that a better design could have prevented the accident. See generally, <a href="https://ius.unibas.ch/fileadmin/user_upload/fe/file/Vorlesung_vom_13.1.__Restatement_of_the_law__thirds__tords_Product_Liability.pdf" rel="nofollow noreferrer">Restatement of Torts (Third) Product Liability</a>, Sections 1-21. See aso Restatement of Torts (Second) Section 402A (discussed in <a href="http://digitalcommons.law.villanova.edu/cgi/viewcontent.cgi?article=2249&amp;context=vlr" rel="nofollow noreferrer">this 1979 article</a>).</p>

<p>A self-driving car that causes in an accident would be subject to this rule and would be considered defective if the accident was caused by some flaw in the design that could have prevented the accident. (Self-driving cars might also be considered "ultra-hazardous" which would also give rise to strict liability.)</p>

<p>Of course, causation and damages would still have to be established. For example, a plaintiff would have to show that the harm was caused by the accident and more particularly by the alleged defect in the self-driving car, and the plaintiff would also have to prove the amount of their economic and non-economic damages arising from the accident, using medical bills, testimony from medical experts, estimate or actual amounts paid due to property damage, testimony showing lost wages, and pain and suffering testimony.</p>

<p>No legislation is required to implement this regime, and it is generally not subject to waiver except by self-driving car user/owners, since third parties injured in accidents would have no contractual relationship with the company that made the car.</p>

<p>There are no laws regarding this at the moment but there are two ways that the situation you described will be avoided.</p>

<ol>
<li>There will be a Government body that has to test every update in a range of conditions and they have to sign off on every update</li>
<li>The auto car industry will take influence from the aeronautics and banking systems. in aeronautics, if someone writes 5 lines of code per day that get accepted, it is seen as a good day. In banking, if your updates get put to live within 6 months, it was some very good code. Within both these industries, the program doing exactly what is intended is key, they have built layers of checks and tests so that faulty code doesn't get to a live environment. If bad code manages to get to a live environment then extra things are checking on it and if things happen that are unexpected then it fails safe (opposed to failing deadly) with banking this is stopping everything, in aeronautics it usually means using backup systems.</li>
</ol>

<p>what will most likely happen is what I described in 2 and if that fails, governments will either mandate that 2 is followed (if it fails due to someone not following it) or what I described in 1.</p>
