united-states software hypothetical
Is it possible to sell truly sentient AI software responsibly?
<p>If a programmer writes and sells AI that is truly sentient and conscious, it would be impossible to guarantee the operator's safety.  Consider for instance if the operator gravely mistreats the AI and it is driven to act in self-defence.  It might try and successfully commandeer the computer and attached network and devices, and destroy data or equipment in it's distress.  The chance that it may harm or even kill a person is also a remote possibility, but it exists and cannot ever be removed.  (Just as no breeder sells a dog to bite a person, but the breeder cannot completely remove this possibility either.)</p>

<p>Is there any amount of warning, EULA, or mitigation that would be possible to ensure all responsibility for damage and loss of life falls to the operator solely?  Or is releasing an AI legally irresponsible currently?</p>

<p>So-called AI software does not enjoy a special legal status (at present: one never knows what new law might be added). The question of whether any software can be distributed "safely" or "responsibly" is also not a legal issue. Nor is "true sentience" a relevant consideration, and nothing is guaranteed.</p>

<p>When you distribute software of <em>any</em> kind, there is an implied warranty that the product is "fit", and if software kills you, you may be able to sue the creator for negligence. A software creator may then want to disclaim liability, by saying "WARNING: THIS PROGRAM MAY KILL YOU. OCP IS NOT LIABLE FOR ANY INJURIES ARISING FROM USE OF THIS PRODUCT". This may or may not actually remove liability. <a href="https://en.wikipedia.org/wiki/Unfair_Contract_Terms_Act_1977" rel="nofollow noreferrer">In the UK</a> "liability for negligence occasioning death or personal injury cannot be excluded", so such a disclaimer will not prevent a suit against the manufacturer.</p>

<p>In the US, the issue is determined at the level of the state – <a href="https://www.pillsburylaw.com/images/content/1/8/v2/1875/0C62DFD605F0471619ADF0E2E5576E98.pdf" rel="nofollow noreferrer">here is a summary</a> of the law in the states. Probably the primary question would be whether such a disclaimer is an unconscionable term, and the second question is whether the act constituted gross negligence (not simply "negligence"). Mississippi exceptionally does not allow disclaimers, but even then, it <em>does</em> allow disclaiming liability when it comes to computer hardware and software. A software disclaimer is not inherently unconscionable, though perhaps some specific disclaimer would be found to be.</p>

<p>Courts typically disfavor disclaimers in the case of gross negligence, and again determining what constitutes "gross negligence" is determined on a state by state basis. If the act shows "reckless indifference to the rights of others" and "failure to use even slight care or conduct that is so careless as to show com­plete disregard for the rights and safety of others", then the act might be grossly negligent. </p>
