AI Liability: When the Model Gets It Wrong, Who Pays

A chatbot tells a customer they're entitled to a refund that doesn't exist. An AI-assisted medical system misses a diagnosis. A hiring algorithm systematically disadvantages candidates from certain backgrounds. An autonomous vehicle makes a decision that causes a collision. A language model generates defamatory statements about a real person.

In each of these scenarios, someone has been harmed. The question that existing legal frameworks struggle to answer: who is liable?

This isn't a hypothetical problem. These scenarios have all happened. And the legal system's response has been, charitably, a work in progress.

The Current Liability Gap

Traditional product liability law was designed for a world where products behave predictably. A manufacturer makes a toaster. The toaster is defective. The toaster causes a fire. The chain of causation is clear, and the manufacturer is liable.

AI systems break this model in several ways.

The Predictability Problem

A defective toaster behaves the same way every time. An AI system can produce different outputs from the same input depending on context, conversation history, random sampling, and model state. The failure mode isn't a consistent defect; it's a probabilistic behavior that manifests unpredictably.

This makes traditional defect analysis difficult. Was the system "defective" if it works correctly 99.7% of the time but fails in the specific instance that caused harm? Products liability law has frameworks for this (manufacturing defects vs. design defects vs. warning defects), but applying them to systems whose behavior is inherently probabilistic and emergent is genuinely new legal territory.

The Multi-Party Problem

When an AI system causes harm, multiple parties contributed to the outcome. The model developer created the base model. A fine-tuning company may have customized it. A platform integrated it into an application. The deploying company made decisions about how to use it, what guardrails to implement, and what disclosures to make to users. The user may have used it in ways that weren't intended.

Traditional liability often looks for a single responsible party. In the AI value chain, responsibility is distributed across multiple actors, each of whom made decisions that contributed to the outcome but none of whom individually "caused" the harm in the way that liability frameworks typically require.

The Product vs. Service Distinction

Product liability law applies to products. Service liability operates under different (generally less strict) frameworks. AI systems blur this distinction in ways that matter legally.

Is a language model a product? Is the API access to that model a service? Is the application built on top of the API a product? The answers determine which liability framework applies, and different frameworks have different burdens of proof, different damage calculations, and different defenses available.

Many AI companies structure their offerings as services precisely because service liability is generally less strict than product liability. Terms of service typically disclaim warranties, limit liability, and require arbitration. Whether these contractual limitations survive contact with actual harm to real people is an open legal question.

The Explanation Problem

Liability often requires establishing causation: what specifically went wrong, and why? With many AI systems, particularly large neural networks, the internal decision-making process is opaque. You can see the input and the output, but the reasoning in between is distributed across billions of parameters in ways that resist human-interpretable explanation.

This opacity makes it difficult for both plaintiffs and defendants. A plaintiff claiming harm from an AI system may struggle to explain the specific defect that caused the harm. A defendant may struggle to demonstrate that its system functioned as intended. Both sides face evidentiary challenges that existing legal frameworks weren't designed to handle.

Real Cases Shaping the Law

The legal landscape is being shaped by cases that are establishing precedents, even if comprehensive frameworks don't yet exist.

The Air Canada Chatbot Case

In one of the more widely discussed cases, Air Canada's customer service chatbot told a customer they could book a full-fare flight and then apply for a bereavement discount retroactively. This was incorrect. Air Canada's actual policy required the bereavement fare to be requested at the time of booking.

When the customer sought the discount they'd been promised, Air Canada argued that the chatbot was a "separate legal entity" responsible for its own accuracy. The tribunal rejected this argument, finding that Air Canada was responsible for information provided by its chatbot and was bound by the representation the chatbot made.

This case established an important principle: companies are responsible for the outputs of AI systems they deploy, even when those outputs are incorrect. You don't get to distance yourself from your own chatbot.

Hallucination Liability

Language models hallucinate. They generate plausible-sounding but false statements with confidence. When those hallucinated statements are about real people, real companies, or real events, they can cause real harm.

There have been documented cases of language models generating false claims that specific individuals committed crimes, falsely associating people with organizations they have no connection to, and fabricating legal citations that were then submitted to courts.

The defamation implications are significant. Traditional defamation law requires a false statement of fact, publication, fault, and damages. When an AI system generates a defamatory statement, the "publication" element is met when the output is displayed to a user. The "fault" question is where it gets complicated: did the company know or should it have known that its system would generate false statements about real people?

Given that hallucination is a known, well-documented behavior of current language models, the argument that companies "should have known" is increasingly strong. Deploying a system known to fabricate statements about real people, without adequate warnings or safeguards, creates foreseeable liability.

Algorithmic Discrimination

AI systems used in hiring, lending, insurance, and housing have faced legal challenges under existing anti-discrimination laws. The Equal Employment Opportunity Commission has made clear that employers using AI in hiring decisions are responsible for discriminatory outcomes, regardless of whether the discrimination was intentional.

Several state laws now explicitly address algorithmic discrimination, requiring impact assessments and creating causes of action for individuals harmed by discriminatory AI decisions.

The important legal principle here is that "the algorithm did it" is not a defense. If you use an AI system to make decisions that affect people, you're responsible for those decisions meeting the same legal standards that would apply if a human made them.

The Product Liability Framework Debate

Legal scholars and policymakers are debating how to adapt product liability frameworks to AI systems. Several approaches are being considered.

Strict Liability

Under strict liability, a manufacturer is liable for harm caused by a defective product regardless of whether the manufacturer was negligent. Applying strict liability to AI would mean that if an AI system causes harm, the developer or deployer is liable even if they took reasonable care.

Proponents argue that strict liability correctly places the cost of harm on the parties best positioned to prevent it and to spread the cost through pricing and insurance. It also incentivizes investment in safety because the cost of failures is internalized.

Opponents argue that strict liability is inappropriate for AI because probabilistic systems will always have some failure rate, and holding developers strictly liable for every failure would chill development. They argue for a negligence standard that asks whether the developer took reasonable precautions.

The EU's AI Liability Directive, proposed in 2022 and still working through the legislative process, takes a middle approach. It creates a presumption of causality for AI systems that don't comply with the AI Act's requirements, effectively creating something close to strict liability for non-compliant systems while maintaining a negligence-like framework for compliant ones.

The Negligence Standard

Under a negligence framework, the question is whether the developer or deployer exercised reasonable care. This raises its own questions. What constitutes "reasonable care" for an AI system? Is it reasonable to deploy a system that hallucinates 3% of the time? 1%? 0.1%? The answer probably depends on the application, but there are no established standards defining what's reasonable in specific contexts.

Industry standards and best practices, like those being developed by NIST and ISO, may eventually provide the benchmarks against which "reasonable care" is measured. But these standards are still being developed, which means the legal standard is currently vague enough to create uncertainty for both developers and potential plaintiffs.

Shared Liability Models

Given the multi-party nature of AI value chains, some proposals envision shared liability frameworks where responsibility is allocated across the parties involved. The model developer might bear liability for fundamental model defects. The deployer might bear liability for integration decisions, guardrail implementations, and user-facing disclosures. The user might bear some responsibility for misuse that was clearly outside intended use cases.

This approach has intuitive appeal but is complex to implement. Determining how to allocate responsibility across parties requires understanding each party's contribution to the harm, which requires the kind of technical analysis that courts are still developing the capacity to perform.

The Insurance Response

The insurance industry is beginning to grapple with AI liability, and its response tells you something about how the risk is being assessed.

Traditional general liability and errors-and-omissions policies often don't clearly cover AI-related claims. Some insurers have begun offering AI-specific coverage, but the market is immature and premiums reflect the uncertainty in the underlying risk.

The challenge for insurers is the same as the challenge for the legal system: the risk is difficult to model. AI systems don't have the kind of actuarial history that allows insurers to price risk confidently. The potential for correlated failures, where a widely deployed model has a systematic flaw that affects many users simultaneously, creates the kind of tail risk that insurers are uncomfortable with.

Some predictions about where the insurance market is heading. Cyber insurance policies will increasingly include AI-related coverage or exclusions (probably exclusions first, then coverage as the risk becomes better understood). AI-specific liability policies will become more common as the case law develops and risk can be priced more accurately. Companies deploying AI in high-risk applications will face increasing pressure from insurers to demonstrate specific safety practices, similar to how cyber insurers now require specific security practices.

The insurance market's response will also shape behavior. If AI liability insurance becomes expensive or conditional on specific safety practices, the cost of insurance becomes a de facto regulatory mechanism, incentivizing safety investment regardless of what the law requires.

Where Frameworks Are Heading

Based on current legislative activity, case law development, and regulatory signals, several trends are emerging.

Toward Deployer Responsibility

The trend across jurisdictions is toward holding deployers, the companies that put AI systems in front of users, primarily responsible for harm. This makes practical sense: deployers choose how to use the system, what guardrails to implement, what disclosures to make, and what decisions to delegate to the AI.

This doesn't let developers off the hook entirely. Developers may face liability for fundamental defects or for failing to provide adequate documentation about limitations. But the primary liability burden is shifting toward the point of deployment.

Toward Disclosure Requirements

Companies deploying AI systems will increasingly be required to disclose that AI is being used, what its known limitations are, and what recourse is available when things go wrong. Failure to disclose becomes its own source of liability, separate from the underlying AI failure.

This is already happening in specific sectors. Financial regulators require disclosure of AI use in lending decisions. Healthcare regulators are developing requirements for AI-assisted diagnosis disclosure. Employment laws in several states require disclosure of AI use in hiring.

Toward Mandatory Risk Assessment

The EU AI Act's risk assessment requirements, and similar requirements emerging in other jurisdictions, create a framework where the adequacy of a company's pre-deployment risk assessment becomes legally relevant. If you assessed the risk and deployed anyway, you need to show that your assessment was thorough and your mitigations were reasonable. If you didn't assess the risk at all, you've essentially conceded negligence.

Toward Human Oversight Requirements

For high-stakes decisions, the trend is toward requiring meaningful human oversight of AI systems. This means not just having a human "in the loop" nominally, but ensuring that the human has the information, authority, and time to actually override AI decisions when appropriate.

The legal significance is that "the AI decided" becomes a weaker defense when regulation requires human oversight. If the human was supposed to be overseeing the AI and didn't catch the error, the question becomes whether the oversight mechanism was adequate.

What This Means for Builders

If you're building or deploying AI systems, the liability landscape has practical implications for how you work.

Document your risk assessment. If your system causes harm, the first question will be what risks you identified and what you did about them. A documented, thorough risk assessment isn't just good practice; it's your primary defense.

Implement meaningful guardrails. Not token guardrails that check a compliance box, but guardrails that actually prevent or mitigate the harms your risk assessment identified.

Disclose AI use clearly. Users should know when they're interacting with an AI system, what its known limitations are, and what they should not rely on it for. Clear disclosure reduces both the likelihood of harm and the liability exposure when harm occurs.

Maintain human oversight for high-stakes decisions. If your AI system influences decisions about people's health, employment, finances, or legal rights, ensure that a qualified human can review and override those decisions.

Track the legal landscape. AI liability law is developing rapidly. What's legally uncertain today may be clearly resolved in twelve months. If AI liability is relevant to your work, staying current with legal developments isn't optional.

The liability gap in AI is real, but it's closing. The direction is clearly toward more accountability, more disclosure, and more structured frameworks for determining who's responsible when AI systems cause harm. Companies that build accountability into their systems now, rather than waiting for the law to force it, will be better positioned for the legal environment that's coming.