Who's liable? Legal accountability in the age of AI: Part 3

In this series, we have explored some of the ways the autonomous and adaptive nature of AI systems looks poised to challenge our traditional legal frameworks around liability, causation, and remoteness of damage.

Each of these elements will hinge on the evidence available to establish relevant facts and satisfy applicable legal thresholds. AI’s autonomous and adaptive nature poses some interesting questions here too, particularly in relation to transparency, explainability, and accountability.

The ‘black box’ nature of AI

Even with excellent data, AI tools can often lack transparency, explainability, with their internal operations being opaque to both the user and developer. Unlike human decision-making, which can be understood in terms of reasoning and intent, AI outputs are essentially generated through complex mathematical and statistical algorithms which are informed by how the model has been trained. This opacity of AI decision-making is commonly referred to as the ‘black box’ problem.

When collating and presenting evidence in court, litigants will need to ask themselves and think about:

• Whether the available evidence can be presented in a comprehensible manner that allows analysis? AI systems based on complex statistical models or deep learning often reach conclusions through internal reasoning that cannot easily be traced or explained in human terms to a human decision maker. This contrasts starkly with the legal system’s expectation that evidence should be capable of logical explanation and open to critical examination. If the reasoning behind an AI output cannot be articulated with any degree of confidence, it becomes problematic to determine whether any flaw lies in the design of the system, the data used to train it, or the human decisions surrounding its deployment.

• Can the original data evidence be preserved and has it been altered? AI tools can alter or infer data in ways not transparent to human reviewers. Systems that enhance, summarise, or infer information from raw data effectively create a new data product: is the AI-generated output an accurate reflection of the original material, or has it introduced new assumptions or artefacts? Maintaining and identifying a clear record of how data has been transformed and by what means may not be straightforward.

• Can expert evidence assist? Evidence that cannot be explained, replicated, or independently verified may be viewed as unreliable. When an “expert” gives their opinion, the court will assess their credentials, as well as the methods and reasoning underpinning their conclusions. When an “expert” has to form a view on whether or not AI has ‘gone wrong’ and why it has done so, such validation is inherently more difficult. The performance of an AI model will vary depending on the data it receives, the conditions in which it operates, and the assumptions built into its design. Given that one of the world’s leading AI developers has admitted that “a surprising fact about modern large language models is that nobody really knows how they work internally” begs the question, who is the “expert” and will they have the necessary experience to offer a fair assessment of the AI tool which will be useful to a judge or arbitrator?

• What confidentiality concerns may impact disclosure? Algorithms may be confidential or amount to trade secrets raising questions over whether the detail of the algorithms can be disclosed. The underlying data, algorithms, and decision parameters may need to be disclosed so that the parties and the court can understand where liability lies, potentially with confidentiality agreements and undertakings to allow review.

All of these issues can create potential barriers to justice for prospective claimants. For example, proving breach or causation may require costly expert evidence just to establish the basic chain of events, and even then, precise explanations may remain elusive.

The AI scenario

Applying some of these evidential challenges to our scenario from Part 1, the typical evidence required would include contractual documentation to assess how the parties pre-allocated risk and liability, evidence of the written representations made by the developer (such as in sales literature and email communications) and, depending on the nature of the claim, witness evidence, for example to set out any oral representations made which the claimant relied upon.

The layer of complexity that AI brings to the scenario also poses new questions:

• Is there a record of the precise dataset used by the AI tool and any accessible evidence of how that compares with any current adapted data?
• What evidence, if any, is available to explain how the AI adapts, the decisions it makes and why?
• Does the evidence show whether the bias was in the training data, errors in calibration or as a result of the historical dataset adopted by the user?
• Would there be any meaningful record or explanation sitting behind the AI tool regarding its autonomous assurance that the outcomes were compliant with any relevant legislation?
• Who might be an appropriate expert to examine and analyse the operation of the AI tool?

On that latter question, even if one can identify an appropriate expert, one also needs to consider what appropriate questions can be asked. Would the expert be able to give an authoritative opinion on whether it was ‘normal’ for the AI tool to adapt the way it did, whether that adaptive behaviour was a result of incorrect design or programming in the first instance and, if so, what should the reasonable developer have done differently? Would they be able to provide answers as to whether the bias results were a foreseeable outcome?

It is easy to see how difficult these questions will be to answer in a vast and rapidly developing area, with little or no comparative data to reference.

Potential solutions

The EU Product Liability Directive 2024/2853 (PLD) updated the 1985 Product Liability Directive (85/374/EEC) to reflect the pace of technological development in products, including those that use AI. The PLD recognises the evidential challenges of complex technologies like AI. EU lawmakers have decided that, in the sphere of product liability more generally, traditional notions of burden of proof (i.e. that the claimant must prove the defect and causal link between defect and damage) may sometimes create a barrier to justice.

Therefore, the imposes an obligation on defendants to disclose relevant evidence in proceedings where a claimant has presented evidence and facts sufficient for a plausible claim which will extend disclosure obligations on manufacturers in some jurisdictions and, in certain circumstances, creates a rebuttable presumption that the product is defective, effectively shifting the burden of proof to the manufacturer. The PLD’s shift in burden materially heightens the risk for manufacturers throughout the supply chain and may be a sign of things to come in other jurisdictions for businesses that design or deploy AI systems.

The EU has also looked at other legislative tools in this space, including:

• Artificial Intelligence Act ((EU) 2024/1689) (AI Act). While this does not itself create new individual rights for those harmed by AI systems it does seek to impose obligations and requirements for providers, deployers, distributors, importers, authorised representatives and product manufacturers to prevent and manage the risks posed by AI systems from the outset.

• Proposals for an AI Liability Directive (AILD). While ultimately withdrawn, this was an interesting attempt to adapt non-contractual civil liability rules to AI to alleviate the burden of proof in claims under national fault-based liability regimes where AI systems caused damage.

Another route might be the adoption of transparency and validation standards specific to AI tools which could include mandatory disclosure around testing, methodology and documentation of performance limitations. However, the breadth and complexity of AI models may mean that creating a universal standard when we are still learning about their capabilities is more aspirational than achievable at this stage in the AI journey.

Conclusion

The challenges presented by AI do not necessarily require a wholly new evidential regime, but they do demand adaptation. The law of England and Wales has traditionally evolved through the steady refinement of existing principles rather than immediate and far-reaching reform, but the speed of development of AI and its wide impact on business and society may mean that a more novel approach is needed. Otherwise, while the same incremental approach may ultimately prove effective, with regulators, courts, practitioners, and industry experts working collaboratively to find solutions to the implications of AI-related evidence, this will need to develop at a significantly faster pace than is normally the case.

'Who's liable? Legal accountability in the age of AI' articles

• Part 1: The AI Liability Gap: Are existing laws capable of responding to AI-related liability?
• Part 2: Can businesses predict the risk, and therefore be held responsible for, harm caused by autonomous, adaptive AI?
• Part 4: Which sectors and industries are currently under the spotlight and why?
• Part 5: Contracting considerations – what is the best approach to take?