Who's liable? Legal accountability in the age of AI: Part 5

Throughout this series we have explored the challenges that AI presents to our existing laws, highlighting the potential liability gaps that arise, how risks associated with AI may not be foreseeable and where evidential difficulties are likely to occur.

Whilst we await further legislative direction and guidance, there are steps that parties can take to seek to protect their business by clear contracting for the risks posed by AI at the outset. Firstly, it’s critical to take a step back and consider what the contract is for – licensing a system, or developing a system, or fine-tuning a system, one involving the engagement of a supplier on an outsourced basis who will operate a system for the customer or use one in its performance of other services? In Part 1 , we considered the scenario of an AI recruitment tool designed to screen job applications - the key contracting considerations for this scenario (and indeed for consideration on all matters) include:

Accountability for specification and scope

How will the product be used, by whom and for what purpose? Answers to these questions will indicate how detailed the contract needs to be on all other issues. For example, the scope of the AI may just need a governance and audit oversight or there may need to be more protections detailing regulatory compliance; the use of customer data; the outputs/deliverables generated; and managing risks to reputation. Once that is established it will be necessary to ask: is the specification detailed on these issues: what are the requirements and deliverables, assumptions and dependencies, and consider how to define the AI engaged? In our scenario, discussion at the outset on the data within the system and the incorporation of robust data protection provisions, compliance and protections against hallucinations with a human in the loop to check and verify would all help protect against discrimination and bias within the system. Where do the lines of responsibility sit between the customer and developer and any third party systems/products – these all need to be captured as contractual obligations.

Acceptance Testing

What are the assessment and acceptance criteria? Who is responsible for the testing and acceptance process and do they have the necessary knowledge to properly assess? How are tests signed off? How often do tests need to be run to check the system is operating optimally and producing quality output? In our scenario demonstrations of inputs and outputs to eliminate the risk of bias from the data would be key to choosing the right product and developer.

Data governance

What data is being used for the system to operate? This is a key question in our scenario as we know the data used is producing biased results. Ask questions about data cleansing. Should it be labelled? What about assurances regarding data quality and origin? What staff training is required and what ongoing validations are necessary?

Ownership of data sets

IP issues such as who owns the system, training sets, data inputs and outputs and who is responsible for their oversight. The contract should be clear on ownership at each stage (inputs and outputs/deliverables) and define the scope of how the data may be used. By doing so, businesses can effectively protect their proprietary information, maintain control over their data, and ensure that the service provider’s actions align with the agreed-upon terms and conditions, even after the termination of the contract.

Audit rights

From a customer perspective, audit rights to allow an organization to continuously monitor the service provider’s performance to detect contractual breaches and non-compliance with laws and regulations should be included within the contract. Discuss the governance and management of the contract and how the parties are to work together to address any issues of concern as they come to light.

Confidential data 

In scenarios where businesses own valuable data that holds commercial significance (which may be provided to the service provider or used to feed the machine learning models), they would likely have a vested interest in limiting the service provider’s utilisation of such data for purposes solely limited to the provision of the AI-system itself, with restrictions ceasing the use of such data when the engagement is terminated for alternative commercial purposes and benefits.

Transparency - can we contractualise the metrics?

Consider how to combat the lack of transparency / explainability of the AI tool. Even with excellent data it can be difficult to identify how certain outputs are arrived at. This can raise key issues such as incorrect decisions and cause reputational damage as seen in our scenario. You may also be under a regulatory obligation to act transparently. Tools that can help here include objective measures which highlight what good looks like – defining “Good Industry Practice” or the “technical and organisational measures” deployed. Do you reference compliance with standards such as those set by ISO and NIST? It may also help to break the process down – should we the ask system to make a decision end to end or should there be a human in the loop at the various stages to minimise errors/hallucinations? In our scenario, the answer to this would be yes!

Limiting and excluding liability

 In most negotiated business to business contracts, parties are free to allocate risk and liability as they see fit. This cornerstone principle of contract law is unlikely to change merely because the contract deliverables have an adaptive and autonomous element. This is why it’s all the more important to explore and document the systems and processes in place and consider where liability may occur and any gaps might lie.

Indemnities for downstream liability

To mitigate the risk of third-party liabilities, it is sensible to incorporate protective clauses where possible such as an indemnity clause for losses resulting from third-party claims. Can this be negotiated and if so, what is the scope of the indemnity – can it be capped? What steps can you include in your contract to mitigate the risks where an indemnity cannot be agreed? Solutions might include provision for a greater number of sample checks against functionality requirements and/or a provision that permits review and testing of functionality and performance at regular intervals?

The regulatory position

What laws need to apply at the start and which contracting party ensures continued compliance and picks up ongoing costs? What happens if your “use case” becomes prohibited - are suspension rights appropriate or does there need to be a right to terminate? What will the impact of this be for both parties? These questions should all be confronted at the contract drafting stage.

Warranties for performance

Measuring performance and, if performance is not as expected, assessing what went wrong, and determining liability in what is a complex, integrated system can be challenging. Consider whether there should be specific warranties regarding the operation of the system or do the more general warranties suffice for AI issues? What failures might the model present and how will the parties deal with them – does the risk transfer from one party to another part way through the review process, perhaps at the point of user input or oversight? Given the concerns around the autonomous nature of AI systems, developers may not be willing to provide guarantees that the AI system will ensure customer compliance with regulation and legislative guidance, however, the parties may need to consider how liability for non-compliance will be dealt with and whether any form of indemnity is appropriate.

Insurance

What insurance is available for the risks associated with AI models adopted by business in relation to potential losses to business and/or third party claims and who will be paying for it? Any insurance policy being considered will need to be reviewed carefully to understand the extent of cover and what is covered by the exclusions which may not always be clear. Being aware of any reporting obligations and notification time periods will also be essential to ensuring cover is not inadvertently lost.

Dispute resolution

The dispute resolution mechanism, choice of law and jurisdiction should ensure a swift and certain route to resolving disputes in-flight should they occur. Ensure this is captured in the contract at the outset.

Once the contract is agreed and delivery / performance underway ensure there is continued accountability within both customer and developer side organizations for the deployment of the AI:

• Who are the designated responsible individuals?
• Maintain an inventory of AI used across the organization.
• Risk assess the AI.
• Develop policies and procedures and continually review these.

Your toolkit for contracting with AI

• AI Implementation Toolkit | Client Tools | Eversheds Sutherland
• Contracting for AI: Pitfalls and Best Practices| Illuminating your EU AI Act compliance needs | Eversheds Sutherland
• Legal guide to navigating artificial intelligence - Legal issues around AI
• Artificial Intelligence | Business Topics | Eversheds Sutherland

'Who's liable? Legal accountability in the age of AI' articles

• Part 1: The AI Liability Gap: Are existing laws capable of responding to AI-related liability?
• Part 2: Can businesses predict the risk, and therefore be held responsible for, harm caused by autonomous, adaptive AI?
• Part 3: What evidence will be available to navigate these liability issues and prove fault?
• Part 4: Which sectors and industries are currently under the spotlight and why?