The GenAI Revolution: Balancing risks, regulations and responsible use

The result is that you may already be procuring and deploying GenAI as more and more applications integrate GenAI features - and we are rapidly approaching a point where it will be difficult, if not impossible, to avoid using GenAI.

In the November 2023 edition of Telescope, we wrote about the key contractual issues when procuring or supplying both “traditional” AI solutions and GenAI, as well as discussing early indications of regulatory risk. In this edition, we consider three further issues:

Why does deployment of GenAI carry with it risk?

With any publicly available GenAI, you will have spotted the disclaimer below every output that the GenAI “may display inaccurate information” and “can make mistakes,” so “double-check the output” and “check important information”, or words to that effect. This is all true, and it is good advice. This will apply similarly to applications that utilize and combine their own corporate data with an LLM in a private instance. This means that careful consideration needs to be given to how you will regulate, monitor and control their use. It also means that careful consideration needs to be given to the allocation of risk, remediation, and remedies in agreements between vendor and customer.

The wider regulatory picture must also be taken into account: we discussed in our November 2023 edition of Telescope the emerging regulatory picture in the European Union and UK, a picture which is continuing to develop as the EU AI Act beds-in and as the new UK Government considers whether to depart from the more indirect, hands-off approach adopted by the previous Government. The recently delivered first King’s Speech of the new Labour Government included a statement that the UK would “seek to establish the appropriate legislation to place requirements on those working to develop the most powerful artificial intelligence models,” reflecting a pledge in the Labour Party’s 2024 election manifesto to “ensure the safe development and use of AI models by introducing binding regulation on the handful of companies developing the most powerful AI models” – but to date the UK Government has not explained what concrete steps it intends to take, nor has any draft legislation been published. While it is likely that the UK will adopt a new regulatory approach in the future, for now, your use, development or supply of AI-enabled products may fall within the scope of a wider range of national and international regulations outside of AI-specific law.

Taking one example, incorporating connected services into a consumer product (including AI features) and then manufacturing, importing or distributing that product in the UK or EU is likely to bring the product into the scope of the UK’s recently-enacted Product Security and Telecommunications Infrastructure Act 2022, which is similar to the EU’s proposed Cyber Resilience Act (which is expected to come into force later this year). These statutes impose specific obligations in relation to the security features of relevant products, with sanctions for noncompliance. The potential impact of the proposed EU AI Liability Directive, intended to provide consumers adversely affected by AI systems with a fault-based right to redress, is also one to pay close attention to – progress of the Directive has been stalled but recent indications are that the EU will continue working on these rules into 2025.

Why does this matter?

Take the example of a self-serve HR advice platform deployed in an organization calling on that organization’s HR policies and procedures using prompts by HR professionals and a private LLM – errors here could lead to incorrect procedures and decisions being made and expose the company to claims from employees. Or the example of the development and deployment of a GenAI system by an IT-managed services provider to certain corporate customers – to what extent does it meet good industry practice, fitness for purpose, satisfactory quality or similar contractual obligations? Similarly, if a company director uses GenAI to assist them with making decisions around business strategy or hiring decisions, will that be consistent with their statutory and fiduciary duties as a director?

So, what can you do about it?

There is no one-size-fits-all answer to the questions raised by GenAI. Responsible use will depend on any number of factors, from the GenAI tool being used, to the application it is being put to, to the nature of the business and person using it. But there are common themes.

Before you use GenAI yourself, take a step back and ask first: do I know what GenAI tools I am actually using and the purposes for which they are being put? After that, ask: do I believe this a responsible way to use the technology, and can I justify and defend that use? This should include consideration of what the consequences would be if the output is wrong, what reliance you intend to place on the output, and what steps you will take to identify inaccurate output and mitigate that risk. It should also include consideration of what data or information is likely to be used as input into the system – and whether or not it may include personal, confidential or otherwise sensitive data.

If your employees have access to tools, which include GenAI systems, you should put in place policies and procedures to manage and control their use and define what your organization considers to be a responsible use of the tools. As noted above, your organization’s answer to this question is unlikely to be the same as another organizations and real thought and consideration should be given to this process.

Any policy or process you put in place should then be tested against how GenAI systems are actually being used by employees in practice: a policy which says that employees should not use GenAI for a particular purpose is of limited benefit and utility if, in practice, employees are regularly using GenAI for that purpose. Your organization is better served ensuring that appropriate safeguards and regulation are being applied to what is actually happening than adopting policies that assume a different state of affairs.

Discover how your business can navigate the AI legal landscape and innovate responsibly. Read our latest AI and ESG Telescope edition here:

Read Telescope