ByteSize: When AI holds the hammer - Chat-GPT as the judge's counsel

In this briefing, we discuss why the verdict of the District Dourt of 7 June 2024 raised questions in the Dutch legal community and how Dutch courts can use new technology such as Chat-GPT.

The legal framework

To understand why the Chat-GPT case became widely-discussed in the Dutch legal market, some background on the Dutch procedural legal framework is useful. In the Netherlands, the parties to a dispute provide the court with the facts and the grounds on which they ask the court for a decision. As a starting point, courts are by the parties. Although Dutch judges are allowed to do their own research on the facts, in a case that became known as the ‘Googling Judge Case’, the Dutch Supreme Court made clear that parties cannot be caught off-guard by a decision and the reasoning behind it. Thus, if a court decides to use Google or other online resources, it is obliged to present what it has discovered with the parties. If a court uses the outcome of its own research in a verdict without consulting parties, it violates the fundamental right to a fair hearing.

The court’s use of Chat-GPT

The Chat-GPT case revolved around a conflict between neighbours. The defendant used the roof of his neighbour while performing construction work. The claiming neighbour argued that in doing so, the solar panels were damaged, resulting in a loss of output and thus financial damage. The calculation of the solar panel yield loss provided by the claiming party did not convince the court. In the verdict, the District Court concluded that the claimant’s calculations were ''difficult for the court to follow''. So far, so good. Instead of rejecting the claim or asking the parties for additional evidence, the District Court decided to proactively seek help from Chat-GPT. Using Chat-GPT, the District Court estimated (i) the average lifetime of the claimants solar panels and (ii) the current solar power price. On this basis, it concluded that there were sufficient reasons to deviate from the damage calculation provided by the claiming party.

Although being applauded for daring to use new technology, the District Court received criticism from the legal community on two grounds.

• ChatGPT provides cohesive and well-structured answers to the questions asked by the court. This does not mean the answers it provided were correct. As a language model, ChatGPT analyses the text of the question to understand its context and intent, then draws upon the information from its training material to generate a relevant and coherent response. Chat-GPT is not an estimation or calculation tool. Furthermore, Chat-GPT does not have real-time access to the internet. Assuming the district judge used the free version of Chat-GPT, the answers Chat-GPT provided are based on information put in the dataset until the end of 2023. Chat-GPT's answers may therefore not have been (fully) correct.

• A second point of criticism was the District Court violating the right to a fair hearing. The court did not provide Chat-GPT’s output to the parties. As set out in this briefing, the Dutch Supreme Court has made it clear in the Googling Judge Case that this is not acceptable.

Commentary

Chat-GPT can be an interesting tool in a courts’ toolbox if used carefully, with full human oversight and used transparently. As a language tool, Chat-GPT can analyse texts to ‘understand’ the context and intent, while also drawing upon information from its training (up to late 2023) to generate a relevant and coherent response. It can therefore act as a language assistant for the court. However, the court needs to have a full understanding of the intended use of this tool. In other words: the court needs to know its limits.

Furthermore, and in light of the principle of a right to a fair hearing, if the court uses Chat-GPT the court could provide the exact given input and output of Chat-GPT during the hearing or shortly after. The court needs to present its findings and make it so that it is fully comprehensible and transparent how the court arrived at its reasoning. Finally, parties should be given the opportunity to respond.

With one court having used Chat-GPT, we expect that other Dutch courts will follow. Parties in Dutch legal proceedings should pay close attention if and when they expect that Chat-GPT will be asked to hold the hammer.