Disclaimer: Do not include confidential, personal data, proprietary, or privileged information in prompts for generative AI products. I am using ChatGPT4.
KEY POINTS:
- The effectiveness of ChatGPT lies in the quality of what you put into it.
- Lawyers and contract professionals should focus on learning how to correctly prime and prompt the model to yield more precise and helpful results.
- ChatGPT can be used to draft a a more reliable limitation of liability clause using the Prompt Template provided in this article.
There are many use cases for ChatGPT in the legal context, but we are still exploring how it can be used to improve contract drafting and negotiations.
According to a recent report by LexisNexis, 59% of respondents use generative AI for researching matters and 45% use it for drafting documents. But the question many legal professionals are asking is whether ChatGPT is accurate and reliable in drafting entire contracts and contractual provisions.
It’s important to remember that ChatGPT was not specifically trained in legal matters. That means, although it can generate language that sounds like legal advice, it does not truly understand the law nor does it stay updated on the latest legal changes. In light of the recent case, Mata v. Avianca, Inc., where attorneys inadvertently cited non-existent case law generated by ChatGPT, it has become abundantly clear that we should not rely on ChatGPT as a primary source for certain legal tasks. In fact, OpenAI’s Terms of Use disclaim output accuracy.[1]
I believe that, rather than focusing on testing ChatGPT’s accuracy, we should focus on learning how to communicate with ChatGPT to yield more precise and helpful results. In this article, I explore ways to leverage ChatGPT to draft a limitation of liability clause. I will walk through the process of priming and prompting my input and measure the accuracy and reliability of the output with each iteration. Additionally, I will share a Prompt Template that I created, which can be used as a tool to guide us and ChatGPT in contract drafting tasks.
ChatGPT Conversation Techniques
Users of ChatGPT and similar tools have developed a set of techniques for communicating more effectively with these generative AI models. The better a user understands and uses these techniques, the more reliable the output will be.
Here are some key terms to understand when using ChatGPT:
- Prompting: This involves asking an initial question or inputting a query to ChatGPT. The prompt can be as simple as a question or as complex as a multi-sentence scenario.
- Priming: This is the process of providing relevant context or instructions to shape ChatGPT’s responses. The prime sets the stage for the conversation and can be used to guide the AI’s responses in a specific direction.
- Refining: This involves modifying or adjusting prompts or priming based on initial responses from ChatGPT to improve the accuracy or relevance of its responses.
- Iterating: This process involves repeating or modifying prompts, priming, or refining to steer the conversation toward the desired direction or improve the quality of responses.
- Probing: This technique involves asking follow-up questions or seeking specific details to delve deeper into a topic or clarify any ambiguous responses from the AI.
Exploring ChatGPT’s Capabilities of Drafting a Limitation of Liability Clause
ChatGPT is not a Legal Language Model and does not have specialized training in law. However, due to the vast amount of internet text it has been trained on, ChatGPT is capable of replicating limitation of liability clauses. This allows it to imitate the structure and pattern of such clauses. However, it does not comprehend the legal implications of the clause, when it should be used, or how to tailor the language for specific situations. That is where human intervention is required to prompt, prime, and refine the responses to improve the output accuracy.
Before leveraging the capabilities of ChatGPT, it’s important for the user to have a clear understanding of the context, terms, and risks pertaining to the agreement. Without this information, I do not recommend using ChatGPT at this time, as there is no way for you to refine the output or check the veracity of the responses.
So, can we trust ChatGPT’s output when drafting limitations to liability clauses? Can we prime, refine, and iterate its output to draft a clause that’s even better than one I could draft? Let’s explore these questions together. The results promise to be quite interesting!
The Plan
- Step 1: Prompt, Prime, and Refine – I will use primary sources from contract experts to prime ChatGPT on key issues involved in drafting and negotiating limitation of liability clauses.
- Step 2: Generate a Prompt Template – After reviewing the output, I will ask ChatGPT to assist me in creating a “Prompt Template” for drafting limitation of liability clauses.
- Step 3: Generate First Draft – Then, I will ask ChatGPT to draft a limitation of liability clause, which I’ll refer to as the “Generic Clause.”
- Step 4: Test The Template – I will test the Prompt Template by pretending to be a lawyer for OpenAI, asking ChatGPT for help in drafting a limitation of liability clause, which we will call the “New OpenAI Clause”.”
- Step 5: Compare the Clauses: I will compare the OpenAI Clause with both the Generic Clause and OpenAI’s actual limitation of liability clause in its Terms of Use, which we will refer to as the “Current OpenAI Clause.”
Disclaimer: I acknowledge this is an oversimplification as I am not taking into account the whole contract and other key provisions that impact limitation of liability like indemnification, but these are the parameters of the experiment for ease of demonstration and testing.
Step 1: Prompt, Prime, and Refine
To ensure I fully grasped the nuances of limitation of liability clauses, I turned to expert resources like Contract Nerds and How to Contract.[2] Their insightful and in-depth content enriched my understanding, enabling me to more effectively guide ChatGPT in our subsequent discussions—essentially, to prime it for better, more accurate responses.
After absorbing the expert content, I wanted to ensure ChatGPT had access to the same knowledge I did. To achieve this, I essentially fed it the information using third-party plugins. For example, I used Link Reader to read and summarize the two online articles, and AskYourPDF to read and summarize the PDF guide.[3] As no plugin currently exists for ingesting sections of content from a book, I typed in my key takeaways from Laura Frederick’s book.
Using this suite of expert content, I refined and iterated ChatGPT responses to align its understanding with mine. Then I tested ChatGPT’s comprehension, by asking it to summarize the five core components of a limitation of liability clause, like this:
Step 2: Generate a Prompt Template
Building on the priming exercise above, and within the same conversation, I decided to create a ChatGPT prompt template for drafting limitation of liability clauses (“Prompt Template”). The purpose of this Prompt Template is to assist in identifying all the relevant data that should be contemplated and incorporated in a limitation of liability clause. Moreover, this template can be used for future drafting tasks.
After several iterations, here is the Prompt Template that I developed, which you can also access and download here:
Step 3: Generate a First Draft – Generic Clause
Having primed and refined ChatGPT based on the above information, I then asked it to draft a limitation of liability clause (“Generic Clause”) without providing any additional context. As expected, the resulting clause was quite generic due to the lack of specific details or context.
Immediately, I noticed an inconsistency between the Generic Clause suggested by ChatGPT and the expert content I had previously inputted. The Generic Clause included a mutual exclusion for “special, consequential, incidental, punitive, or indirect damages.” However, Rebecca Grey’s article advocated for the use of an inclusive list as opposed to an exclusive one.
So I highlighted this discrepancy to ChatGPT, which it then corrected by removing the damages language. While ChatGPT’s initial attempt at a limitation of liability clause is fairly decent, it is not that helpful. It needs more information to draft a useful clause. So, it was time to create a more specific prompt for ChatGPT, as if I were a lawyer representing OpenAI and I wanted to draft a limitation of liability clause for our Terms of Service.
Step 4: Test the Template – New OpenAI Clause
I put the Prompt Template to the test by assuming the role of a lawyer for OpenAI. I asked ChatGPT to assist me in drafting a limitation of liability clause, which we will refer to as the “New OpenAI Clause”.
Here is the text I provided to ChatGPT for this test:
This was the resulting output, the New OpenAI Clause:
Analysis of the New OpenAI Clause
The disparity between the Generic Clause and the New OpenAI Clause is quite apparent. In the NewOpenAI Clause, the response was considerably more specific. It also did not include the “indirect, consequential and special damages” language from the Generic Clause, .
Although the Prompt Template resulted in a specifically tailored clause, it contained language that could be interpreted broadly. Because ChatGPT directly incorporated my very general descriptions of the risks (i.e. “The deal-specific risks are that the user relies on the output of ChatGPT to their detriment which could occur personally, personally or possibly could result in death (for example, if used as a solution for therapy.”), I wondered whether this broad interpretation could potentially preclude the recovery of any damages? Could such an interpretation render the entire contract ‘illusory, void, and unenforceable’?
Here is the specific language to which I am referring:
The Broad Risk Paragraph
With the above concerns in mind, I refined and iterated on this clause. ChatGPT generated a list of six typical situations as an alternative to the Broad Risk Paragraph as follows:
The Risk List Paragraph
While I can’t definitively answer whether the Broad Risk Paragraph or the Risk List Paragraph is better, I generally favor specificity and simplicity. I believe I would be better positioned to argue that the New OpenAI Clause with the Broad Risk Paragraph covers a wider range of unpredictable scenarios and provides clearer notice to the user than a generic list of software exclusions in the Risk List Paragraph.
Returning to the idea that disclaiming all liability could render the contract void, I wondered whether it would be more prudent to remain silent or explicitly try to exclude liability for action that: (a) cannot legally be contracted out of or (b) should be in OpenAI’s control, and therefore, OpenAI should not attempt to exclude.
This question arises from the following clause provided by ChatGPT:
Again, I can’t provide a definite answer, but we’ll return to this clause later when comparing it to the Current OpenAI Clause.
Overall, you can see the vast difference in output between the Generic Clause created without specifics and the OpenAI Clause drafted from the Prompt Template with specific details. And now, you have a Prompt Template (and an alternative way to create your own template) to try with your contract drafting tasks!
Step 5: Compare the Clauses – OpenAI’s Limitation of Liability vs. New OpenAI Clause
Here is OpenAI’s current Limitation of Liability clause:
What is intriguing is that the language generated by ChatGPT in one of the iterations—with the Risk List Paragraph, the six-item laundry list of exclusions, and the “use at your own risk” phrasing—is not drastically different from OpenAI’s actual language.
Had I not primed ChatGPT to exclude the typical “’indirect, incidental, special” damages language, my output might have aligned even more closely with OpenAI’s actual language.
One notable difference, and a unique addition by OpenAI, is the phrase “even if we have been advised of the possibility of such damages.” For the New OpenAI Clause, I wondered whether to include the explicit disclaimer of gross negligence, willful misconduct, etc. Now, considering the language in the Current OpenAI Clause, I would be more inclined to incorporate such language to explicitly disclaim actions that could be labeled as gross negligence or willful misconduct where OpenAI has been advised of the possibility of such damages.
However, the final decision—which of the clauses is most protective and most likely to withstand legal scrutiny—is up to you.
Overall Experience – Not magic, but can be helpful!
The effectiveness of ChatGPT lies in the quality of what you put into it. As an AI model not specifically trained in legal language, it needs careful priming and quality prompting to generate text relevant to distinct legal scenarios.
- Priming is time-consuming: As General Counsel, I am, by definition, a generalist. Therefore, I had to ensure my knowledge about the intricacies of limitation to liability clauses was up-to-date by consulting expert resources. This preparation was critical to prompting, priming, and assessing the output from ChatGPT. The major drawback was the initial time and effort needed to become an expert so I could: (1) prime ChatGPT and (2) verify its output.
- But also very valuable: By leveraging third-party plugins and transcribing my research notes, ChatGPT was able to quickly synthesize, summarize, and prioritize key aspects of a large amount of information more efficiently than I could. ChatGPT proved exceptional at assimilating vast amounts of synthesized information and incorporating my proposed changes into the existing text, thereby saving time and enhancing the final clause.
- Using a Prompt Template is helpful: This led to the easy creation of a Prompt Template, which captured the essential factors to consider when formulating a limitation of liability clause. I liked that it helped me break down the clause into individual components based on reliable primary sources. With the Prompt Template, it was quick and easy to make modifications to my input. This allowed me to compare different drafts which provided new perspectives on risk and the ability to cherry-pick the best language. This exercise was helpful to methodically think through the risks needing to be limited.
While ChatGPT is no magic solution, it’s an effective tool that can quickly move from a blank slate to a thoughtfully composed limitation to liability clause— in a fraction of the time of what would otherwise be a laborious manual task. In contract negotiation, where risk mitigation is paramount, ChatGPT serves as a creative and efficient aid, streamlining the process and saving precious time.
For more expert tips about using ChatGPT with contracts:
- Read more articles in the author’s guest column Chatting Contracts with GPT
- Follow Laura on LinkedIn and reach out to her if you want customized trainings for your legal team
- Subscribe to the Contract Nerds newsletter for weekly tips about contracts straight to your inbox ????????
[1] Section 3 of OpenAI’s Terms of Use on Accuracy state: Artificial intelligence and machine learning are rapidly evolving fields of study. We are constantly working to improve our Services to make them more accurate, reliable, safe and beneficial. Given the probabilistic nature of machine learning, use of our Services may in some situations result in incorrect Output that does not accurately reflect real people, places, or facts. You should evaluate the accuracy of any Output as appropriate for your use case, including by using human review of the Output.
[2] Here is a list of the expert content I relied on for this exercise: Limitation of Liability Clauses, Beyond Just Boilerplate by Rebecca Grey; Five Key Limitation of Liability Negotiation Points by Colin Levy; Practical Tips on How to Contract: Learn How to Draft and Negotiate From a Former Big Law and Telsa Commercial Contracts Lawyer by Laura Frederick; Guide to Limitation of Liability Clauses for Technology Agreements by Nada Alnajafi.
[3] Note AskYourPDF’s inability to interpret visual data or graphs.
