When we wonder how AI manages to write so well, the Large Language Model (LLM) is deployed as the explanation. We are told that it’s all about massive computing power trained on huge databases of text, picking the next word on the statistical probability that it is appropriate. For example, blogger and AI pundit Ethan Mollick[1] writes that LLMs work by
“analyzing a piece of text and predicting the next token, which is simply a word or part of a word. Ultimately, that is all ChatGPT does technically – act as a very elaborate autocomplete like you have on your phone”.
But this seems too simple to explain why any AI-produced text makes sense as an argument, not just a string of sentences. Or why it passes as part of a conversation between two people, with particular motives and histories, who bring a lifetime’s experience of human interaction to its interpretation.
Geoffrey Hinton, often called the ‘godfather of AI’ was interviewed recently on the BBC.[2] He seemed to dismiss the LLM as the sole explanation:
“Some people think that these things don’t really understand, that they’re different from us, just using statistical tricks. That’s not the case. [Take] these big language models for example. The early ones were developed as a theory of how the brain understands language – they are the best theory we’ve got about how the brain understands language. We don’t understand either how they work, or how the brain works in detail, but we think they probably work in very similar ways.”
In other words, the creators of AI have simulated the neural networks in our human brains, and set them in motion to think, to plan, to advise, and to communicate with an apparently accurate sense of audience – in fact to pass the Turing Test with ease. But what actually happens inside these simulated brains is apparently a mystery.
I've been working in the context of legal information design, and one of the issues much discussed there is the simplification of business contracts. It's easy to get ChatGPT or a similar system to rewrite your contract in plain English, or to answer questions on its content. But I wanted to set it a kind of enhanced Turing Test. Can it engage in an informed discussion of its own capabilities in the same way as another researcher in the field might do?
In particular I wanted to know how much insight it had into the context and intentions behind a communication - the pragmatics, if you like. So I pointed it to the six standards of textuality from Robert De Beaugrande and Wolfgang Dressler's Introduction to Text Linguistics (1981). They distinguish between mere grammaticality – the use of correct language forms – and textuality: qualities that turn a string of language into a communication between people.
So I gave it the following prompt:
I am writing a paper on the potential for AI to help rewrite business contracts. I know you can produce correct and grammatical language, but I want to know to what extent you can imagine the motives of the writer and the reader. This has been termed 'textuality' as distinct from 'grammaticality', in a book published by Robert De Beaugrande and Wolfgang Dressler. Please comment on the extent to which you take them into account when rewriting a business contract. I will paste their six standards in next.
Well, ChatGPT didn’t wait for the quote from De Beaugrande and Dressler. It already knew it, and off it went:
The distinction between "textuality" and "grammaticality" is an important concept in understanding how AI like me approaches language tasks, especially when it comes to rewriting business contracts. While I can certainly generate correct and grammatically sound text, understanding the motives of the writer and the interpretation by the reader—both key elements of textuality—requires a broader comprehension of the context, the relationship between the parties, and the desired outcomes of the contract.
In a business contract, the intentions of the writer are often to clearly outline the terms of an agreement, avoid ambiguity, protect legal rights, and ensure that obligations are clearly stated. On the other hand, the reader (often the other party or a legal advisor) will be focused on interpreting the contract in a way that aligns with their own interests, searching for clarity, fairness, and the minimization of risk. The concept of textuality, as Robert de Beaugrande and Wolfgang Dressler describe, involves recognizing the communicative function of the text—understanding not just the literal meaning of the words, but also the pragmatic and discursive layers involved in communication.
When I rewrite a business contract, my approach currently focuses more on maintaining precision, clarity, and consistency in language, as these are critical to the enforceability and legal integrity of a contract. However, I can incorporate elements of textuality by paying attention to…
Chat GPT then listed and accurately explained the six standards. And finishes by explaining its own limitations (with my bold, for emphasis):
While I can simulate an understanding of motives to an extent—such as crafting text that protects one party’s interests or reflects an equitable agreement—there are still nuances in human intention and negotiation dynamics that may escape my grasp, particularly in highly specific or emotionally charged scenarios. Understanding the strategic choices that a writer or a reader might make, or how certain phrasing might influence interpretation, can be difficult for AI because it requires a deep comprehension of human psychology, negotiation tactics, and unspoken norms.
ChatGPT sensed I might want to know more about these limitations and suggested “Feel free to paste the six standards from de Beaugrande and Dressler's work, and I can elaborate further on how I might apply them when rewriting a business contract.” It then went through each standard and explained its own limitations, finishing with wise words about the roles of AI and humans in contract drafting. I have used bold to draw attention to AI’s acknowledgement of its need for human help.
1. Cohesion: Grammatical dependencies on the text surface
AI’s approach: I can effectively maintain cohesion in business contracts by ensuring that there are clear grammatical ties between sentences and clauses. This includes using conjunctions, references, pronouns, and other linking devices that make the text flow logically. For example, clauses that define terms and conditions can be linked to sections on obligations and remedies in a consistent manner.
Challenge: While I can manage surface-level cohesion well, maintaining nuanced legal connections between clauses that subtly reinforce certain rights or responsibilities might require human insight. For example, recognizing when a particular clause needs to be reiterated in later sections for emphasis or legal protection.
2. Coherence: Conceptual dependencies in the textual world
AI’s approach: Coherence in a business contract involves structuring the document so that the underlying concepts (such as liabilities, warranties, or payment terms) are presented in a logically ordered way. I can ensure that key concepts are clearly introduced, defined, and related to each other in a way that aligns with the overall purpose of the agreement.
Challenge: Achieving coherence on a deeper level requires understanding the strategic intentions behind the contract. For example, one party might want to subtly introduce a higher degree of protection against unforeseen risks. Ensuring that this intent is conceptually sustained throughout the document (and is balanced against the other party’s interests) requires an understanding of negotiation dynamics, which can be difficult for an AI to fully grasp.
3. Intentionality & Acceptability: The attitudes of the participants towards the text
AI’s approach: I can be programmed to recognize that a business contract often represents a formal agreement between a Buyer and a Seller. The intent of the Buyer is typically to secure value, manage risks, and ensure that deliverables are provided as expected. The Seller’s intent, on the other hand, is to guarantee payment while outlining the scope of their obligations. I can craft text that reflects these standard attitudes and aims for clarity, fairness, and enforceability.
Challenge: Understanding the strategic attitudes of each participant is where AI may fall short. For instance, one party might intentionally leave a clause vague to create negotiation leverage later, or emphasize certain conditions to signal good faith or flexibility. Capturing these subtleties—reflecting not just the surface-level attitudes but also the underlying power dynamics and negotiation tactics—is difficult for AI to do autonomously.
4. Informativity: Incorporating the new and unexpected into the old and expected
AI’s approach: Informativity is crucial in business contracts, especially when signaling departures from standard templates or introducing novel terms. For example, I can signal changes to standard industry terms by including phrases such as “Notwithstanding the foregoing…” or “Except as otherwise provided…” to clearly mark where a particular clause deviates from established norms. I can integrate innovative clauses that address new circumstances, such as evolving technology or changing market conditions.
Challenge: The challenge here is recognizing when and how to introduce unexpected information without causing confusion or imbalance in the agreement. A human writer might better understand the context and use such deviations strategically to achieve a particular outcome in negotiations.
5. Situationality: The setting
AI’s approach: A business contract operates in a specific context—an agreement between parties typically involving the exchange of goods, services, or money. The roles of Buyer and Seller are clearly defined, with the Buyer focused on obtaining value and minimizing risk, and the Seller focused on delivering the promised goods or services while securing payment. I can write with an understanding of these basic roles, addressing the concerns typically associated with each party (e.g., warranties, liabilities, deadlines, etc.).
Challenge: Situationality also involves understanding the broader social and economic factors that influence the contract. A contract in the tech industry may need clauses around intellectual property and data security, while a construction contract might emphasize timelines and regulatory compliance. Capturing these industry-specific nuances and adapting the contract to reflect them often requires human experience and insight.
6. Intertextuality: Mutual relevance of separate texts
AI’s approach: Business contracts are rarely isolated; they interact with other legal documents, including NDAs (Non-Disclosure Agreements), Terms and Conditions, and Purchase Orders. I can recognize common legal frameworks and templates, ensuring that rewritten contracts are compatible with related documents. This helps ensure consistency across a business relationship’s legal structure.
Challenge: However, understanding the precise interplay between separate documents—such as how a contract might need to reference an NDA’s confidentiality clause or how it aligns with regulatory compliance documents—requires a nuanced understanding of how different texts function together in a specific business setting. AI can assist by suggesting cross-references but may miss subtle strategic alignments that a human negotiator would catch.
Conclusion
While AI can incorporate many elements of textuality when rewriting business contracts—especially in terms of coherence, cohesion, and maintaining the basic roles of Buyer and Seller—there are limits to how well it can capture the underlying motives, strategic intent, and negotiation dynamics. The challenges often lie in understanding the deeper, pragmatic layers of communication, such as when to emphasize or de-emphasize certain clauses, how to signal flexibility or rigidity, and when to introduce or depart from standard norms.
Human oversight remains essential in more complex contract drafting, particularly when there are nuances related to power dynamics, industry-specific concerns, or novel legal strategies. Nonetheless, AI can greatly enhance the efficiency of drafting and ensure that contracts meet high standards of clarity, consistency, and precision.
There's no point in my adding a Conclusion – Chat GPT's done it for me.
[1] Ethan Mollick, Co-intelligence: living and working with AI, WH Allen, 2024, page 9.
[2]Geoffrey Hinton interviewed by Faisal Islam on BBC Newsnight, 17 May 2024.
