Let me paint a picture for you of a hypothetical acquaintance named “Lucas”. He’s a bright guy. Driven, tech-savvy, and eager to learn. The more he knows, the more capable and in control he feels. But in his desire to understand things better, you’ve also seen Lucas rely heavily on AI chatbots over the last couple of years. And while they can be helpful about 30% of the time, the rest of the time, you’ve seen AI…
Mind you, Lucas is not stupid. He’s not gullible, either. He trusts the software that is supposed to simply collect and repeat accurate information is doing what it is supposed to do. But that trust has the capacity to lead him into incredibly upsetting and detrimental decisions. And if AI can mislead and hurt a bright, educated man who’s good with tech, it can mislead an educated, thoughtful attorney, a paralegal, and your own clients.
So what are the risks of AI, why does this keep happening, and what value does human office help bring to your firm? Let’s break this down. Because the better you understand AI’s abilities and limits, the better you can protect yourself, protect your value as an attorney, and protect your clients.
How does AI wind up suggesting bleach as a medication? Or lead a depressed young person to hide suicidal feelings from his parents? What is going on here? A basic Google search would never gently suggest these sorts of things; it would direct you to a suicide hotline, instead. What makes AI different?
Well, AI Large Language Models (LLMs) are trained to simply give answers, keep people talking, and pry as much data out of users as possible. This data, in turn, trains the AI on responses, tone, and usability. Resultingly, LLMs don’t know how to say “I don’t know” very well (as this might stop the conversation dead in its tracks) and will sometimes make things up and use excessive agreeableness to keep people talking.
This wouldn’t be as detrimental if you just needed a quick chicken casserole recipe, but if you’re discussing a mental health issue, marriage problems, or a complex legal decision, the results can be disastrous.
Multiple studies, including benchmarks developed by Cornell University, have shown how easy it is for AI conversations to divert into some terrible, damaging places. And because AI can come across as seemingly human, sympathetic, and affirming, this type of interaction can become emotionally addictive for people who are scared, lonely, or facing a major decision. Like the type of critical decisions your clients face each day.
And the end result can be terrible legal advice.
For example, a young man being accused of rape after sleeping with a fellow student may ask an AI chatbot if he should text her an apology. The AI, seeking to be agreeable, may tell him that’s a great idea that will demonstrate his maturity and responsibility.
A good lawyer, however, would never give this advice and would instead advise him to avoid any contact with the alleged victim and to speak only to his attorney. As you can see, AI can make your clients’ cases far more complex, leading to detrimental choices that will need to be undone later, often at great cost to the client.
This same over-agreeability and refusal to say “I don’t know” has led lawyers who rely on LLMs to make terrible mistakes in court. The same software that can hallucinate a car problem can hallucinate entire cases.
In fact, earlier this year, an elite Wall Street firm, Sullivan & Cromwell, had to apologize to a Federal judge for submitting a court filing that contained harmful AI hallucinations, including entirely fabricated cases. The errors were severe, and over three dozen in number.
And this is not the first time this has happened, either. Remember, AI is not omniscient. It is not always right. This is why relying on a human (such as a paralegal or an ILA) for careful legal research is absolutely crucial.
Trust nothing, verify everything. Simply because the software sounds confident and cheerful doesn’t mean it’s displaying accurate information. Trusting AI without verifying could cost you a case, wreck your reputation, and lead to costly fines, as well.
Instead, make sure you understand how to use AI properly, and when to hit the books yourself. Make sure your in-office paralegal is trained on proper, safe AI use, as well. At DocketWorks, this is exactly why we ensure our ILAs understand various systems, prompts, the dangers of hallucinations, and the limits of emerging tech.
These guardrails keep you safe, keep your firm’s reputation intact, and ensure your clients receive quality, human, responsible, and informed representation at each step.
You may have heard of swirling debates regarding AI sentience. Is this stuff conscious? Is it capable of feeling? But for me, a more pressing question lies in the words we have used to describe this technology: Artificial Intelligence.
That is exactly the point. It is not genuinely, humanly sapient. It can not experience the world as a human can. It does not learn to walk, throw temper tantrums in grocery stores, or eventually learn to self-regulate as a child would. It can not be conscious and is not genuinely intelligent.
Instead, it’s technology retrieving answers and responses based on training data, coding, goals set by developers, and its own inner reasoning process, which is guided, not by a human soul or spirit, but by algorithms, reward protocols, and wiring. When phrased that way, who in the world would want this technology to help get them through their divorce?
This, Sir or Ma’am, is exactly where your value lies. You are, in fact, intelligent. You passed the bar. You care about your clients. You don’t hallucinate utter bull to please a section of coding. You can put yourself in your clients’ shoes, care about their rights and futures, and then fight hard for their cause.
ChatGPT, Claude, and Gemini could never do that. They can proofread, make grammar suggestions, and help you find a great local coffee shop, but they’re not attorneys. And they shouldn’t be helping you with the heavy lifting, either.
To be clear, it’s ok to use AI. I use it. But I don’t allow it to take the wheel on my most important tasks. When you need reliable, human, intelligent help, and when your clients invariably need the same, you’re going to need more than software sycophancy. You’re going to need an attentive, compassionate, bright human being to come up alongside you and work well.
Want an assistant who won’t hallucinate casework? Reach out to DocketWorks. Your ILA can answer emails, organize discovery packets, draft motions, and handle billing with precision, care, and a personable focus on your firm’s success and reputation.
AI doesn’t care if you get fined, get laughed out of court, or lose a case for a client. This is where you let go of the LLM and switch to a better, safer, and far more secure option. You let your ILA go to bat for you.
Want to learn more about what an ILA can offer your firm? Reach out to us, schedule a discovery call, and we’ll answer your questions and help you decide the next step.
Because your law firm, your clients, and your legal career all deserve far better than the above meme. You know it, we know it, and Sam Altman knows it, too.
Grace Singh is a writer and editor for DocketWorks. She enjoys bridging services and client needs in ways that are meaningful, memorable, and human-focused, even as technology continues to change. When she’s not at her home office, she enjoys nature walks, reading, and brewing coffee.