Study: About Half of Lawyers Have Used Generative AI, At Least 20% Don’t Fact Check the Work

Though in their absolute infancy, generative artificial intelligence (gen AI) tools like ChatGPT have reverberated across the world of work for their potential to create efficiencies at a bare minimum or replace human workers and transform the labor force at an extreme maximum.

The legal industry, where researching and writing are core responsibilities, is ripe for the gen AI revolution.

A study done by researchers at Princeton University, University of Pennsylvania, and New York University found “legal services” to be the industry most exposed to AI. Goldman Sachs economists estimated 44% of legal work could be automated, which was second to only the work done by office and administrative support jobs (46%).

According to a recent Bloomberg article, law firms are increasingly hiring artificial intelligence experts, not tort law aces.

Much legal industry work is tailor made for gen AI. But this is also work that has profound impacts on people. From the most heinous of crimes to the most petty of offenses, the courts of law determine who is guilty and who is innocent every single day.

For lawyers to potentially rely too much on gen AI so soon is cause for concern. Just recently on June 23, a federal judge fined two lawyers and a law firm $5,000 when their use of ChatGPT resulted in six fictitious legal cases being cited to support their written arguments.

To dig deeper on gen AI’s present and potential impact on the legal industry, Expertise.com surveyed 1,000 practicing lawyers to gauge how many are already using it, what they’re using it for, what they’re worried about, and what we should be worried about.

(See full survey results and methodology at the bottom)

Findings & Analysis

Find the following key findings and analysis below:

• 47% of Lawyers Have Used Gen AI, 43% Say Their Clients Aren’t Aware

• At Least 20% of Lawyers Using Gen AI Don’t Fact the Work

• 47% of Lawyers Say Their Gen AI Work Is Better Than Work They Previously Did, Just 13% Say It’s Worse

• 46% of Lawyers Don’t Believe Gen AI Poses Ethical Concerns for the Legal Industry

• 44% of Law Firms Have Enacted Gen AI Policies, 31% Have Not

• Plurality of Lawyers Aren’t Concerned Gen AI Will Replace Them

47% of Lawyers Have Used Gen AI, 43% Say Their Clients Aren’t Aware

Simply enough, we asked our respondent pool of 1,000 practicing lawyers ages 25 and up if they’ve used gen AI to help with their legal work:

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up.

Nearly half of current lawyers have used tools like ChatGPT, while 28% have not.

With 25% preferring not to answer the question, it’s possible the actual percentage of lawyers using gen AI for work is a lot higher than 47%.

Gen AI is still an incredibly young technology, with ChatGPT being released less than a year ago in November 2022. Yet still, at least 47% of practicing lawyers already are using it in their work.

As Gen AI progresses in its capabilities and becomes more acceptable as a workplace norm, expect the technology to become the rule rather than the exception amongst legal professionals.

Interestingly, when we asked the 47% of lawyers who have used Gen AI if their clients are aware this technology was being used to help with work related to their cases, here’s how they answered:

• 37% said “yes”

• 43% said “no”

• 20% said “I’d rather not say.”

The plurality of lawyers are not telling their clients they’re using Gen AI. And with another 20% opting not to say, this could actually be the majority.

It’d be interesting to hear what clients would think about their cases being leg worked with Gen AI. Like a lot of things in the workplace, full disclosure is probably going to be the safest policy for law firms who are using Gen AI.

This is one of the issues when it comes to Gen AI in the legal profession. Perhaps the biggest relates to fact checking Gen AI legal work, which we tackle in the next section.

At Least 20% of Lawyers Using Gen AI Don’t Fact Check the Work

When we asked the lawyers using Gen AI for their work if they’re subsequently fact checking that work, here’s how they answered:

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up. This specific question was only asked to lawyers who answered they've used Gen AI to help with their work.

While the majority of lawyers are fact checking their Gen AI legal work, at least 20% are not. We say at least because another 20% of lawyers preferred not to answer the question.

Regardless, 20% is still too high a percentage of lawyers not double checking their Gen AI legal work for accuracy.

One of the biggest concerns when it comes to the proliferation of Gen AI is the potential rampant spread of misinformation. It’s an incredibly powerful tool capable of pulling information (real or not) from all corners of the internet and then spitting that information back out as an educated, plausible response.

And it does this in just seconds. And it’s in the hands of billions of internet users.

If Gen AI is not managed and checked by responsible humans, misinformation could circulate at unprecedented velocity and cause great harm.

This is the perilous potential of Gen AI. Leveraging it’s undeniable ability to increase productivity at the expense of accuracy.

Perhaps nowhere is this of more concern than within the legal industry, where lives and futures hang in the balance of precedents and research every single day. As noted in the introduction, two lawyers and a law firm were already fined by a federal judge when their use of ChatGPT resulted in six fictitious legal cases being cited to support their written arguments.

When we asked lawyers who have used Gen AI for work what they have used it for, here’s how they answered:

• 45% of respondents said, “document review and analysis”

• 49% said, “legal research (i.e. historical precedents, previous cases)”

• 46% said, “document writing and preparation (i.e., legal briefs, summaries)”

• 23% said, “other”

Gen AI’s use within the legal industry is here to stay, and it has the potential to greatly increase research capabilities and productivity, while also giving lawyers more time to focus on human aspects of the profession that are impossible to replace.

It’s a great marriage but must be used responsibly, and the work must be fact checked at every turn.

47% of Lawyers Say Their Gen AI Work Is Better Than Work They Previously Did, Just 13% Say It’s Worse

Amongst lawyers who have used Gen AI for their work, we asked if they believe the Gen AI work is better, worse, or of the same quality as the work they would have previously done in a pre-AI world:

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up. This specific question was only asked to lawyers who answered they've used Gen AI to help with their work.

Not only does the clear plurality of lawyers believe their Gen AI legal work improves upon work they would have previously done, just 13% thought the Gen AI work was worse.

It further proves the amazing potential of Gen AI in the legal industry, but still with the caveat that this technology must be used responsibly and with diligent fact-checking.

Interestingly, we also asked lawyers who have used Gen AI if they believe the technology has made them a better lawyer. So a bit different than asking them if they believe the Gen AI legal work is better than their comparable work in the past.

• 39% said, “yes”

• 42% said, “no”

• 19% said, “not sure/I’d rather not say.”

Most believe their Gen AI legal work is better, but most lawyers also believe Gen AI has not actually made them a better lawyer.

When viewed in tandem, the results to these two questions demonstrate where Gen AI work might be most effective and where it’s limited.

Producing great legal research does not equate to being a great lawyer.

There are so many human aspects of being a lawyer, like speech delivery or emotional intelligence, that Gen AI might never be able to improve or replace. Going forward in the legal industry, perhaps these are the skills that hiring legal firms place tremendous weight on, while other more AI-replaceable, research-based skills hold less value.

Building upon this theme that there are certain irreplaceable legal skills, we asked lawyers who have used Gen AI if they believe Gen AI gives them an advantage over lawyers who are not using Gen AI:

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up. This specific question was only asked to lawyers who answered they've used Gen AI to help with their work.

While most lawyers do believe Gen AI is giving them an advantage, the gap is not huge.

Based on everything we’ve already discussed, it’s likely most of the advantages coming from lawyers using Gen AI relates to increased productivity and new efficiencies as Gen AI takes on the leg work around research and writing.

We asked Gen AI-using lawyers if the technology has made them more productive and/or efficient, and here’s how they answered:

• 45% said, “yes”

• 37% said, “no”

• 18% said, “I’d rather not say.”

46% of Lawyers Don’t Believe Gen AI Poses Ethical Concerns for the Legal Industry

We asked all lawyers, whether they’ve used Gen AI or not, if they believe the technology poses ethical concerns for their profession.

• 29% said, “yes”

• 46% said, “no”

• 25% said, “I’d rather not say.”

Despite already known issues around Gen AI produced misinformation and lawyers not fact-checking their Gen AI produced work, it’s interesting to see the plurality of lawyers still don’t see any ethical concerns.

Amongst lawyers from our survey who do see some ethical concerns surrounding Gen AI in the legal industry, we asked them what specific concerns:

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up. This specific question was only asked to lawyers who think there are ethical concerns around the use of Gen AI in the legal profession. 

Most lawyers see issues around a lack of transparency in AI decision-making, while bias in AI algorithms was the second most cited concern. Inadequate human oversight was the third most populated answer.

It appears most ethical concerns stem from Gen AI’s already well-published shortcoming and greatest risk. If this powerful technology is not harnessed and moderated by responsible human beings, it has the perilous potential to mislead, misinform, and misjudge.

Lawyers and law firms who simultaneously understand the promises and pitfalls of Gen AI will be the ones that maximize the various tools to their benefit. Those who don’t run the risk of ethical breaches and reputation loss, no matter how much extra work they get done.

And this offers a perfect segway into our next section of this research.

44% of Law Firms Have Enacted Gen AI Policies, 31% Have Not

We asked all 1,000 practicing lawyers if their law firm has set up guidelines, rules, or restrictions around the use of Gen AI in work for the firm, here’s how they answered:

• 44% said, “yes”

• 31% said, “no”

• 25% said, “I’d rather not say.”

It’s great to see the plurality of law firms have already set policies for how Gen AI can be utilized in firm work, yet there’s still a significant portion of firms who have seemingly not set any guardrails around using this technology.

At some point in the future, the vast majority of law firms (if not all) should have clear policy around the use of Gen AI, especially considering the serious implications this technology can have on legal cases and people’s lives.

It should be no different than pretty much any type of company today that has set rules and guidelines around things like workplace harassment, DE&I, privacy, and PTO.

Gen AI is here to stay and its impact on the legal industry could be especially profound. Law firms that haven’t established Gen AI guidelines are not only delaying the inevitable, but setting themselves up for a major issue they could have prevented.

And when we asked lawyers who are working for firms that have already set up Gen AI guardrails if they’re allowed to be using Gen AI according to said guardrails, here’s how they answered:

• 59% said, “yes”

• 23% said, “no”

• 18% said, “I’d rather not say.”

While the benefits of Gen AI in the legal industry likely outweigh the risks if it’s used responsibly, having a workplace policy that bans the use of the technology is more prudent than having no policy whatsoever.

Plurality of Lawyers Aren’t Concerned Gen AI Will Replace Them

When it comes to Gen AI and the world of work, the prevailing conversation relates to how many and what jobs will be replaced by the technology.

We asked our respondent pool of practicing two questions related to this:

1. Are you concerned Gen AI will replace jobs in your industry?

2. Are you concerned Gen AI will replace your job?

The above results derive from an online survey of 1,000 practicing US lawyers ages 25 and up. 

Slightly more lawyers (29%) are concerned Gen AI will replace jobs traditionally held by lawyers compared to lawyers who believe Gen AI will replace their job (27%).

Fitting with this theme, slightly more lawyers (46%) are not concerned Gen AI will replace their job compared to lawyers who are not concerned Gen AI will replace industry jobs (45%).

It’s likely natural to think your job is more secure compared to your industry colleagues, but it’s interesting to compare the results of these two questions side-by-side.

As we’ve seen throughout this research, ultimately the lawyers who embrace Gen AI for work will likely get a leg up when it comes to productivity and efficiency, but that doesn’t mean they’ll necessarily become better or more valuable lawyers.

There are still too many human elements and skills related to the legal profession that have a moat against the proliferation of Gen AI.

Nevertheless, lawyers should learn and develop Gen AI skills for their work or they run the risk of getting left behind like accountants who never embraced Excel. It’s a valuable skill to add to a lawyer’s resume and could be a differentiator when looking for a new job or pay raise.

And finally, lawyers who develop Gen AI skills must pair their learnings with responsible use and rigorous fact-checking, or they run the risk of making a major mistake which, ironically, could result in them losing their job.

Survey Results

All survey results derive from an online survey of 1,000 US-based adults ages 25 and up who are “legal services” professionals. For an additional layer of screening, a screener question was asked to insure “legal services” professionals were currently working as practicing lawyers.

Some answers won’t add to 100% due to rounding.

1. How concerned are you that generative artificial intelligence (AI) tools like ChatGPT are going to replace jobs historically held by lawyers?

• 14% – “Extremely concerned”

• 15% – “Somewhat concerned”

• 13% – “Neutral”

• 26% – “Not too concerned”

• 19% – “Not concerned at all”

• 13% – “I’d rather not say.”

2. Are you personally concerned about your own job as a result of the proliferation of generative artificial intelligence (AI) tools like ChatGPT?

• 14% – “Extremely concerned”

• 13% – “Somewhat concerned”

• 13% – “Neutral”

• 28% – “Not too concerned”

• 18% – “Not concerned at all”

• 14% – “I’d rather not say.”

3. Has your law firm enacted guidelines/rules/restrictions around the use of generative artificial intelligence (AI) tools like ChatGPT in work for the firm?

• 44% – “Yes”

• 31% – “No”

• 25% – “I’d rather not say.”

4. (“Yes” to Q3) Are you allowed to use generative artificial intelligence (AI) tools like ChatGPT at your law firm?

• 59% – “Yes”

• 23% – “No”

• 18% – “I’d rather not say.”

5. Do you think the use of generative artificial intelligence (AI) tools like ChatGPT in the legal profession poses any ethical concerns?

• 29% – “Yes”

• 46% – “No”

• 25% – “I’d rather not say.”

6. (“Yes” to Q5) What do you believe are some of the ethical concerns around the use of generative artificial intelligence (AI) tools like ChatGPT in the legal profession (Select all that apply)?

• 43% of respondents – “Lack of transparency in artificial intelligence decision-making”

• 34% – “The proliferation of incorrect information via artificial intelligence tools”

• 42% – “Bias in artificial intelligence algorithms”

• 37% – “Inadequate human oversight in artificial intelligence decision-making”

• 35% – “Unequal access to artificial intelligence tools amongst lawyers”

• 30% – “Other”

7. In your opinion, how prevalent is the current use of generative artificial intelligence (AI) tools like ChatGPT amongst practicing lawyers?

• 16% – “Extremely prevalent”

• 31% – “Somewhat prevalent”

• 18% – “Not sure”

• 20% – “Not very prevalent”

• 15% – “Not prevalent at all”

8. Have you personally used generative artificial intelligence (AI) tools like ChatGPT to help with your work as a practicing lawyer?

• 47% – “Yes”

• 28% – “No”

• 25% – “I’d rather not say.”

9. (“Yes” to Q8) How often are you using generative artificial intelligence (AI) tools like ChatGPT to help with your work as a practicing lawyer?

• 19% – “All the time”

• 41% – “Sometimes”

• 20% – “Very rarely”

• 8% – “Once”

• 12% – “I’d rather not say.”

10. (“Yes” to Q8) Are your clients aware that you’ve used generative artificial intelligence (AI) tools like ChatGPT to help with your legal work related to their cases?

• 37% – “Yes”

• 43% – “No”

• 20% – “I’d rather not say.”

11. (“Yes” to Q8) In what specific ways have you used generative artificial intelligence (AI) tools like ChatGPT to help with your legal work (Select all that apply)?

• 45% of respondents – “Document review and analysis”

• 49% – “Legal research (i.e. historical precedents, previous cases)”

• 46% – “Document writing and preparation (i.e., legal briefs, summaries)”

• 23% – “Other”

12. (“Yes” to Q8) How does the work you’ve completed using generative artificial intelligence (AI) tools like ChatGPT compare to the same work you would have done in the past without the help of AI?

• 47% – “It’s better”

• 28% – “About the same”

• 13% – “It’s worse”

• 11% – “Not sure/I’d rather not say.”

13. (“Yes” to Q8) Has your use of generative artificial intelligence (AI) tools like ChatGPT made you a better lawyer?

• 39% – “Yes”

• 42% – “No”

• 19% – “Not sure/I’d rather not say.”

14. (“Yes” to Q8) Has your use of generative artificial intelligence (AI) tools like ChatGPT made you a more productive/efficient lawyer?

• 45% – “Yes”

• 37% – “No”

• 18% – “Not sure/I’d rather not say.”

15. (“Yes” to Q8) Do you fact check the legal work that you have completed by generative artificial intelligence (AI) tools like ChatGPT?

• 60% – “Yes”

• 20% – “No”

• 20% – “I’d rather not say.”

16. (“Yes” to Q8) As a practicing lawyer that uses generative artificial intelligence (AI) tools like ChatGPT, do you believe you have an advantage over lawyers that are not using the same tools?

• 47% – “Yes”

• 36% – “No”

• 17% – “Not sure/I’d rather not say.”

Methodology

All survey data in this report derives from an online survey of 1,000 US-based adults ages 25 and up who are “legal services” professionals. For an additional layer of screening, a screener question was asked to insure “legal services” professionals were currently working as practicing lawyers and not clerks or back-office employees for example. The appropriately-aged respondents were found via an age-filtering feature. “Legal services” professionals were found via a career-filtering feature. This was an online survey commissioned by Expertise.com and conducted online by online survey platform Pollfish. This survey was conducted in May 2023.