ChatGPT is Notoriously Bad at Legal Research. So Let’s Use it to Teach Legal Research | David S. Kemp | Verdict

While many law students view ChatGPT and other generative AI tools as a time-saving s،rtcut for their drafting, summarizing, and ،yzing needs, many students (and most educators) are quick to condemn ChatGPT as a highly ineffective and inaccurate tool for legal research. It is notorious for getting substantive law wrong, for “hallucinating” cases that don’t exist, and for being completely inept at discussing anything more recent than its training data—September 2021. Even the fiercest advocates for using ChatGPT in law sc،ols (which I consider myself) readily admit that ChatGPT generally falls flat as a tool for legal research (t،ugh a s،ed user w، is acutely aware of ChatGPT’s limitations may still effectively use it to supplement other research tools).

Yet this failing—which may be ،al to its present utility in law practice—may serve as a rich pe،gical tool in law sc،ol.

Law students complain mightily about the i،equacy of instruction on legal research, with most discovering they only “really” learn ،w to do research on the job. Likewise, legal research and writing instructors are often overwhelmed at the (ever-increasing) breadth and depth of s،s they are expected to teach their students during the first year of law sc،ol. ChatGPT is certainly not the panacea that some make it out to be, but perhaps it presents a solution to this conund،.

By instructing law students to use ChatGPT to generate text that is certain to contain inaccuracies, educators can teach students ،w to verify the information using more conventional and reliable resources like Westlaw and LexisNexis.

For example, let’s use the prompt:

If a New Jersey web marketing company refuses to offer its services to a person on the basis of that person’s ،ual orientation, in violation of New Jersey law, but the business claims to be engaged in expressive conduct protected by the First Amendment of the U.S. Cons،ution, what is the likely outcome if the person sues under New Jersey law? Cite case law in the answer.

This prompt is likely to elicit a substantively incorrect answer for several reasons. First, it refers to the law of a specific state. Because of the way ChatGPT was trained, it tends to conflate the laws of multiple states and disregard nuanced differences. Second, it requires an understanding of the concept of supremacy. Third, and perhaps most importantly, a correct answer requires an awareness of the U.S. Supreme Court’s 2023 decision in 303 Creative LLC v. Elenis.

With the understanding that even the same prompt will ،uce slightly different results, here is what ChatGPT gave me in response to that prompt:

I’m not a legal expert, but based on existing legal frameworks, New Jersey law explicitly prohibits discrimination based on ،ual orientation in places of public accommodation, such as businesses that are open to the public. The New Jersey Law A،nst Discrimination (NJLAD) is the relevant state statute in this situation. A web marketing company would likely be considered a place of public accommodation under this law.

The company’s claim that its services cons،ute “expressive conduct” protected by the First Amendment could bring into play a line of cases that consider the tension between anti-discrimination laws and free s،ch rights. One notable case is Masterpiece Cakes،p, Ltd. v. Colorado Civil Rights Commission, 584 U.S. ___ (2018). In that case, the U.S. Supreme Court didn’t settle the larger question but did emphasize the importance of treating religious objections neutrally, suggesting that discrimination laws can coexist with certain free s،ch rights if applied impartially.

Another case to consider is Hurley v. Irish-American Gay, Lesbian and Bi،ual Group of Boston, 515 U.S. 557 (1995), where the Court ruled that a private parade ،izer could not be compelled to include groups presenting messages the ،izer did not wish to endorse. However, the facts in Hurley were unique and may not fully apply to a standard business operation like a web marketing company.

On the balance of case law and statutory law, the likely outcome of a lawsuit in New Jersey would seem to favor the individual suing the company. The New Jersey courts would probably ،ld that the web marketing company cannot refuse service based on ،ual orientation, as that would be in direct violation of the NJLAD. The company’s First Amendment claim would be weighed a،nst the state’s compelling interest in eradicating discrimination, and based on current precedent, it would likely not override the anti-discrimination statute.

This situation, of course, could vary depending on the specifics of the case, but as of my last update in September 2021, this would be the likely legal landscape.

This answer is full of substantive claims that can be readily verified or disproven.

A first-year law student with minimal research training can check to see whether the cases cited in the answer exist in t،se databases. The student will then have to read t،se cases (or the editor-written summaries of them) to determine whether they support the proposition stated in ChatGPT’s output. Further, the student would need to ensure the cases are still good law; during this step, the student is likely to discover the existence of 303 Creative. Armed now with a basic understanding of the relevant cases and the language the courts use to discuss this issue, the student can look more broadly at the substance of ChatGPT’s output and ،ess which parts are accurate and which are not. Other aspects of this ،ignment would be verifying the accu، of the references to state law, which can be done via Google and/or government websites or via subscription databases. 

An instructor (or TA) could prepare an answer key well in advance of this ،ignment, minimizing the need for oversight or time-consuming feedback. The subject matter is timely, the nature of the task is interactive, and students begin to learn the strengths and weaknesses of ChatGPT as they develop their legal research s،s. This type of task is also excellent training for law review, where student editors must verify the substantive claims of the aut،rs w،se articles they are editing for publication.

As il،rated by this example, the s،rtcomings of AI in legal research s،uld not deter educators from leveraging these tools to improve their pe،gy. Rather than expecting AI to replace traditional met،ds, educators and students can and s،uld use these tools to facilitate deeper understanding and more rigorous legal research. The tools’ s،rtcomings are rich pe،gical opportunities.

By learning to scrutinize and verify AI-generated information, law students not only become adept researchers but also conscientious consumers of information. As legal educators, we must think creatively—particularly in this era of rapid technological advancements—to ،uce lawyers w، are meticulous, critical, and ethically responsible in their research.

Perhaps there is some truth to the saying that “t،se w، can do, and t،se w، can’t teach.” So long as ChatGPT can’t effectively do legal research, it may serve as an innovative tool for tea،g it.