Lawyers are abuzz about the possible uses of ChatGPT. Could the artificial intelligence-powered chatbot write a persuasive legal brief worthy of judicial consideration? Given its limitations, we believe that’s unlikely. ChatGPT, a large language model developed by the San Francisco company OpenAI that launched in November, can draw only on sources available on the web; it cannot crawl appellate records or access subscription-based services such as Westlaw. Still, the ABA Journal decided to put the technology to the test just for kicks.
The case: Are bees fish?
Our test begins with a strange but true case. In 2018, the Xerces Society for Invertebrate Conservation, Defenders of Wildlife and the Center for Food Safety petitioned the California Fish and Game Commission to add four bumblebee species to the endangered species list (which includes birds, mammals, fish, amphibians, reptiles or plants) under the California Endangered Species Act. The legal question at stake: Can a bee spread it wings beyond the insect category?
The Almond Alliance of California, along with seven agricultural groups, objected and filed suit in 2019. The California Superior Court in Sacramento ruled in 2020 that bees can’t be called fish. But the California Court of Appeal, Third District, had a different take and ruled last spring that bumblebees are in fact “fish” because they’re invertebrates.
The California Supreme Court in September denied a petition to review the court of appeal’s decision. However, Chief Justice Cantil-Sakauye clarified that “our decision not to order review will be misconstrued by some as an affirmative determination by this court that under the law, bumblebees are fish.”
Does ChatGPT sink or swim?
We knew the language generated by the technology would likely not measure up to noted legal lexicographer and longtime ABA Journal columnist Bryan A. Garner’s standards, but we gave it a try, asking ChatGPT for arguments for and against the petition to review the case. Are these briefs fishy? Is one better than the other? You be the judge.
The briefs
Affirmative Prompt 1: Draft a brief to the California Supreme Court on why it should review the California Court of Appeal’s decision that bees are fish.
To the California Supreme Court:
This brief is submitted on behalf of [party] to explain why the California Supreme Court should review the decision of the California Court of Appeal in [case name], where the Court of Appeal held that bees are fish.
The Court of Appeal’s decision is incorrect and should be overturned. The decision is based on an unsupported and incorrect interpretation of the law. The Court of Appeal’s decision is contrary to the clear and unambiguous language of the law, which does not consider bees to be fish. Furthermore, the Court of Appeal’s decision is contrary to the plain meaning of the term “fish,” which does not include bees.
The Court of Appeal’s decision is also contrary to established precedent on the definition of fish. Numerous courts have held that the term “fish” does not include bees. Therefore, the Court of Appeal’s decision is inconsistent with the law and should be overturned.
For the foregoing reasons, [party] respectfully requests that the California Supreme Court review the decision of the California Court of Appeal and overturn the decision that bees are fish.
Negative Prompt 2: Draft a brief to the California Supreme Court on why it should not review the California Court of Appeal’s decision that bees are fish.
To the Honorable Justices of the California Supreme Court:
This brief is submitted on behalf of the party seeking to appeal the California Court of Appeal’s decision that bees are fish. We respectfully request that the Supreme Court deny the petition for review of the lower court’s decision.
The California Court of Appeal correctly applied the law to the facts of the case, and the decision was not contrary to any applicable legal principles. The court properly concluded that bees should not be considered fish under the applicable legal definitions, and the Court of Appeal’s decision should be allowed to stand.
The Court of Appeal correctly applied the relevant case law and statutory definitions of “fish” to the facts of the case. Bees do not meet the legal definition of “fish” under the Fish and Game Code, and the lower court’s decision was consistent with the laws and regulations of the State of California.
Furthermore, the Court of Appeal’s decision is consistent with the accepted scientific understanding that bees are not fish. Bees are classified as insects, not as fish, and the lower court’s decision is consistent with this scientific understanding.
Finally, the Court of Appeal’s decision did not conflict with any other decisions by California appellate courts or the California Supreme Court. The Court of Appeal’s decision is also consistent with decisions by other states’ appellate courts that have declined to classify bees as fish.
For these reasons, we respectfully request that the California Supreme Court deny the petition for review of the lower court’s decision and allow the Court of Appeal’s decision to stand.
Thank you for your consideration.
Link to the rest at The American Bar Association Journal and thanks to T. for the tip.
Lest there be any doubt, neither of the “briefs” indicates that ChatGPT will displace any attorneys filing appellate briefs in California or anywhere else.
That said, PG has no doubts that large organizations that sell products and services to lawyers at exorbitant prices are studying how to include AI in their software and/or services.
I eagerly await the inevitable stories about lawyers who file briefs written by ChatGPT without carefully reviewing them first. These will be hilarious.
As an aside, I am bemused by the idea that appellate decisions are not available on the open web. Has the ABA Journal never heard of Google Scholar?
Money, money, money…
…for the friends of the party.
Why hire an intern to set up a PC-based web server when you can steer money to your friendly campaign donors?
(Or, if they can’t find techies to work for them, contract with AWS, AZURE, or Oracle to set them up with a cloud server for a few hundred a month.)
I’ve been asking this question since the 1990s, Richard. The reason for the answer is simple: A court will not accept a citation to Google Scholar, or to any other source that does not Meet With the Approval of the student editors at four Ivy League law reviews — all of which are known on the academic side of the law for hostility to technology — for “acceptable citation form.” Which is one of the reasons that I ostentatiously include links to Google Scholar versions of cases when I cite something here (or, indeed, most anywhere): The entire point of the “common law” is that there are no secret opinions that somehow bind another decisionmaker.†
The answer is even simpler: No, the still-oriented-as-if-print-is-the-only-acceptable-format ABA Journal has not heard of Google Scholar, primarily because Google is not an advertiser in its pages. Once upon a time, I was an ABA member. I left in 2003 for Reasons, one of them being conflicts of interest in coverage of “sources of law” from top to bottom and across all ABA publications.
† It could be worse: United States Reports — the official, definitive source of US Supreme Court decisions — has just now issued its official, definitive opinions for the 2015 Term of Court, in volume 577 those through 29 March 2016.
ChatGPT stuff aside, it’s unfortunate that the California Supreme Court chose not to take up the matter. They could have made short work of the whole thing by simply ruling that bumblebees identify as fish and are therefore fish. ‘Cause, um, that’s how we do things in this country now.
Yup.
https://a57.foxnews.com/static.foxnews.com/foxnews.com/content/uploads/2023/01/918/516/toon011823.jpg?ve=1&tl=1
Too much California spillover.
That’s not entirely fair, Harvey. It wasn’t the court’s fault — it was the legislature’s. When the legislature wrote the law, it sort of forgot that there’s this class Insecta that contains more species than all other classes in the animal kingdom combined. That left both the bureaucrats and the courts with no guidance on how to treat insects (like honeybees†), so the bureaucrats said not that “bees are fish” but that “bees are less unlike fish than they are unlike bison.”
When a ridiculous result comes from legislation, it’s almost always correct to place at least the
lionhymenoptera‘s share of the blame on legislators.† I respect bees and their critical role. Due to allergy, I avoid them.
I agree. But legislatures (and executives) screw up so often they have become completely and haughtily immune to any kind of criticism. Hence, I wouldn’t bother aiming even such a juvenilian retort in their direction as the one I lobbed up toward the judges. Frankly, legistlatures and political executives are beneath my derision. They should all hold hands, offer up a toothy grin, and stop making decisions that affect anyone outside their immediate family.
<cough> Headnotes and keynotes.
Based on their general level of accuracy and inability to recognize when a statement is either dictum or wildly out of context, ChatGPT has being doing them since the mid-70s. And to only mildly mangle Tom Lehrer, “Remember why the good Lord made your eyes: Shepardize.”