Author archives

Sean Harrington is a Law and Technology Librarian Sandra Day O’Connor College of Law at Arizona State University. Sean has a deep-seated passion for the intersection of law and computer science. As a law and technology librarian, I am intrigued by the possibilities of these advanced computational models, particularly how they can transform the way we approach legal and ethical challenges. I aim to bring clarity to complex concepts and ensure that LLMs are understood and utilized effectively by students and faculty.

Evaluating Generative AI for Legal Research: A Benchmarking Project

It is difficult to test Large-Language Models (LLMs) without back-end access to run evaluations. So to test the abilities of these products, librarians can use prompt engineering to figure out how to get desired results (controlling statutes, key cases, drafts of a memo, etc.). Some models are more successful than others at achieving specific results. However, as these models update and change, evaluations of their efficacy can change as well. Law Librarians and tech experts par excellence, Rebecca Fordon, Sean Harrington and Christine Park plan to propose a typology of legal research tasks based on existing computer and information science scholarship and draft corresponding questions using the typology, with rubrics others can use to score the tools they use.

Read more

Subjects: AI, KM, Legal Research, Legal Research Training, Legal Technology, Search Engines, Search Strategies

The Case For Large Language Model Optimism in Legal Research From A Law & Technology Librarian

The emergence of Large Language Models (LLMs) in legal research signifies a transformative shift. This article by Sean Harrington critically evaluates the advent and fine-tuning of Law-Specific LLMs, such as those offered by Casetext, Westlaw, and Lexis. Unlike generalized models, these specialized LLMs draw from databases enriched with authoritative legal resources, ensuring accuracy and relevance. Harrington highlights the importance of advanced prompting techniques and the innovative utilization of embeddings and vector databases, which enable semantic searching, a critical aspect in retrieving nuanced legal information. Furthermore, the article addresses the ‘Black Box Problem’ and explores remedies for transparency. It also discusses the potential of crowdsourcing secondary materials as a means to democratize legal knowledge. In conclusion, this article emphasizes that Law-Specific LLMs, with proper development and ethical considerations, can revolutionize legal research and practice, while calling for active engagement from the legal community in shaping this emerging technology.

Read more

Subjects: AI, KM, Law Librarians, Legal Research, Legal Research Training, LEXIS, Search Engines, Search Strategies, Westlaw
➤