AI Answers Law Student Questions Better Than Professors, Stanford Study Finds
The study challenges assumptions about AI's role in judgment-intensive fields like law and suggests AI tutors could complement classroom instruction with high-quality, on-demand support.
Reporting from 1 sources: GIGAZINE.
A Stanford Law School study found that law professors rated AI-generated answers to student questions significantly higher than human-written answers, with a 75% win rate for AI in direct comparisons. AI answers were also flagged as educationally harmful only 3.5% of the time, versus 12% for human professors.
Julian Nyarko, a law professor at Stanford Law School and director of the Legal Innovation and Frontier Technology Lab, led a study with colleagues from Yale and New York University. Sixteen law professors from U.S. law schools created 40 representative questions a student might ask during a contract law lecture, then wrote their own answers. The research team had AI generate answers to the same questions, adjusted to match the length and structure of the human answers, and had the professors evaluate all answers without knowing which were human-written and which were AI-generated.
After evaluating 2,918 answers, the professors rated AI-generated answers significantly higher. AI answers had a win rate of about 75% in direct comparisons with human professors. The proportion of answers flagged as educationally harmful was 12% for human professors and only 3.5% for AI. Nyarko said the study challenges important assumptions about the role of AI in legal education, noting that the questions required integrating complex content and applying it to new situations.
Synthesized by Yomimono from the 1 cited source below, including Japanese-language reporting where cited, then editorially reviewed before publishing.