AI Lawyers Are Already Better Than Law Professors at Reasoning—Say Law Professors
A Stanford-led study found that law professors often preferred AI-generated answers to answers written by other professors in contract law tasks. Sixteen professors from 14 U.S. law schools created 40 questions on doctrine, case law, hypotheticals, and policy. In 2,918 blinded comparisons, Google’s Gemini 2.5 Pro was preferred 75.92% of the time and NotebookLM 74.75%, meaning AI won about three-quarters of matchups. The results also held across recall, hypotheticals, and policy questions, and were partly consistent across graders, suggesting some shared professional standards. AI responses were also rated harmful less often than human ones. The researchers noted, however, that the study did not test whether AI matched any individual professor’s teaching style, only whether it produced generally acceptable answers. The findings come as courts, firms, and law schools increasingly adopt AI while still confronting hallucinations and citation errors.
