The Brief:

  • AI now outperforms lawyers on legal research.

  • Counsel Stack, Alexi and Midpage beat human lawyers — and even ChatGPT.

AI is officially better at legal research than lawyers.

A new Vals Legal AI Report has found that four AI tools, like Counsel Stack, Alexi, Midpage and ChatGPT, outperformed a lawyer control group across 200 legal research questions.

All AI tools topped the human baseline of 69%, with the legal-specific AIs scoring 76–78%, and ChatGPT close behind at 74%. Accuracy was high — roughly 80% for legal and general AIs, compared with 71% for human lawyers.

But the key differentiator was citations.

Legal-specific systems scored six points higher on authoritativeness, drawing on proprietary databases — even if much of the data is publicly available.

The study, designed with input from Paul Weiss, McDermott, Reed Smith and Paul Hastings, found that AI outperformed lawyers on three-quarters of all questions, sometimes by margins exceeding 30 points.

But luckily, humans still held an edge where context, nuance and multi-jurisdictional judgment were required. We’ll take that as a win.

Several major legal AI companies declined to participate, despite appearing in Vals’ earlier report on tasks like redlining and document review. The study also relied on “zero-shot prompts” — single, context-free questions — which Vals concedes don’t reflect how lawyers or AI tools typically handle real research workflows.

Still, it’s a hard pill to swallow.

AI isn’t just matching lawyers. It’s overtaking them.

For Australian firms, that’s a wake-up call. Those investing early in AI-ready infrastructure and platforms like Harvey, Legora and CoCounsel will likely seize the lead, while others risk being left behind in law’s next productivity boom.

Comment

Avatar

or to participate

You might like