Case Facts
Concord Music Group and a group of music publishers sued Anthropic PBC in the Northern District of California, alleging the company trained its Claude models on copyrighted song lyrics without authorization. Publishers’ complaint alleges that Anthropic “knowingly trained its AI models on infringing content on a massive scale in order to enable those models to generate responses to user prompts that infringe Publishers’ copyrighted lyrics.” As discovery proceeded, Magistrate Judge Susan van Keulen was asked to resolve three separate disputes in a single order: Anthropic’s motion to compel Publishers to produce prompts and outputs from their pre-suit investigation that they had not relied on in their pleadings; the scope of a sampling protocol for Claude prompt-and-output records responsive to Publishers’ discovery requests; and Publishers’ challenge to Anthropic’s confidentiality designations on nearly 9,500 produced Claude records, usage statistics, and two training datasets. The sampling dispute - over how many Claude prompt-output pairs touching song lyrics Anthropic would have to produce, and within what margin of error — is where the AI-hallucination issue arose.
The Setup
To support its position on the sampling protocol, Anthropic submitted a declaration from its own expert, identified in the order as Ms. Olivia Chen, addressing the statistical parameters for a representative sample. At a May 13, 2025 hearing, Publishers asked the court to strike Chen’s declaration, arguing that at least one of its citations “appeared to have been an ‘AI hallucination’: a citation to an article that did not exist and whose purported authors had never worked together.” The court gave Anthropic time to investigate before ruling on the challenge.
The Failure
Anthropic’s counsel came back with an explanation the court didn’t fully accept. Counsel “protests that this was ‘an honest citation mistake’ but admits that Claude.ai was used to ‘properly format’ at least three citations and, in doing so, generated a fictitious article name with inaccurate authors (who have never worked together) for the citation at issue.” The court was blunt about what that amounted to:
“That is a plain and simple AI hallucination.”
It drew a distinction, though, from a wholesale abdication of judgment to AI: “the underlying article exists, was properly linked to and was located by a human being using Google search,” so this was “not a case where ‘attorneys and experts [have] abdicate[d] their independent judgment and critical thinking skills in favor of ready-made, AI-generated answers.’” What troubled the court more was what came next. Anthropic represented that a “manual citation check” had been performed but “did not catch th[e] error,” and the court found that hard to square with the scale of the mistake: “it is not clear how such an error - including a complete change in article title - could have escaped correction during manual cite-check by a human being.” The court also noted that the presiding district judge’s civil standing order requires counsel to certify “that lead trial counsel has personally verified the content’s accuracy,” and found that “neither the certification nor verification has occurred here.”
The Ruling
The court struck paragraph 9 of Chen’s declaration - the paragraph containing the fabricated citation - while leaving the rest of the declaration intact, and found that the episode “undermines the overall credibility of Ms. Chen’s written declaration, a factor in the Court’s conclusion.” Turning to the underlying dispute the declaration was meant to inform, the court adopted the parties’ agreed statistical inputs - a 95% confidence level, a Z-score of 1.96, and an expected prevalence of 0.00006 - and split the difference on the disputed margin of error, landing on approximately 11.3% between Anthropic’s proposed 25% and Publishers’ proposed 5%. The court ordered Anthropic to produce a random sample of 5 million Claude prompt-output pairs, divided evenly between a pre-suit period (September 22 to October 18, 2023) and a post-suit period (October 19, 2023 to March 22, 2024), due no later than July 14, 2025.
The Kicker
A single mis-cited paragraph in a discovery-fight declaration didn’t cost the expert her testimony - but it did cost her a credibility finding in a federal order, in a case about whether an AI company’s own outputs infringe copyrighted work.
How the AI Issue Unfolded
→ Publishers flagged a citation in Anthropic statistics expert Ms. Chen’s declaration as an apparent AI hallucination at a May 13, 2025 hearing.
→ The court gave Anthropic time to investigate before ruling on the challenge.
→ Anthropic’s counsel confirmed Claude.ai had been used to “properly format” at least three citations, generating a fabricated article title with mismatched authors for the one at issue.
→ The court found the underlying article was real and had been located by a human researcher, distinguishing the error from a full delegation of judgment to AI.
→ Anthropic could not explain how a “manual citation check” had missed a complete change in the article’s title.
→ A standing-order certification requiring lead trial counsel to personally verify a filing’s content had not been satisfied.
→ The court struck paragraph 9 of Chen’s declaration and counted the episode against her overall credibility as a factor in its ruling.
The Lesson
The court’s handling of the Chen declaration reads as calibrated rather than reflexive: it did not treat one fabricated citation as proof that the underlying opinion was AI-generated, because the article itself was real and had been found through ordinary human research. What the court would not excuse was the gap between that finding and Anthropic’s own representation that a manual citation check had been performed - a check that, on this record, could not have actually caught a citation whose title had been invented outright. The ruling also ties the failure to a concrete procedural requirement already in place in the case: a standing-order certification that lead trial counsel personally verified the content, which the court found had not happened. The strike itself was narrow - one paragraph, not the whole declaration - but the court still treated the fabrication as a credibility finding it weighed in reaching its conclusion.
Takeaways
If you’re retaining experts:
Have an expert’s citations checked against the actual source before a declaration is filed, and make sure whoever performs that check is confirming article titles and author lists - not just confirming that a source on a similar topic exists somewhere. A “manual” check that doesn’t catch an invented title didn’t do the job the court expected of it.
If you’re an expert witness:
If a citation-formatting tool touched any part of your declaration, disclose it and verify its output line by line before you sign. Sound underlying research didn’t insulate this declaration from a stricken paragraph - the fabricated citation was enough on its own, even though the court found no broader abdication of judgment.
If you’re opposing counsel:
A single fabricated citation can be worth raising even in a discovery dispute, not just at the Daubert stage - it can get part of a declaration struck and become a credibility finding the court weighs in the underlying ruling, without needing to prove the whole opinion was AI-generated.
