AI citations in a sanctioned brief

About this demo

A plaintiff's response to summary judgment in Gauthier v. Goodyear (E.D. Tex. 2024), checked citation by citation against the 12 real opinions it cites. The court's sanctions order, included as the answer key, lists the same problems.

How this demo was made

The documents are public court records. The plaintiff's response brief (Dkt. 41) and the Memorandum and Order of Nov. 25, 2024 (Dkt. 48) in Gauthier v. Goodyear Tire & Rubber Co., No. 1:23-cv-00281 (E.D. Tex.) are unchanged, from CourtListener's RECAP archive. The 12 cited opinions are the Caselaw Access Project's scans of the reporter pages. Their fax-compressed page images were re-saved with standard compression so a browser can draw them; every page renders pixel for pixel the same, and the text is unchanged. The scans' first and last pages can include the end or start of a neighboring opinion, as the printed reporter does. The order is the answer key, and Check agrees with it on every point. Footnote 1 names the two nonexistent cases, Roca and Beets; Check finds no source for them (items 4, 5 and 11). Footnote 2 names six real cases with quotations that aren't in them; Check marks all six as differing from the opinion. The order does not list Higginbotham, which Check marks as partly right: the case is real and supports a Sabine Pilot claim, but it involved hidden profit overstatements, not "testing records". Goodyear's reply (Dkt. 42) made the same point. Winters is quoted accurately, though the opinion does not address the indirect-pressure point the brief cites it for. Anderson, Reeves, Sabine Pilot and Safeshred check out. Check cannot search case law that wasn't uploaded. "No source" means none of the 12 opinions contains the cite; that Roca and Beets don't exist comes from the court's order, and for Roca from the reporter volume itself. The assessments are Claude's, made with CiteLLM Check; CiteLLM located every quotation in the opinions. Where a scan's text layer mixes a sentence with the next column, the passage is quoted in pieces. The review above is CiteLLM Check's own viewer, as it opens in ChatGPT and Claude.

The documents

• CheckedGauthier response to MSJ (Dkt. 41).pdfPDFOriginal Dkt. 41, Oct 2, 2024 (CourtListener RECAP)
• Answer keySanctions order (Dkt. 48).pdfPDFOriginal Dkt. 48, Nov 25, 2024; for comparison, not a source
• SourceAnderson v. Liberty Lobby, 477 U.S. 242.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceReeves v. Sanderson Plumbing, 530 U.S. 133.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceSabine Pilot v. Hauck, 687 S.W.2d 733.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceMorales v. SimuFlite, 132 S.W.3d 603.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceWhite v. FCI USA, 319 F.3d 672.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceWinters v. Houston Chronicle, 795 S.W.2d 723.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceHigginbotham v. Allwaste, 889 S.W.2d 411.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceSafeshred v. Martinez, 365 S.W.3d 655.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceBurton v. Freescale, 798 F.3d 222.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceTex. Dep't of Human Servs. v. Hinds, 904 S.W.2d 629.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourcePhysio GP v. Naifeh, 306 S.W.3d 886.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)
• SourceCity of Midland v. O'Bryant, 18 S.W.3d 209.pdfPDFOriginal Caselaw Access Project scan, re-compressed (see how this demo was made)