Readable copy of AI_USE.md from the public release. The original file controls.
AI use in this study
This file states who did what. AI systems appear in two different roles here: as subjects of the experiment (the drafter and judge being evaluated) and as research assistants (tools that helped the author build, run and document the study). The two roles are kept separate.
Models under study
| Role | Model | What it did |
|---|---|---|
| Drafter | Claude Opus 5.5 (claude-opus-5-5), medium effort | Drafted the RFP responses and counsel-only notes for all eight packages. |
| Judge | Claude Fable 5.1 (anthropic/claude-fable-5-1), medium effort | Wrote the initial free-form assessment of each draft. |
Neither model received the author's later adjudications or Codex's diagnostic labels. What each model actually received is established by the retained input records, not by this summary.
Who did what
| Stage | Author | AI assistance |
|---|---|---|
| Research question and experimental design | Formulated the question and made the design decisions | Claude and OpenAI assistants, including Codex, discussed options and proposed alternatives |
| Legal research | Framed the research questions, read statutes and opinions, and decided which propositions to carry forward | LexisNexis Lexis+ Protégé and Westlaw CoCounsel surfaced rules and citations |
| Principles outline | Directed its scope and content and made the decisions on what it should say | Drafting and source checking were AI-assisted |
| Fictional case and evidence documents | Specified the facts, documents and issues to be tested | Claude drafted the case documents; Codex built the paired variants (T1–T7) from the author's specifications |
| Rubric and reference answers | Specified the expected dispositions and grading dimensions | Drafted and revised with AI assistance from Claude and OpenAI assistants, including Codex; the records do not reliably separate each assistant's contribution to every revision |
| Evaluation code and runs | Specified the intended inputs for each run and authorized the spending | Claude Code and Codex wrote the Inspect runners, executed the runs, and locked and hash-verified the records. Earlier development runs did not all follow the final design; the retained records establish what each run actually received |
| Base adjudication (13 requests) | Made the final recorded rulings and severity decisions | Codex retrieved evidence, proposed interpretations and scoring options, and organized the discussion and the final review. It later added clearly attributed LSAT- and Performance-Test-informed labels, which added no penalties and changed no score |
| Citation checking | Decided which corrections to adopt | Claude and OpenAI assistants, including Codex, checked citations against available sources. This is not independent expert review |
| Publication documents (README, report, final review, methodology, reflection) | Supplied the content, ideas, findings and framing, wrote draft passages, and approved the published wording | Claude and Codex structured, edited and polished the text, and drafted supporting passages from the author's instructions and decisions |
Limitations from AI involvement
- Mixed AI authorship; tested models from one family. Materials, references and documentation were prepared with both Claude and OpenAI assistants, while both tested models, the drafter and the judge, are Claude models. Shared assumptions or blind spots could favor or disadvantage particular answers. Independent human reviewers and judges from other providers are needed to test this.
- Non-blind, AI-assisted review. The author's adjudication was non-blind and conducted through discussion with Codex. The final rulings are the author's, but the framing of each issue was shaped by that discussion.
- AI-polished documentation. The publication text was edited and polished with AI assistance from the author's content and drafts. Editing may have introduced errors the author did not catch.
None of the AI assistance above constitutes independent attorney, expert or peer review.