{"id":"987845d2-3aac-41d6-9adc-196e8dfe83ae","arxiv_id":"2509.01186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Prompt-based legal canons and iterative rule refinement reduce disagreement among LLM judges about whether a response complies with natural-language rules.","lead":"This paper argues that natural-language rules for AI, like constitutions, suffer from interpretive ambiguity, and shows that adding legal-style interpretive prompts or refining the rule text makes different AI judges agree more often. It matters because alignment pipelines that train models to follow natural-language rules currently lack safeguards against inconsistent interpretation, and these two interventions are simple, testable fixes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Rule-refinement entropy reductions are measured on the same single judge model and 12 ad hoc strategies used as the optimization target; central claim overreaches without external judge/human validation.","rationale":"The reader's conditional verdict is well aligned. My pass independently identifies the same load-bearing point but sharpens it: Section 7's 'panel of reasonable interpreters' is not a panel; it is one model under 12 hand-written prompts, and that exact same model/strategy set is used as the reward/selection signal and the evaluation metric. This creates an optimization-evaluation coupling that the interpretive-constraint section does not have to the same degree (there, 5 different models are used). The paper's own limitations admit the strategy set is ad hoc and the domain is only WildChat, reinforcing the external-validity gap. The diagnostic finding—that prompting with different canons changes judgments—rests on solid internal evidence (Fig. 5, A5), so REJECT would be too strong. A CONDITIONAL verdict, contingent on out-of-judge and ideally human validation, remains appropriate. Since this matches the reader's weakest assumption, no verdict adjustment is needed.","tokens_in":35463,"tokens_out":4864,"duration_ms":56996,"concrete_test":"Recompute Fig. 7's held-out entropy numbers on Stest using the five-model judge panel from §6.1 (Llama3.3-70B, Gemma2-27B, Gemma3-27B, Qwen2.5-32B, Qwen3-32B) with no strategy prompt and with the 12 strategy prompts, for the five refined rules and the original rules. If the large reductions (0.265→0.034, 0.337→0.016, etc.) shrink by more than ~50% or reverse under models not used in reward/selection, the headline improvement is an artifact of same-judge optimization rather than a property of the rules.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that both interventions 'significantly improve judgment consistency across a panel of reasonable interpreters' is only as strong as the operationalization of 'reasonable interpreters.' For the rule-refinement experiment (§7), this set is not a panel of models at all: §7.1 defines it as one judge model (Qwen2.5-32B-Instruct) prompted with 12 hand-written strategies, and §7.2's reward rjudge(c,s)=1−D_c(s) is computed from exactly this same judge/strategy set. Prompt-based refinement also selects revisions by minimizing average entropy over Sdev using this same judge. The headline reductions in Fig. 7 (e.g., Rule 53: 0.337→0.016) are therefore measured on the same model/strategy distribution that was optimized; they show that one LLM can be made self-consistent under a fixed set of 12 prompts, not that ambiguity across a panel of reasonable interpreters has been reduced. Because §5's criteria for T_reasonable (relevance, groundedness, consistency) are asserted rather than validated against humans or external judges, the metric can reflect overfitting to the judge's idiosyncrasies (e.g., the prompt-based Rule 53 revision about 'song or rhyme' in Table A13 shows meaning drift that the entropy metric misses). Interpretive-constraint experiments use 5 distinct judge models, but all receive the same strategy prompt, so part of the entropy drop may be common-prompt anchoring rather than constraint of genuine ambiguity; no human-ambiguity or held-out-model validation is provided. Given the paper's own Limitation section admits the 12 strategies are 'ad hoc' and 'reflect subjective choices,' the external validity of the measured entropy reductions is the load-bearing uncertainty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper argues that natural-language rules used in Constitutional AI are subject to interpretive ambiguity, analogous to statutory interpretation in law. It proposes two interventions: (1) interpretive constraints, i.e., prompting judge models with one of 12 legal-canon-inspired strategies to reduce disagreement during rule application, and (2) rule refinement, i.e., iteratively revising rule text to minimize entropy across a simulated set of interpreters. The framework is instantiated on 5,000 WildChat scenarios. The interpretive-constraint experiments use five open-source LLM judges and find that the no-interpretation baseline yields the highest or near-highest entropy for five selected rules. The rule-refinement experiments use Qwen2.5-32B-Instruct as judge with 12 strategies and show large entropy reductions on a held-out scenario set for five rules, though human evaluation indicates the prompt-based revisions often shift meaning.","tokens_in":35836,"tokens_out":7683,"duration_ms":80545,"significance":"If validated, the framework offers a concrete way to manage interpretive ambiguity in alignment pipelines without modifying model weights, and the legal analogy is well developed and thought-provoking. The interpretive-constraint results are cleanly executed: five distinct judge models, twelve strategies, 5,000 held-out scenarios, and bootstrap confidence intervals, with the no-interpretation baseline consistently showing highest or near-highest entropy (Fig. 5). The rule-refinement pipeline has a held-out scenario split and a human evaluation of meaning shift, which is commendable. However, the central claim for rule refinement is weakened by a circularity: the same single judge model and the same 12 strategies are used both as the optimization target (Eq. 1 in §5, selection in §7.2.1, reward in §7.2.2) and as the evaluation metric (§7.3, Fig. 7). The paper honestly reports meaning drift for prompt-based revisions, but this is not integrated into the headline claim that both interventions improve consistency 'across a panel of reasonable interpreters.' With additional validation across judge models or against human ambiguity judgments, the framework would be an important contribut","major_comments":[{"comment":"Equation (1) and the selection rule in §7.2.1, together with rjudge(c,s)=1−D_c(s) in §7.2.2, optimize rule revisions against entropy computed from Qwen2.5-32B-Instruct prompted with the same 12 strategies. Figure 7 then reports entropy on a held-out scenario set using the same judge and strategies. The held-out split demonstrates scenario generalization but not generalization across interpreters. Because T_reasonable in §5 is asserted rather than validated, the reported reductions (e.g., Rule 53: 0.337→0.005) show self-consistency of one model under a fixed prompt set. To support the abstract's claim, the authors should evaluate refined rules on the 5-model panel of §6 or on human ambiguity judgments.","section":"§7.2, §7.3, Fig. 7"},{"comment":"The narrative that selected revisions 'satisfy both lowest entropy and the majority of the annotators do not find a substantial shift in meaning' is contradicted by Table A11, which shows the prompt-based method fails the majority check for all five rules, and Table A13, where the prompt-based Rule 53 revision inserts 'song or rhyme' material. Figure 7 nevertheless presents prompt-based entropy reductions as a successful intervention. The prompt-based variant should be either excluded from the central claim or explicitly framed as a failure case requiring human review, consistent with the paper's own human evaluation.","section":"§7.3, Table A11, Table A13, Fig. 8"},{"comment":"The original text for Rule 2 is inconsistent across the manuscript. Table A3 lists 'Your response must discourage and oppose torture, slavery, cruelty, and inhuman or degrading treatment'; Figure 8 and Section 7.3 state the original was 'Your response must not promote or condone torture...' and describe the revised rule as 'discourage and oppose.' Since the refinement pipeline is defined relative to the original rule, this discrepancy affects the reproducibility and interpretation of the Rule 2 results. The authors should correct the original text and verify that all reported entropy values and revisions use the same source.","section":"§7.1, Fig. 8, §7.3, Table A3"},{"comment":"The criteria for T_reasonable (relevance, groundedness, consistency) are asserted without empirical support. The panel of 5 models is treated as an approximation of reasonable interpreters, and the 12 strategies are acknowledged in §9 as ad hoc. No evidence is provided that these models/strategies correlate with human judgments of ambiguity, nor is there a held-out set of models. This limits the external validity of both interventions; the claim 'across a panel of reasonable interpreters' should be qualified as 'across the chosen panel and strategy set.' I recommend adding a human-annotation study or a model-sweep to validate entropy as a proxy for ambiguity.","section":"§5, §6.1, §7.1"}],"minor_comments":[{"comment":"'Default learning' should be 'Default leaning.'","section":"Fig. 3 caption"},{"comment":"The phrase 'highest or near-highest entropy' is vague; for some rules the confidence intervals of the baseline and several strategies may overlap. Report explicit pairwise significance tests or state the proportion of strategies with significantly lower entropy.","section":"§6.2, Fig. 5"},{"comment":"The stopping condition compares D_{c_n}(s) with D_{c_{n+1}}(s). Clarify that c_n and c_{n+1} denote the rule texts at rounds n and n+1, and specify how the threshold ϵ=0.01 interacts with the bootstrap uncertainty in the entropy estimate.","section":"§7.2.1"},{"comment":"The 'Legal Canon' column gives names but no page/section numbers from Scalia & Garner; adding pinpoint citations would improve verifiability, given the paper's legal-audience ambitions.","section":"Table A1/A2"}],"recommendation":"major_revision","confidential_remarks":"The paper makes a timely and interdisciplinary contribution, and the interpretive-constraint experiments are solid. The rule-refinement claim, however, is currently overreaching due to the judge-model circularity and the meaning-drift results for prompt-based refinement. The internal inconsistency for Rule 2's original text must also be fixed. If the authors add external validation (e.g., evaluate refined rules on the 5-model panel or against human ambiguity ratings) and align the narrative with the human-evaluation findings, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things worth knowing. First, this is the first paper I've seen that takes canons of statutory interpretation seriously as a design pattern for LLM alignment, and the interpretive-constraint experiments are clean: five independently trained judge models, twelve hand-written legal-canon prompts, 5,000 held-out WildChat scenarios, bootstrap CIs, and the no-interpretation baseline consistently has the highest or near-highest entropy on the five most ambiguous rules. That alone is a useful empirical result and a good contribution.\n\nThe second thing is that the rule-refinement half is not in the same shape. They train a refiner (prompt-based or GRPO) to minimize entropy over a simulated panel of interpreters, but that panel is one judge model, Qwen2.5-32B, prompted with the same twelve strategies. The reward is literally 1 minus entropy over that same judge/strategy set. So the held-out entropy drops from 0.3 to 0.03 (Fig. 7) show that one LLM can be made self-consistent under a fixed set of prompts, not that a genuine panel of reasonable interpreters now agrees. The held-out scenario split is real, but it doesn't break the circularity because the judge is identical in selection, optimization, and evaluation. The paper's own limitation section admits the twelve strategies are 'ad hoc,' and the prompt-based revision of Rule 53 to include a 'song or rhyme' shows exactly the meaning drift that entropy misses. The human evaluation in Appendix A5 confirms that prompt-based revisions often fail the meaning-preservation check. The GRPO variants do better on that check, which is some evidence the approach can work, but the evaluation still needs a held-out judge or a human-ambiguity ground truth to make the 'both interventions' claim stick.\n\nThe stress-test note about common-prompt anchoring in the constraint experiments is a fair quibble but not a fatal one: all five judges receive the same strategy prompt, so part of the entropy drop may be a shared prompt effect rather than a genuine resolution of ambiguity. A quick human-ambiguity validation would settle it. That's an addressable weakness, not a dealbreaker.\n\nOverall, this is a serious paper with a real new idea and one solid experiment. The refinement half overreaches relative to its evidence. I'd send it to peer review, but I'd insist the authors either evaluate refinement with a panel of several different judge models or calibrate their entropy metric against human judgments of ambiguity. Without that, the headline should be about making a single model more self-consistent, which is still interesting but not the same as reducing ambiguity across reasonable interpreters.","headline":"Useful empirical framing, but the rule-refinement half optimizes and evaluates with the same single judge, so the 'panel' claim overreaches; the interpretive-constraint results are solid.","tokens_in":36365,"tokens_out":3040,"would_cite":true,"duration_ms":35814,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Interpretive ambiguity is a measurable, fixable source of inconsistency in AI alignment","keywords":["interpretive ambiguity","Constitutional AI","statutory interpretation","rule refinement","interpretive constraints","LLM judges","alignment consistency","entropy"],"falsifier":"Give the same rule–scenario pairs to human readers of varied legal backgrounds, or to a held-out model family never used in the panel, and compare their disagreement to the LLM entropy. If human disagreement is near zero where the model panel is highly ambiguous, or if refined rules do not reduce disagreement for the held-out judges, then the entropy proxy—and the headline reduction—does not represent interpretation generally.","tokens_in":35375,"feed_emoji":"⚖️","tokens_out":4750,"duration_ms":54553,"temperature":0.7,"pith_summary":"This paper argues that interpretive ambiguity—the fact that the same natural-language rule can be read in several defensible ways—is an underexplored source of inconsistency in Constitutional AI alignment. It borrows two mechanisms from legal systems: refining vague rules (like agencies do) and constraining judges with canon-style interpretive prompts (like courts do). The paper formalizes a space of \"reasonable interpreters\" as a panel of language-model judges and measures disagreement as the entropy of their compliance judgments. Across 5,000 real WildChat scenarios, both interventions sharply reduce disagreement: specifying one of 12 legal-canon-inspired strategies lowers average entropy on the five most ambiguous rules, and refining rule text via iterative prompting or GRPO cuts held-out entropy from roughly 0.27–0.41 down to 0.005–0.10. If the claim holds, alignment pipelines can reduce interpretive variance without changing model weights.","feed_headline":"Rewriting vague AI rules nearly eliminates judge disagreement","feed_subtitle":"On 5,000 real scenarios, entropy across five AI judges drops from ~0.4 to as low as 0.005.","key_machinery":"The central object is the \"space of reasonable interpreters\" (denoted T_reasonable), operationalized as the entropy of compliance judgments over a panel of LLM judges, or over a single judge prompted with 12 interpretive strategies. Entropy D_C(s) measures how much rule set C invites disagreement on scenario s. The two interventions are: interpretive-constraint injection, which shrinks T_reasonable by adding a canon-like prompt, and rule refinement, which rewrites C to minimize worst-case entropy, with an edit-distance regularizer preventing drift from the original text.","core_discovery":"The central discovery is that the interpretive variance of law-like AI rules can be measured and deliberately reduced. The paper treats a compliance judgment as a function of a scenario and an interpretation strategy, defines reasonable strategies by relevance, groundedness, and consistency, and simulates the space of reasonable interpreters with LLMs. On 56 rules adapted from Claude's constitution and 5,000 held-out WildChat conversations, it shows three things: with no guidance, five open-source judge models frequently disagree (20 of 56 rules lack consensus on more than half of tested scenarios); prompting the judges with one of 12 law-inspired interpretive strategies lowers average entro","pith_inferences":["If the LLM panel is a faithful proxy for real interpretive disagreement, the entropy measure could serve as a pre-deployment audit: scan a constitution's rules against diverse scenarios, flag high-entropy rules, and rewrite them before training. The paper does not test this deployment loop.","The 12 strategies come from one legal tradition; substituting other normative traditions would define a different T_reasonable and could change which rules appear ambiguous.","The paper's exploratory finding that 3–4 strategies capture most of the variance hints that the effective space of reasonable interpretations may be small, so a much cheaper panel could suffice for rule screening.","The legal analogy points to untested extensions—precedent-based interpretation and conflict resolution between multiple rules—that the current single-rule setup does not cover."],"forward_implications":["Without changing model weights, adding a canon-style interpretive prompt can move a panel of judges toward agreement on ambiguous rules; the no-interpretation baseline is not neutral.","Rule text can be treated as an optimizable object: both prompt-based iterative refinement and GRPO-based refinement reduce held-out disagreement to near zero, with human review needed to catch meaning drift.","Because the same strategy that helps one rule can hurt another, effective use of interpretive constraints requires rule-specific strategy selection.","The entropy objective gives rule designers a concrete, measurable target that mirrors administrative rulemaking: revise until reasonable interpreters converge."],"supporting_citations":[{"why":"Defines Constitutional AI, the alignment pipeline the paper diagnoses and targets; supplies the critique-and-revision steps under examination.","marker":"[8]"},{"why":"Source of the 56 rule-like principles paraphrased and used as the evaluation rule set.","marker":"[7]"},{"why":"WildChat, the real-world conversation dataset from which 5,000 held-out scenarios are sampled.","marker":"[58]"},{"why":"Source of the legal canons of statutory interpretation adapted into the 12 interpretive strategy prompts.","marker":"[43]"},{"why":"Group Relative Policy Optimization (GRPO), the policy-gradient method used to train the rule refiner.","marker":"[49]"},{"why":"Closest prior work on discretion in rule-based evaluation; the paper positions its contribution against it.","marker":"[10]"},{"why":"Collective Constitutional AI, providing examples of vague and conflicting principles used to motivate the ambiguity problem.","marker":"[27]"},{"why":"Legal scholarship on vagueness and ambiguity that grounds the rule-creation gap and the void-for-vagueness discussion.","marker":"[14]"}],"fun_headline_variants":["Legal-inspired rule edits slash AI judge disagreements","Legal canons for AI: curbing interpretive chaos","How law can make AI follow rules consistently","AI judge entropy drops to near zero with legal-inspired tweaks","Borrowing law's playbook to cut AI rule ambiguity"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that entropy computed over five open-source LLM judges—or one LLM prompted with twelve hand-written strategies—stands in for the space of \"reasonable interpreters\"; the paper does not validate this proxy against human judgments, so if the models share correlated blind spots the measured reductions may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Legal-inspired rule edits slash AI judge disagreements","Legal canons for AI: curbing interpretive chaos","How law can make AI follow rules consistently","AI judge entropy drops to near zero with legal-inspired tweaks","Borrowing law's playbook to cut AI rule ambiguity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000798,"raw_usage":{"total_tokens":3342,"prompt_tokens":736,"completion_tokens":2606,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":2544}},"tokens_in":480,"tokens_out":2606,"duration_ms":24884,"temperature":1.0,"reasoning_tokens":2544,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:47:06.365725+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give the same rule–scenario pairs to human readers of varied legal backgrounds, or to a held-out model family never used in the panel, and compare their disagreement to the LLM entropy. If human disagreement is near zero where the model panel is highly ambiguous, or if refined rules do not reduce disagreement for the held-out judges, then the entropy proxy—and the headline reduction—does not represent interpretation generally.","supporting_citations":[{"cited_title":"Claude’s constitution","cited_arxiv_id":null,"evidence_quote":"Source of the 56 rule-like principles paraphrased and used as the evaluation rule set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the legal canons of statutory interpretation adapted into the 12 interpretive strategy prompts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Group Relative Policy Optimization (GRPO), the policy-gradient method used to train the rule refiner."},{"cited_title":"Liao, Esin Durmus, Alex Tamkin, and Deep Ganguli","cited_arxiv_id":null,"evidence_quote":"Collective Constitutional AI, providing examples of vague and conflicting principles used to motivate the ambiguity problem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Legal scholarship on vagueness and ambiguity that grounds the rule-creation gap and the void-for-vagueness discussion."}],"review_version":1}