{"id":"aceee32c-682c-4619-a4fa-1eb39c437ead","arxiv_id":"2602.04291","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"In orchestrations of multiple LLMs, experts that are routed most often are often not the ones whose representations most influence routing decisions.","lead":"An interpretability tool called INFORM measures which experts in a multi-LLM orchestration pipeline actually matter, and finds that frequency of use is not the same as functional importance. It gives practitioners a way to see hidden dependencies in expert-routing systems beyond accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The gradient-norm definition of intrinsic importance makes the §4.5 masking validation nearly tautological: the metric and the intervention act on the same h_i, so top-gradient experts trivially disrupt routing. Functional necessity for the task itself is never measured.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing issue: local gradient sensitivity of the routing log-probability is not shown to correspond to functional necessity, and the masking validation intervenes on the same h_i used to rank the expert. My independent reading of §2.2, §4.5, and Appendix A (especially Q1 and Q8) supports this. The paper's own disclaimers weaken the 'causal' language, but the abstract and contribution sections still assert a functionally relevant distinction beyond routing mass. The divergence between routing mass and gradient sensitivity is likely real in this setup, but the strong version of the claim requires an external validation against final output quality. The proposed concrete test would settle this: if I(E_i) is not more predictive of accuracy drop than routing mass, the central claim collapses into a comparison of two routing-derived quantities. Since the paper already receives CONDITIONAL, my concern does not move the verdict; it sharpens the condition that the authors must meet.","tokens_in":68839,"tokens_out":5968,"duration_ms":69629,"concrete_test":"Run an inference-time leave-one-out ablation on the trained orchestrator for GSM8K, MMLU, and HumanEval: for each expert i, mask h_i (or exclude the expert) and measure the change in final task accuracy/Pass@1. Compute Spearman rank correlations between (a) these accuracy drops and I(E_i), and (b) the same drops and routing mass u_i. If I(E_i) does not predict accuracy drop significantly better than u_i, the claim that intrinsic importance captures functional necessity fails. Additionally, run a matched-perturbation control: add Gaussian noise to each h_i with norm equal to I(E_i) and show the top-I expert's routing KL is not merely an artifact of the perturbation direction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that routing dominance is a poor proxy for functional necessity, with I(E_i)=||∇_{h_i} log P(E_i|x)||_2 measuring intrinsic importance. But I(E_i) is exactly the first-order sensitivity of the selected-expert log-probability to the expert representation h_i, and the §4.5 masking ablation zeroes out precisely that h_i. If the selection logit is roughly linear in h_i over the mask magnitude, masking the argmax-I expert will by construction produce the largest routing KL among single-expert masks. The comparison with 'frequent peers' (Appendix J.2) only demonstrates that routing mass and routing sensitivity are different, not that either tracks functional necessity for the downstream task. The title and abstract use 'functional necessity' and 'causal importance', but the measured quantity is influence on the orchestrator's own selection distribution. The authors' FAQ Q1 disclaims formal causal structure and Q8 concedes that masking important experts often does not reduce accuracy. Thus the validation is internal to the routing policy; the missing external criterion is the effect of expert removal on final output quality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces INFORM, an interpretability framework for multi-expert LLM orchestration. The authors train a differentiable orchestrator over a fixed consortium of ten frozen instruction-tuned LLM experts (homogeneous and heterogeneous settings) and analyze three signals: the collaboration matrix C(x), the selection distribution s(x), and a gradient-based Intrinsic Expert Importance I(E_i)=||∇_{h_i} log P(E_i|x)||_2. The central claim is that routing dominance (relational importance, measured by routing mass) is a poor proxy for functional necessity: frequently selected experts can be interaction hubs with low intrinsic influence, while sparsely routed experts can be structurally critical. The paper also reports emergent dynamics (asynchronous centralization and routing confidence) and ablation results, most notably that masking the single most intrinsically important expert on MMLU induces a 5.5× larger KL divergence in the routing distribution than in the sequence distribution.","tokens_in":69110,"tokens_out":5417,"duration_ms":58112,"significance":"If the central claim were fully supported, INFORM would be a useful diagnostic for multi-expert orchestration, separating structural centrality from functional indispensability. The paper has clear strengths: it ships code, includes a secondary heterogeneous consortium, reports error bars on key numbers, and includes an explicit FAQ that acknowledges several limitations (e.g., Q1 disclaims formal causal structure, Q8 concedes that masking important experts often does not reduce accuracy). The weakness is that the validation of intrinsic importance is internal to the routing policy: both the metric and the masking intervention act on the same representation h_i, and no external task-level measure of functional necessity is supplied. The contribution is therefore presently a comparison of two routing-derived quantities rather than a demonstration that routing dominance fails to track functional necessity for the end task.","major_comments":[{"comment":"The intrinsic-importance measure I(E_i)=||∇_{h_i} log P(E_i|x)||_2 is the gradient of the selection log-probability with respect to the representation h_i, and the masking ablation in §4.5 removes exactly that representation and measures KL divergence of the routing distribution. If the selection logit is approximately linear in h_i over the mask magnitude, the top-gradient expert is by construction the one whose masking produces the largest routing shift among single-expert masks. The 'validation' therefore confirms that the metric predicts the intervention outcome because both are defined on the same variable. The paper's own FAQ Q8 concedes that masking important experts often does not reduce accuracy, and Q1 disclaims formal causal structure. To support 'functional necessity,' the authors need an external criterion—e.g., the effect of masking on final answer correctness, or a compari","section":"§2.2, §4.5, Appendix A (Q1/Q8)"},{"comment":"The headline result '5.5x higher routing KL divergence than sequencing divergence' on MMLU compares KL(sequence) = 0.428 ± 0.072 with KL(routing) = 2.366 ± 0.497. These are KL divergences over different probability spaces: the sequence distribution has 10 categories (9 degrees of freedom), while the transition matrix has 10×10 entries (90 degrees of freedom). Raw KL values are not comparable across spaces; the relative magnitude is partly an artifact of dimensionality. Report a normalized measure (e.g., average KL per free parameter, or symmetrized Jensen–Shannon with a common support) and provide the corresponding comparison for the 'frequent peer' control in Appendix J.2.","section":"§4.5/Table 8, abstract"},{"comment":"The paper's central claim is that 'routing dominance is a poor proxy for functional necessity.' However, I(E_i) measures the sensitivity of the orchestrator's selection log-probability, not the expert's contribution to the downstream task. FAQ Q2 explicitly states 'attribution scores reflect influence on routing, not expert correctness or output quality,' and Q8 states that masking important experts 'often' does not reduce accuracy. The measured divergence between routing mass and I(E_i) is therefore a divergence between two routing-derived quantities. The claim should either be reworded to 'routing dominance is a poor proxy for routing sensitivity' or supplemented by a task-level functional-necessity experiment (e.g., excluding the expert's output from the context and measuring final-answer accuracy).","section":"§4.2 and abstract"},{"comment":"The masking ablation zeros the representation h_i used by the routing adapter, but the expert's generated response remains in the conversation history that later experts receive. The intervention therefore alters only the router's internal computation, not the information available to downstream experts. Consequently, the 'functional necessity' of the expert for the final answer is not tested. Please clarify whether masking removes only h_i or also the expert's output from the context, and if only the former, refrain from calling it a functional/expert-removal ablation.","section":"§2.1, §4.5"}],"minor_comments":[{"comment":"In the version provided, the heatmap captions render as broken /uni... character sequences; if this is not a rendering artifact, the figures need to be repaired for review.","section":"Figures 4, 5, 13, 14"},{"comment":"The 95% CIs are computed across the 5 training epochs; state the number of epochs/samples used for the CI and report per-epoch values in the text or appendix.","section":"Table 8"},{"comment":"I(E_i) is defined in §2.2 but later referred to as 'intrinsic attribution,' 'gradient attribution,' and 'causal attribution'—unify the terminology.","section":"Notation throughout"},{"comment":"The claim 'INFORM explicitly does not require attention-based interaction, or oracle distillation' is not demonstrated; only the proposed attention-based orchestrator is analyzed. Add a small experiment with a different router or soften the claim.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript presents a useful diagnostic toolkit and the code release is a plus. The main gap is the mismatch between the internal routing-sensitivity validation and the 'functional necessity' framing. If the authors can add a task-level intervention (or clearly relabel the contribution), the paper would be publishable. I would also ask the editor to ensure the appendix with the 'frequent peers' statistical comparison is included in the review version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The headline claim—routing dominance is a poor proxy for functional necessity—is not supported by the experiments as run. What is supported is a weaker, still useful claim: an orchestrator's routing mass and its gradient-based sensitivity to expert representations diverge, and the divergence is task-dependent. Second, the main validation for the strong claim is nearly tautological, though the authors half-know this (FAQ Q1 disclaims formal causality; Q8 concedes masking important experts often does not reduce accuracy).\n\nWhere credit is due: the framework sensibly combines gradient attribution, interaction topology, ordering entropy, and targeted masking, applied to a trained orchestrator rather than a canned benchmark. The setup is careful—two consortia, three tasks, controlled temperature variation—and they ship code. The observation that frequently routed experts can be interaction hubs with low gradient sensitivity is new in this setting and could help practitioners spot brittle routing before accuracy drops.\n\nNow the soft spots. Intrinsic importance I(E_i) is the gradient of the selected-expert log-probability with respect to h_i, and the §4.5 masking ablation zeroes out exactly that h_i. If the selection logit is roughly linear in h_i, masking the top-gradient expert will by construction perturb the routing distribution more than masking any other expert. That is evidence of influence on the router's own policy, not of functional necessity for the task. The 5.5x KL claim on MMLU is also apples-to-oranges: a KL over a 10x10 transition matrix is not directly comparable to a KL over a 10-element sequence distribution, and the frequent-peer baseline that would make the comparison meaningful is buried in Appendix J.2. The heatmap figures in the rendering I had were unreadable, so I could not independently verify the sparsity claims quantitatively.\n\nWho this is for: people working on multi-agent interpretability or LLM routing diagnostics. They will get a plausible toolkit and an honest FAQ, but they should not cite the causal framing.\n\nBottom line: this deserves serious peer review, because the divergence result is new and the diagnostic layer could be useful, but it needs major revision. Reframe the claims as influence on routing policy rather than functional necessity, add an external criterion (final-output quality or task accuracy under expert removal), and fix the KL comparison. Conditional, not reject.","headline":"The routing-mass vs gradient-sensitivity divergence is likely a real and useful diagnostic, but the paper's 'functional necessity' framing and the masking validation that supports it do not survive close reading.","tokens_in":69603,"tokens_out":2567,"would_cite":false,"duration_ms":28820,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In multi-expert LLM systems, the most frequently routed experts are often not the ones the orchestrator actually depends on; INFORM separates relational popularity from gradient-based intrinsic necessity.","keywords":["interpretability","multi-expert orchestration","routing attribution","gradient sensitivity","LLM collaboration","expert importance","interaction structure","orchestration analysis"],"falsifier":"Find an expert ranked highly by the gradient norm, mask it, and compare against masking a same-frequency control expert: if the control produces equal or larger routing KL divergence and no smaller performance drop, the claim that gradient attribution captures causal necessity is falsified. Equivalently, if leave-one-out accuracy drops do not correlate with the gradient ranking over a random sample of inputs, the measure is not capturing functional necessity.","tokens_in":68705,"feed_emoji":"🧠","tokens_out":2846,"duration_ms":32591,"temperature":0.7,"pith_summary":"This paper argues that how often an orchestrator routes to an expert is a poor measure of how much the system needs that expert. It introduces INFORM, a probe that splits orchestration into interaction structure, sequencing, and causal attribution, and applies it to a learned router coordinating ten instruction-tuned LLMs on GSM8K, HumanEval, and MMLU. The central empirical claim is that routing mass (relational importance) and gradient sensitivity of selection logits (intrinsic importance) diverge: frequently selected experts often act as interaction hubs with little causal influence, while sparsely routed experts can be structurally critical. If correct, this means accuracy-driven evaluations of multi-expert systems miss hidden structural dependencies, and expert pruning or failure diagnosis should be guided by attribution rather than usage counts.","feed_headline":"Most-used AI experts aren't the ones that matter","feed_subtitle":"A new probe separates how often an orchestrator calls an expert from how much it depends on it—sparse experts can be critical.","key_machinery":"The central object is INFORM, an interpretability toolkit that treats the orchestrator as an explicit computation. Its load-bearing pieces are the conditional interaction matrix C(x) computed by query-key attention over expert representations (which yields relational importance as incoming mass u_j(x) = sum over i of C_ij(x)) and the selection module's marginal distribution P(E_i|x) via Gumbel-Softmax. The paper's intrinsic importance measure is the gradient norm I(E_i) = ||nabla_{h_i} log P(E_i|x)||_2, the sensitivity of selection log-probability with respect to the expert's representation. The mechanism's work is to decouple observed usage from functional necessity: one quantity tracks who","core_discovery":"The paper's central discovery is the divergence between two notions of expert importance in a sequential orchestrator. Relational importance, measured as incoming routing mass in the learned collaboration matrix, reflects how often an expert is selected as successor. Intrinsic importance, measured as the L2 norm of the gradient of selection log-probability with respect to the expert's representation, reflects how much the orchestrator's decision causally depends on that expert's content. Across MMLU, HumanEval, and GSM8K, these two measures disagree: experts with high routing mass can have low gradient attribution, and vice versa. Masking the single most intrinsically important expert produc","pith_inferences":["The gradient norm measures local sensitivity, not full counterfactual necessity; a natural next test would compare it with leave-one-out performance drops per expert across tasks.","A practical diagnostic suggested by the paper: track the gap between routing mass and gradient attribution over time; a growing gap may predict cascading failure before accuracy degrades.","Because the measure requires white-box access, an approximate version could be derived from log-probability sensitivity under input perturbation, extending the insight to API-only orchestrators.","The asynchronous emergence finding—centralization before confidence—implies early-training routing statistics can mislead interpretation of final policies, so attribution should be computed at the deployment epoch."],"forward_implications":["Expert pruning based on routing frequency risks removing structurally critical specialists while keeping interaction hubs.","Monitoring the alignment between intrinsic and relational importance can reveal brittle hubs, redundancy, and failure-propagation risks invisible to accuracy metrics.","Orchestrator ordering should be treated as a learned, non-deterministic soft constraint rather than a fixed policy, because initializer importance is task-dependent.","Targeted ablations that mask top-gradient experts provide a structural diagnostic that complements end-task accuracy.","The same gradient-attribution lens applies to any differentiable routing policy, including confidence-based cascade orchestration."],"fun_headline_variants":["Most-used AI experts often aren't essential","In AI teams, frequent experts may not be key","Sparse AI experts can be the crucial ones","AI orchestration: who's really important? Not the busy ones","Why AI's most-called experts may be the least critical"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole distinction between intrinsic and relational importance rests on the assumption that the gradient norm with respect to an expert's representation is a valid measure of functional necessity, and that the masking ablation validates it by perturbing exactly that same representation.","fun_headline_variants_meta":{"raw":{"variants":["Most-used AI experts often aren't essential","In AI teams, frequent experts may not be key","Sparse AI experts can be the crucial ones","AI orchestration: who's really important? Not the busy ones","Why AI's most-called experts may be the least critical"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000275,"raw_usage":{"total_tokens":1507,"prompt_tokens":802,"completion_tokens":705,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":628}},"tokens_in":546,"tokens_out":705,"duration_ms":7428,"temperature":1.0,"reasoning_tokens":628,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:39:49.682486+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find an expert ranked highly by the gradient norm, mask it, and compare against masking a same-frequency control expert: if the control produces equal or larger routing KL divergence and no smaller performance drop, the claim that gradient attribution captures causal necessity is falsified. Equivalently, if leave-one-out accuracy drops do not correlate with the gradient ranking over a random sample of inputs, the measure is not capturing functional necessity.","supporting_citations":[],"review_version":1}