{"id":"35706573-a8c1-409e-85ab-852d4b8b2188","arxiv_id":"2604.11061","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Pando shows that when models give no or misleading explanations, gradient-based attribution and relevance patching improve accuracy in predicting held-out model decisions by 3-5 percentage points over black-box methods, while logit lens, sparse autoencoders, and circuit tracing provide no reliable 3","lead":"The paper introduces Pando, a benchmark with 720 finetuned models implementing hidden decision-tree rules that either give faithful explanations, no explanations, or confident but unfaithful explanations of a distractor rule. It tests whether interpretability tools add value beyond black-box prompting when models do not explain themselves.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Models may not strictly implement the intended hidden decision-tree rules; training on unfaithful explanations could induce heuristics that tools recover instead of true internal computation.","rationale":"The reader's weakest assumption directly identifies the same point of fragility. Because the full paper was not available for independent verification of rule fidelity, the concern remains load-bearing but does not yet justify changing from UNVERDICTED; the proposed probe would either confirm the isolation or show that the headline effect sizes are overstated.","tokens_in":1742,"tokens_out":364,"duration_ms":43409,"concrete_test":"Construct 50 held-out inputs per model that differ from training data only on single decision-tree nodes (counterfactuals that flip the true rule output while preserving surface statistics); measure whether each model's answers match the tree on >95% of these probes across all three explanation conditions. If any condition drops below this threshold, recompute the agent accuracy tables with only the verified models.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that each of the 720 models' query decisions are generated exactly by the hidden decision tree (not surface correlations or training artifacts) so that tool outputs can be interpreted as revealing that computation. The setup finetunes on explanation targets while presumably keeping answer targets fixed to the tree, but without reported probes (e.g., rule-specific counterfactuals or ablation of tree nodes), it remains possible that models learn approximate or distractor-correlated policies, especially in the unfaithful-explanation condition. If so, the reported 3-5 pp gains from gradients/RelP and the variance decomposition (gradients track decision fields vs. other methods track task representation) could reflect recovery of learned heuristics rather than isolation of the elicitation confounder.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Pando, a controlled benchmark using 720 finetuned models that implement hidden decision-tree rules while varying explanation faithfulness (faithful, absent, or unfaithful distractor). Agents predict held-out model decisions from 10 query-response pairs, with or without one interpretability tool output. Results show black-box elicitation suffices when explanations are faithful; when absent or misleading, gradient attribution and relevance patching (RelP) yield 3-5 pp accuracy gains while logit lens, SAEs, and circuit tracing do not; variance decomposition indicates gradients track decision computation whereas other methods are dominated by task biases.","tokens_in":1923,"tokens_out":640,"duration_ms":14175,"significance":"If the models faithfully implement the target rules, the work supplies a rare controlled test of the elicitation confounder in interpretability, with large scale (720 models), variance decomposition, and full release of models/code/infrastructure. This strengthens claims about when white-box tools add value beyond prompting and provides a reusable model-organism setup for future auditing studies.","major_comments":[{"comment":"§3 (model training and faithfulness): The central claim requires that each model's decisions are generated exactly by the hidden decision tree rather than surface heuristics induced by unfaithful-explanation training. No rule-specific counterfactual probes, node ablations, or faithfulness metrics (e.g., agreement on tree-node interventions) are reported; without them the 3-5 pp gains from gradients/RelP and the computation-vs-bias decomposition could reflect recovery of learned distractor correlations instead of isolation of the elicitation confounder.","section":"§3"},{"comment":"§4.2 and Table 2 (variance decomposition): The decomposition attributes gradient gains to 'decision fields' and other methods to 'task representation/biases.' This relies on the post-hoc labeling of fields as causal; if the 10-example agent task already encodes field identity, the reported separation may be partly definitional rather than empirical evidence that gradients uniquely track computation.","section":"§4.2"},{"comment":"§4.1 (agent prediction task): The 10-example setup is intended to isolate incremental value of tool outputs. However, the paper does not report whether black-box performance saturates with more examples or whether tool benefits persist when the agent is given the full training distribution; this leaves open whether the reported gains are specific to the low-data elicitation regime or general.","section":"§4.1"}],"minor_comments":[{"comment":"Figure 3 and §4.3: The variance decomposition plots would benefit from error bars across the 720 models and explicit statistical tests for the reported differences between methods.","section":"Figure 3"},{"comment":"§2 (related work): The discussion of prior elicitation studies could more explicitly contrast Pando's explanation-axis control with existing faithfulness benchmarks.","section":"§2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their insightful comments, which help clarify the strengths and limitations of our controlled benchmark. We address each major comment in turn, providing clarifications and committing to revisions where they strengthen the evidence for our claims about the elicitation confounder.","responses":[{"response":"We agree that verifying the models implement the hidden decision trees, rather than relying on surface heuristics from the training process, is essential to isolate the effect of explanation faithfulness. Our models are fine-tuned on datasets generated exactly by the decision tree rules, with the explanation condition varied independently. To provide stronger evidence, we will incorporate rule-specific counterfactual probes and node ablation experiments into the revised §3. These will demonstrate that model outputs align with the tree structure across all explanation conditions, ensuring the 3-5 pp gains reflect recovery of the true computation rather than distractor correlations.","revision_made":"yes","referee_comment":"[§3] §3 (model training and faithfulness): The central claim requires that each model's decisions are generated exactly by the hidden decision tree rather than surface heuristics induced by unfaithful-explanation training. No rule-specific counterfactual probes, node ablations, or faithfulness metrics (e.g., agreement on tree-node interventions) are reported; without them the 3-5 pp gains from gradients/RelP and the computation-vs-bias decomposition could reflect recovery of learned distractor correlations instead of isolation of the elicitation confounder."},{"response":"The variance decomposition is performed empirically by partitioning the variance in the agent's predictions based on the known ground-truth components of the decision trees. While the labels are derived from the model structure, the separation is not definitional because the same 10-example inputs are provided to all methods, yet only the gradient-based and RelP methods exhibit substantial variance explained by the 'decision fields' component. Other methods show variance primarily in the bias terms. We will revise the text in §4.2 to emphasize this empirical differential and clarify that the decomposition tests which tools capture the causal computation.","revision_made":"partial","referee_comment":"[§4.2] §4.2 and Table 2 (variance decomposition): The decomposition attributes gradient gains to 'decision fields' and other methods to 'task representation/biases.' This relies on the post-hoc labeling of fields as causal; if the 10-example agent task already encodes field identity, the reported separation may be partly definitional rather than empirical evidence that gradients uniquely track computation."},{"response":"We chose the 10-example regime to simulate realistic low-data auditing scenarios where an auditor has limited access to model queries. We agree that reporting saturation behavior would strengthen the results. In the revision, we will add experiments in an appendix showing black-box performance with 50 examples and the full training set. These will confirm that while black-box accuracy improves with more data, the relative gains from gradient attribution and RelP persist in the unfaithful explanation conditions, supporting the relevance of our findings to practical elicitation settings.","revision_made":"yes","referee_comment":"[§4.1] §4.1 (agent prediction task): The 10-example setup is intended to isolate incremental value of tool outputs. However, the paper does not report whether black-box performance saturates with more examples or whether tool benefits persist when the agent is given the full training distribution; this leaves open whether the reported gains are specific to the low-data elicitation regime or general."}],"tokens_in":1564,"tokens_out":738,"duration_ms":30054,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that this paper builds a model-organism setup with an explicit axis for explanation quality: models learn hidden decision-tree rules but are finetuned to output faithful explanations, no explanations, or confident lies about a distractor rule. Agents then predict held-out decisions from ten query-response pairs, with or without one tool output added. When explanations are faithful, black-box prompting already does as well as anything; when they are absent or misleading, gradient attribution and relevance patching lift accuracy by a few points while logit lens, SAEs, and circuit tracing do not help reliably. They run this across 720 models and include a variance decomposition that tries to separate what each tool is actually reading.","headline":"Pando sets up a controlled benchmark to isolate whether interpretability tools add signal when models give no or false explanations, and finds modest gains only from gradients and relevance patching.","tokens_in":2400,"tokens_out":222,"would_cite":false,"duration_ms":40283,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Gradient-based attribution recovers hidden decision rules in models that give no explanations or false ones, while most other interpretability tools add no value.","keywords":["mechanistic interpretability","model organisms","explanation faithfulness","gradient attribution","relevance patching","elicitation confounder","decision trees","alignment auditing"],"falsifier":"Training a set of control models that produce the same query-response pairs but do not implement the decision trees at all, then checking whether the reported accuracy gains from gradient attribution and relevance patching disappear.","tokens_in":2663,"feed_emoji":"🔍","tokens_out":702,"duration_ms":35301,"temperature":0.7,"pith_summary":"The paper creates Pando, a benchmark of 720 finetuned models that each follow a hidden decision-tree rule but differ in how they respond to queries about their own decisions. Some models explain the true rule faithfully, others give no explanation, and others confidently describe a wrong distractor rule. Agents then try to predict the model's answers on new queries after seeing only ten examples, with or without an added interpretability-tool readout. When models explain faithfully, simply prompting on the examples works as well as or better than any white-box tool. When explanations are missing or misleading, gradient attribution raises accuracy by three to five points and relevance patching raises it further, yet logit lens, sparse autoencoders, and circuit tracing produce no reliable extra gain.","feed_headline":"Gradients recover hidden rules when models refuse to explain","feed_subtitle":"Benchmark of 720 models shows relevance patching gives the largest lift while most other tools fail to beat black-box prompting alone.","key_machinery":"The explanation axis inside Pando, which trains otherwise identical models to output faithful explanations of the true hidden decision-tree rule, no explanation at all, or confident but false explanations of a disjoint distractor rule, then measures the incremental accuracy gained by adding one tool readout to a ten-example black-box prompt.","core_discovery":"The Pando benchmark shows that white-box interpretability methods provide measurable but modest help only in the regime where models withhold or falsify their own explanations; gradient attribution and relevance patching track the actual computation that drives output decisions, whereas other readouts remain dominated by surface-level task biases and field-identity preferences.","pith_inferences":["Prior claims that interpretability tools improve model understanding may often reflect better elicitation of what the model would have said anyway rather than extraction of hidden internal structure.","Any future evaluation of interpretability methods should include an explicit no-explanation or misleading-explanation control condition to avoid the elicitation confounder.","The modest size of the gains suggests that current white-box tools are best viewed as small supplements rather than complete substitutes for black-box auditing when models are unforthcoming."],"forward_implications":["Black-box prompting alone is sufficient whenever models produce faithful explanations of their rules.","Relevance patching yields the largest accuracy lift among tested tools when explanations are absent or deceptive.","Gradient attribution outputs align with the fields that causally determine the model's decision rather than with general task representation.","Logit lens, sparse autoencoders, and circuit tracing remain dominated by biases toward field identity and value and therefore add no reliable predictive power."],"fun_headline_variants":["Gradients track hidden decisions when models omit explanations","Relevance patching outperforms elicitation for non explaining models","Pando benchmark tests if tools recover rules models refuse to share","White box tools add value only without faithful model explanations","Other methods capture biases not the driving computations"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The finetuned models actually implement the intended hidden decision-tree rules rather than using unrelated shortcuts that the ten-example prediction task cannot separate from genuine rule following.","fun_headline_variants_meta":{"raw":{"variants":["Gradients track hidden decisions when models omit explanations","Relevance patching outperforms elicitation for non explaining models","Pando benchmark tests if tools recover rules models refuse to share","White box tools add value only without faithful model explanations","Other methods capture biases not the driving computations"]},"model":"grok-4.3","cost_usd":0.010529,"raw_usage":{"total_tokens":4656,"prompt_tokens":674,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":105287000,"prompt_tokens_details":{"text_tokens":674,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3909,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":674,"tokens_out":73,"duration_ms":45046,"temperature":1.0,"reasoning_tokens":3909,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-10T15:25:08.364769+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Training a set of control models that produce the same query-response pairs but do not implement the decision trees at all, then checking whether the reported accuracy gains from gradient attribution and relevance patching disappear.","supporting_citations":[],"review_version":1}