{"id":"407ba2a0-9410-431e-a7f6-c02febe64fd4","arxiv_id":"2606.08696","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Comp-MCTS is a training-free tree-search framework that maximizes yield of unique oracle-validated counterfactuals under fixed LLM budgets on tabular datasets, outperforming LATS-style baselines in experiments.","lead":"The paper introduces Comp-MCTS, an agentic tree-search method that uses LLMs to generate multiple unique, oracle-validated counterfactual recourse options under a fixed budget of LLM calls. This addresses the practical cost of prompting LLMs while aiming to give affected individuals several feasible ways to change an unfavorable AI decision.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"LLM proposal generation + pruning may fail to yield diverse directions without per-dataset tuning","rationale":"The load-bearing concern identified is identical to the reader's weakest assumption; no stronger internal inconsistency or missing formal step was located from the provided material.","tokens_in":1763,"tokens_out":278,"duration_ms":14773,"concrete_test":"Re-execute the four-dataset experiments while logging the set of distinct feature subsets proposed by the LLM before any pruning; compute the entropy of the proposed intervention directions per run. If entropy is markedly lower on the dataset where Comp-MCTS underperforms, and if replacing the LLM proposer with uniform random feature flips closes the yield gap, the diversity assumption is the limiting factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim requires that LLM-based proposals, under fixed budget and compression-guided pruning, reliably surface sufficiently many distinct, oracle-valid intervention directions. This is least secure because LLMs can exhibit strong priors on tabular features (e.g., favoring income or credit-score edits in lending data) and prompt engineering is itself a form of implicit tuning; nothing in the method description guarantees that the same prompt template discovers novel directions across heterogeneous datasets. The reported result that favorable trade-offs appear on only three of four datasets is consistent with this assumption being dataset-dependent rather than generally reliable.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript frames counterfactual recourse generation as a fixed-budget search problem under LLM call constraints and proposes Comp-MCTS, an agentic Monte Carlo tree search framework. It allocates budget via LLM-based proposal generation for intervention directions, oracle validation of counterfactuals, and compression-guided pruning to maximize the yield of unique, oracle-validated counterfactuals while preserving quantity-quality-efficiency trade-offs. Experiments on four real-world tabular datasets are reported to show that Comp-MCTS substantially outperforms single-candidate LATS-style baselines in yield and achieves comparable or higher yield at similar or lower oracle cost than stronger multi-candidate variants on three of four datasets, with competitive proximity, sparsity, and novelty metrics.","tokens_in":1869,"tokens_out":509,"duration_ms":21145,"significance":"If the empirical results hold under the reported conditions, the work is significant for making LLM-based recourse practical when multiple alternatives are desired and LLM calls dominate cost. The training-free, oracle-only design avoids gradient access or per-dataset model fine-tuning, which is a positive attribute for deployment. The explicit focus on yield under fixed budget and the comparison across search strategies address a real operational constraint not fully covered by prior single-candidate or unconstrained methods.","major_comments":[{"comment":"Abstract: the central claim of substantial outperformance in yield of unique oracle-validated counterfactuals and favorable trade-offs on three of four datasets is load-bearing, yet the abstract (and by extension the summary of results) provides no quantitative numbers, error bars, dataset sizes, or ablation details; without these the magnitude and reliability of the reported gains cannot be assessed.","section":"Abstract"},{"comment":"Abstract and method overview: the assumption that LLM proposal generation plus compression-guided pruning will reliably surface sufficiently diverse intervention directions within the fixed budget, without domain-specific tuning, is load-bearing for the headline result. The observation that favorable trade-offs appear on only three of four datasets is consistent with dataset-dependent behavior; the manuscript should include explicit tests (e.g., prompt-template sensitivity or per-dataset direction diversity statistics) to substantiate that the method does not require implicit tuning.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract would benefit from a single sentence stating the key quantitative improvements (e.g., relative yield gain on the best dataset) to allow readers to gauge effect size immediately.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments. We address each major comment below and indicate planned revisions.","responses":[{"response":"We agree that the abstract would be strengthened by including quantitative details. The current abstract prioritizes a high-level summary within typical length limits, while the full experimental section reports yields, costs, metrics with standard deviations over runs, and dataset sizes. In revision we will add concise quantitative highlights (yield deltas, dataset sizes, error-bar references) to the abstract to make the central claims self-contained.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central claim of substantial outperformance in yield of unique oracle-validated counterfactuals and favorable trade-offs on three of four datasets is load-bearing, yet the abstract (and by extension the summary of results) provides no quantitative numbers, error bars, dataset sizes, or ablation details; without these the magnitude and reliability of the reported gains cannot be assessed."},{"response":"The experimental protocol applies identical prompts, temperature, and hyperparameters to all four datasets with no per-dataset tuning, as stated in Section 4. The three-of-four result is already reported transparently. To further substantiate robustness we will add, in the revision, a prompt-template sensitivity study on one dataset and per-dataset statistics on the number and diversity of unique intervention directions generated before pruning. These additions will directly test the claim of reliable diversity without implicit tuning.","revision_made":"yes","referee_comment":"[Abstract] Abstract and method overview: the assumption that LLM proposal generation plus compression-guided pruning will reliably surface sufficiently diverse intervention directions within the fixed budget, without domain-specific tuning, is load-bearing for the headline result. The observation that favorable trade-offs appear on only three of four datasets is consistent with dataset-dependent behavior; the manuscript should include explicit tests (e.g., prompt-template sensitivity or per-dataset direction diversity statistics) to substantiate that the method does not require implicit tuning."}],"tokens_in":1488,"tokens_out":425,"duration_ms":18496,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that this paper recasts generating multiple oracle-validated counterfactuals as a fixed-budget search task and solves it with Comp-MCTS, which allocates LLM calls via proposal generation, validation, and compression-guided pruning. That framing and the specific pruning rule are the concrete engineering moves.\n\nIt does a clean job of stating the practical constraint (LLM calls dominate cost) and of comparing against both single-candidate LATS-style methods and stronger multi-candidate variants. The claim that it improves yield while keeping proximity, sparsity, and novelty competitive on three of four real tabular datasets is the empirical core.\n\nThe soft spots are straightforward. The abstract gives no quantitative results, no error bars, no dataset sizes, and no ablation numbers, so the reported outperformance cannot be sized or checked from the summary. The fact that favorable quantity-quality-efficiency trade-offs appear on only three of four datasets already signals that the LLM proposal step may not surface diverse directions reliably across data distributions. The stress-test concern about LLM priors on features like income or credit score is therefore on target; nothing in the method description shows that the same prompt template works without implicit per-dataset adjustment. The work is training-free and oracle-only, which is a plus, but that also means any hidden sensitivity to the LLM or the oracle will show up only in the full experiments.\n\nCitation pattern is ordinary for the recourse and agentic-search literature; no circular definitions or self-referential fitting appear. The math is not the focus; the contribution is the search allocation rule.\n\nThis is for people who already work on LLM-based recourse or budgeted agentic methods on tabular data and want a practical way to increase the number of distinct valid outputs. A reader who needs reproducible numbers or broad generalization will find the current evidence thin. It still deserves a serious referee because the problem statement is clear, the baseline comparisons are relevant, and the engineering combination is new enough to warrant checking the full results and ablations.","headline":"Comp-MCTS frames budgeted counterfactual search as an agentic tree search with LLM proposals and pruning, and claims better yield than baselines on tabular data, but the abstract supplies no numbers to judge the size of the gains.","tokens_in":2337,"tokens_out":486,"would_cite":false,"duration_ms":18648,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Comp-MCTS uses agentic tree search to generate more unique oracle-validated counterfactuals under a fixed LLM-call budget than baseline methods.","keywords":["counterfactual recourse","LLM agents","tree search","fixed budget","oracle validation","tabular datasets","actionable explanations","multi-alternative recourse"],"falsifier":"Running the experiments on the four datasets and finding that Comp-MCTS does not produce a higher number of unique oracle-validated counterfactuals than the single-candidate baselines under the same LLM budget would falsify the performance claim.","tokens_in":2659,"feed_emoji":"","tokens_out":671,"duration_ms":15107,"temperature":0.7,"pith_summary":"The paper frames counterfactual recourse as a fixed-budget search problem where the goal is to maximize the number of distinct, validated feature-change sets that flip an unfavorable model decision. It introduces Comp-MCTS, which uses LLM proposals for new directions, validates them with an oracle, and prunes via compression to allocate the budget efficiently without training or gradients. A sympathetic reader would care because real-world users need multiple practical alternatives rather than one explanation, and each LLM prompt costs money or compute. Experiments on four tabular datasets demonstrate higher yields and good trade-offs in quantity, quality, and efficiency.","feed_headline":"Tree search yields more unique recourse options per LLM call","feed_subtitle":"Comp-MCTS produces higher numbers of validated counterfactuals than baselines on tabular data without training or gradients.","key_machinery":"Comp-MCTS, an agentic tree-search framework that allocates budget to novel intervention directions using LLM proposals, oracle checks, and compression-guided pruning.","core_discovery":"In the LLM-agentic setting, counterfactual recourse generation is treated as a fixed-budget search problem. Comp-MCTS is proposed as an agentic tree-search framework that maximizes the yield of unique, oracle-validated counterfactuals through LLM-based proposal generation, oracle validation, and compression-guided pruning in a training-free, oracle-only setting. On four real-world tabular datasets, it substantially outperforms single-candidate LATS-style baselines in yield and provides favorable quantity-quality-efficiency trade-offs against stronger multi-candidate variants on three of four datasets.","pith_inferences":["This approach could be tested on image or text data to see if the pruning mechanism generalizes beyond tabular features.","If successful, it suggests that search-based methods can reduce the effective cost of LLM usage in explanation tasks across domains.","The method implies that diversity in recourse can be achieved by guiding the search tree rather than generating many independent prompts."],"forward_implications":["Generating multiple alternatives becomes feasible within the same LLM budget as single explanations.","No need for model gradients or domain-specific tuning to achieve diverse recourse options.","Better performance in yield of unique counterfactuals on tabular data compared to LATS-style methods.","Comparable or superior efficiency in oracle evaluations while maintaining proximity, sparsity, and novelty metrics."],"fun_headline_variants":["Comp-MCTS tree search yields more unique validated counterfactuals","Agentic MCTS increases unique oracle-validated recourse options","Fixed LLM budget search produces more unique counterfactuals","Tree search maximizes unique recourse yield under call limits","Comp-MCTS outperforms baselines in validated counterfactual variety"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"LLM-based proposal generation combined with compression-guided pruning reliably discovers sufficiently diverse intervention directions within the fixed budget without domain-specific tuning or access to model gradients.","fun_headline_variants_meta":{"raw":{"variants":["Comp-MCTS tree search yields more unique validated counterfactuals","Agentic MCTS increases unique oracle-validated recourse options","Fixed LLM budget search produces more unique counterfactuals","Tree search maximizes unique recourse yield under call limits","Comp-MCTS outperforms baselines in validated counterfactual variety"]},"model":"grok-4.3","cost_usd":0.006967,"raw_usage":{"total_tokens":3259,"prompt_tokens":729,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":69674500,"prompt_tokens_details":{"text_tokens":729,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2458,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":729,"tokens_out":72,"duration_ms":17129,"temperature":1.0,"reasoning_tokens":2458,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T18:29:11.891539+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the experiments on the four datasets and finding that Comp-MCTS does not produce a higher number of unique oracle-validated counterfactuals than the single-candidate baselines under the same LLM budget would falsify the performance claim.","supporting_citations":[],"review_version":1}