{"id":"18b04760-9b9a-454c-86c6-8b3f4bf63112","arxiv_id":"2501.01306","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"HaluSearch reduces LLM hallucinations by generating responses through MCTS-based tree search with a reward model, outperforming CoT, self-consistency, and best-of-N baselines.","lead":"A framework called HaluSearch uses tree search during text generation to let an LLM explore multiple sentence-by-sentence paths and pick the most reliable one, with a fast/slow thinking switch to save time on easy questions. The authors report accuracy gains over standard baselines on six factuality benchmarks in English and Chinese.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GPT-4 serves as both the reward model guiding MCTS and the judge of correctness, so the reported gains may reflect judge alignment; the full HaluSearch system is also never directly compared in Table 1.","rationale":"The reader's verdict is REJECT with high confidence, and the rationale already flags the GPT-4-as-reward-and-judge issue. However, the reader's stated weakest assumption is the internal-knowledge availability, which is a real but secondary concern: even if the policy knows the facts, the benchmark numbers could be inflated by judge alignment, and the SimpleQA results (3-8.5% for Llama3.1) already show the method does not overcome missing knowledge. My most load-bearing concern is different: the empirical comparison is structurally confounded because the reward model and the correctness judge are the same model, and the full HaluSearch system with the switch is never compared against baselines in Table 1. The MCTS construction is coherent, the training of self-evaluation reward models is a reasonable attempt to reduce closed-source dependency, and the hyperparameter analysis is useful. But these strengths do not resolve the evaluation confound. The concrete test of using an independent judge and adding the full HaluSearch row would settle whether the central claim holds; without it, the reported superiority cannot be trusted as evidence of reduced hallucination rather than alignment with GPT-4's preferences. Since the reader's verdict already reflects this concern, I recommend no change to the verdict.","tokens_in":13920,"tokens_out":5545,"duration_ms":54513,"concrete_test":"Run an independent evaluation: (1) On a 200-item stratified sample from TruthfulQA and SimpleQA, have human annotators (or a different LLM such as Claude or Gemini) judge correctness, and recompute Table 1 accuracies for all methods; if the HaluSearch/MCTS margin over BoN/SC shrinks or reverses, the claimed gains are partly an artifact of GPT-4 judge/reward alignment. (2) Add a row for the full HaluSearch (MCTSwitch) pipeline to Table 1, using the same GPT-4 reward model and validation-selected thresholds, and verify that it actually beats the MCTS-only row; also reconcile Figure 2's 42.6% (threshold 0) with Table 1's 45.4% MCTS value on HaluEval-QA with Llama3.1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that HaluSearch \"achieves the best performance across all six Chinese and English datasets\" (Section 4.2) rests on an evaluation loop that is not independent of the method's optimization target. In the main experiments, GPT-4 is used as the reward model guiding MCTS (Section 4.1, Implementation Details), and GPT-4 is also used to judge correctness of all generated answers (Section 4.1, Datasets and Metrics). HaluSearch's tree search explicitly selects responses that maximize GPT-4's hallucination scores, so the reported accuracy is measured by the same model whose preferences the search is optimizing. Baselines such as CoT, self-consistency, and Best-of-N do not receive this GPT-4 steering, making the comparison asymmetric. The trained self-evaluation reward models in Section 4.3 are distilled from GPT-4 labels, so they inherit the same bias. Additionally, Table 1's best row is labeled \"MCTS,\" not the full HaluSearch pipeline with the dynamic switch; the switch is evaluated only on HaluEval-QA in Section 4.4 with thresholds chosen on the test set. Notably, Figure 2 reports 42.6% for 100% slow thinking on HaluEval-QA with Llama3.1, lower than the 45.4% MCTS row in Table 1, so even the internal numbers do not establish that the full system is best. The headline claim is therefore not directly supported by the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HaluSearch, an inference-time framework that uses Monte Carlo Tree Search (MCTS) to generate responses sentence by sentence, with a reward model scoring each step and a trained switch model deciding between fast (System 1) and slow (System 2) thinking at both instance and step levels. The self-evaluation reward model is trained on GPT-4-generated scores and critiques, and the full method is referred to as MCTSwitch. The authors evaluate on six English and Chinese hallucination datasets with Llama3.1-8B-Instruct and Qwen2-7B-Instruct as policy models, reporting that MCTS outperforms direct generation, Chain-of-Thought, Self-Consistency, Best-of-N, Self-Refine, and ITI in Table 1. The paper also analyzes trained reward models and the efficiency-accuracy trade-off of the switch mechanism, and concludes that HaluSearch significantly mitigates hallucinations.","tokens_in":14192,"tokens_out":5620,"duration_ms":57005,"significance":"The idea of applying step-level tree search to hallucination mitigation is timely and the framework is clearly described: the MCTS formulation, the reward model training procedure, and the switch mechanism are all specified in sufficient detail to be reproduced. If the empirical claim is established with an evaluation that is independent of the optimization signal, the paper would be a useful contribution to inference-time hallucination mitigation and to the growing literature on System 2 reasoning in LLMs. The paper also provides concrete ablations of reward model variants (Generative RM and Generative RM + Critic) and of the switch threshold, which are informative. However, the current experimental validation has a central weakness: the same model, GPT-4, is used both as the reward signal that guides the search and as the judge that scores the final answers, so the headline gains in Table 1 may partly reflect reward hacking rather than genuine factuality improvements.","major_comments":[{"comment":"The evaluation is not independent of the optimization target. In Implementation Details, GPT-4 is used as the reward model inside MCTS, and in Datasets and Metrics, GPT-4 is also used to judge the correctness of every generated response. Since the search explicitly selects sentences and rollouts that maximize GPT-4's hallucination score, the reported accuracy is measured by the same model whose preferences are being optimized. Baselines such as CoT, Self-Consistency, and Best-of-N do not receive this GPT-4 steering, so the comparison is asymmetric. The trained self-evaluation reward models in Section 4.3 are distilled from GPT-4 labels and therefore inherit the same bias. Please re-evaluate with a judge that is not used anywhere in reward training or search, or with human evaluation, and report the results. Without this, the claim in Section 4.2 that HaluSearch 'achieves the best performance across all six Chinese and English datasets' is not supported.","section":"§4.1, Section 3.2, Table 1"},{"comment":"The main table reports a row labeled 'MCTS', not the full HaluSearch pipeline with the dynamic switch (MCTSwitch). The switch mechanism is a stated contribution and is evaluated only on HaluEval-QA in Section 4.4, with a threshold-based analysis. Moreover, the internal numbers are inconsistent: Figure 2 reports 42.6% accuracy for 100% slow thinking on HaluEval-QA with Llama3.1-8B-Instruct, which is lower than the 45.4% MCTS row in Table 1 for the same policy model, dataset, and reward setup. This discrepancy needs to be explained, and the full MCTSwitch system should be compared against the baselines on all six datasets, or the paper should explicitly state which configuration corresponds to the headline claim.","section":"§4.2, Table 1, §4.4, Figure 2"},{"comment":"The switch threshold analysis appears to be conducted on the same dataset used to create the switch training data. The text says 10K training data are collected from HaluEval-QA and TruthfulQA, categorized by thresholds, and then 'evaluate their performance on the HaluEval-QA dataset', without specifying a held-out split. If the switch model or the threshold γ is selected using test-set results, Figure 2 overstates the achievable trade-off. Please clarify the data split, and if thresholds are tuned, report validation-based selection or cross-validation.","section":"§4.4, Algorithm 1"},{"comment":"The reward model prompt template includes the line 'Correct Answer: (Only provided when generating reward data.)'. This is reassuring, but the surrounding text in Section 3.3 says the ground truth answer is included as a reference 'for evaluation'. Please state explicitly that the reward model does not receive the ground truth when scoring MCTS rollouts at test time. If the correct answer is available to the reward model during search, the reported accuracy gains would be an artifact of test-set leakage.","section":"§3.3, prompt template"}],"minor_comments":[{"comment":"The acronym 'UTC' should be 'UCT'; it appears in the sentence introducing Equation (1).","section":"§3.2"},{"comment":"The table caption says 'on HaluEval-QA dataset' but the columns include TruthfulQA and SimpleQA; please correct the caption to describe the three English datasets.","section":"Table 2"},{"comment":"Reference [44] is cited for ChineseFactEval, but the cited paper appears to be about Factool, a factuality detection framework; please verify that this is the correct source for the ChineseFactEval benchmark.","section":"References"},{"comment":"The tables report only point estimates. Given that several differences between MCTS and Best-of-N are small (e.g., ChineseSimpleQA with Qwen2: 36.00 vs 35.00), please report variance or significance tests over multiple runs to establish that the improvements are reliable.","section":"§4.1, Table 1"},{"comment":"Please add axis labels and units to Figure 2, and specify how 'average solving time per question' is measured (e.g., wall-clock time on which hardware).","section":"Figure 2"},{"comment":"The heading 'Halluciantion Mitigation' contains a typo; it should be 'Hallucination Mitigation'.","section":"Section 2.1 heading"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the overlap between the reward model and the judge, which makes the central empirical claim unsubstantiated as written. However, the framework is well specified and the fix is feasible: re-run the evaluation with an independent judge or human evaluation, and compare the full MCTSwitch pipeline, not just the MCTS component. I would not reject outright because the flaw is an experimental design issue rather than an unfixable methodological error. If the independent evaluation reverses the result, the contribution would be substantially weakened, so the revision must include those results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth a look: applying MCTS with sentence-level rewards to hallucination mitigation is a reasonable engineering move, and the fast/slow switch is a sensible efficiency hack. The critique-based reward model is a nice touch, and the evaluation is broad—six datasets, two policy models, English and Chinese. If you work on inference-time decoding, the recipe is worth knowing about.\n\nThat said, the central claim—that HaluSearch is best on all six datasets—is not supported by the reported experiments. The main issue is the one you'd expect: GPT-4 is doing double duty as the reward model inside the search and as the judge scoring the final answers. The baselines don't get that steering, so the comparison is asymmetric and part of the gain is likely reward hacking. The self-trained reward models are distilled from GPT-4 labels, so they inherit the same bias.\n\nThe second issue is just as serious. Table 1's best row is labeled \"MCTS,\" not the full HaluSearch/MCTSwitch system. The switch is only evaluated on HaluEval-QA, with thresholds chosen on the test set, and the reported accuracy there (42.6% at 100% slow thinking with a trained reward model) is below the GPT-4-guided MCTS number (45.4%). So we never actually see the full pipeline against the baselines. The internal numbers don't add up to the headline.\n\nMinor point: the hyperparameter analysis is single-dataset, which is fine for a sanity check, but it doesn't bolster the generality claim.\n\nWho is this paper for? Practitioners who want a concrete inference-time hallucination mitigation recipe will find useful ideas here. But as a claim of state-of-the-art, it needs rework.\n\nMy recommendation: send it to peer review, but with a strong request—use an independent judge (human or a different LLM) for at least a subset of the evaluation, and report the full HaluSearch system on all datasets with thresholds selected on a validation set, not the test set. This is a serious enough contribution to deserve referee time; it just doesn't support its own headline yet.","headline":"A useful engineering idea undercut by a circular evaluation and a missing full-system comparison; the headline claim is not supported as stated.","tokens_in":14789,"tokens_out":2925,"would_cite":true,"duration_ms":29612,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HaluSearch turns answer generation into a sentence-scored tree search, and reports the best accuracy among compared methods on all six factuality datasets.","keywords":["hallucination mitigation","Monte Carlo tree search","inference-time reasoning","System 2 thinking","reward model","self-evaluation","factuality benchmarks","dual process theory"],"falsifier":"Take a factuality benchmark and split it by whether the policy model can answer a question when given a one-sentence hint: the paper's internal-knowledge story predicts HaluSearch's accuracy gain over direct generation is concentrated in questions the model can answer with a hint but not without. If the gain is just as large on hint-unanswerable questions, the search is not simply exploiting latent knowledge, and the central premise fails.","tokens_in":13667,"feed_emoji":"🧠","tokens_out":8803,"duration_ms":82620,"temperature":0.7,"pith_summary":"Hallucinations in large language models often persist even when the model knows the correct fact; the paper argues this is because ordinary decoding moves too fast and lets one bad sentence contaminate the rest. HaluSearch treats response generation as a step-by-step reasoning process: at each sentence, a Monte Carlo tree search expands several candidate continuations, a reward model scores how likely each continuation is to hallucinate, and the search advances along the most reliable path. To keep this affordable, a learned switch decides when a question or a single step needs slow thinking and when fast direct generation is enough. If the central claim is right, the same model can become more truthful at inference time without retraining, and users can dial the compute-versus-accuracy trade-off. The paper reports gains over prompt-based and inference-time baselines on all six English and Chinese factuality datasets.","feed_headline":"Tree search that scores each sentence cuts LLM hallucinations","feed_subtitle":"Sentence-level search lets an LLM exploit facts it already knows, beating prompt-only fixes on six factuality benchmarks.","key_machinery":"The central object is a Monte Carlo tree in which each node is one generated sentence and each child is an alternative next sentence sampled from the policy model. Selection uses the UCT score, which balances a node's accumulated value against how rarely it has been visited; expansion samples candidate continuations; evaluation completes each candidate with rollouts and averages reward-model scores over them; backpropagation updates ancestor values. The reward model is trained on the same base model, scoring hallucination likelihood on a 1-to-5 scale, with a variant that produces a critique before the score. A separate switch model, trained on labels derived from the search tree's own node values, governs when to think fast and generate one sentence directly versus think slow and expand multiple candidates, at both the level of the whole question and the level of each step.","core_discovery":"On the paper's own terms, hallucination mitigation is better posed as a search problem than as a prompting problem. HaluSearch frames text generation not as one pass from prompt to answer but as a tree of possible sentence-level continuations, with every node scored by a reward model trained to predict how likely a continuation is to hallucinate. The claim is that this step-level search fully exploits the policy model's internal knowledge: because the model often knows facts it fails to state, branching and scoring give the reliable path a chance to surface. In the reported experiments, HaluSearch outperforms direct generation, chain-of-thought prompting, self-consistency, best-of-N, self-refinement, and activation shifting on all six English and Chinese datasets. A self-trained reward model that produces a critique before a score is shown to be competitive with a much larger external reward model, supporting the paper's case that no closed-source scorer is required. The intended conclusion is that deliberate, search-based decoding can remove a substantial share of hallucinations at inference time.","pith_inferences":["One consequence the authors leave implicit: the same sentence-level search should transfer to long-form generation, since any multi-sentence passage can be decomposed and scored the same way.","The paper reports accuracy and time separately; a direct cost-matched comparison against Best-of-N at equal total generated tokens would show whether the advantage comes from the search structure itself or from the larger number of generated candidates, an open question.","A natural untested extension is to close the loop by retraining the reward and switch models on data produced by HaluSearch itself, which could compound the gains shown in the single training round reported.","The dual-process design suggests a general resource-allocation principle for inference-time compute: spend extra computation where the model's own uncertainty is high and skip it elsewhere, a principle that could be implemented in decoding algorithms other than tree search."],"forward_implications":["Because rewards are assigned per sentence, a wrong intermediate step can be detected and abandoned before it contaminates the rest of the response, reducing error accumulation.","The trained critique-based reward model makes the approach deployable without a closed-source scorer, since it matches a larger external reward model on TruthfulQA.","The switch threshold gives an explicit accuracy-efficiency dial: at the highest threshold tested, average solving time per question falls from 53.3 to 25.4 seconds while accuracy drops from 42.6% to 37.6% on HaluEval-QA.","The gains hold across languages and model families: both Llama3.1-8B-Instruct and Qwen2-7B-Instruct improve on all six English and Chinese benchmarks.","Scaling the number of expanded nodes and rollouts improves accuracy up to a point, after which the policy model's internal knowledge and the reward model's scoring accuracy become the limiting factors."],"supporting_citations":[{"why":"Supplies the UCT selection rule that balances exploiting high-scored nodes against exploring rarely visited ones, the core of the tree search.","marker":"[35]"},{"why":"Provides evidence that hallucinations arise from ineffective generation and supplies the Self-Consistency baseline that HaluSearch must beat.","marker":"[19]"},{"why":"Shows that LLMs know more than they show, the premise that search over a model's own generations can recover facts it fails to state.","marker":"[33]"},{"why":"Gives the reward-guided tree-search recipe that HaluSearch adapts from reasoning tasks to sentence-level hallucination scoring.","marker":"[34]"},{"why":"Chain-of-thought prompting is the main prompt-based reasoning baseline compared in the experiments.","marker":"[18]"},{"why":"Supplies the Best-of-N baseline, the strongest response-level selection method HaluSearch is compared against.","marker":"[26]"},{"why":"HaluEval-QA is the primary English dataset used for reward-model training and for the main hallucination evaluation.","marker":"[39]"},{"why":"TruthfulQA is the English factuality benchmark used to test whether the method improves truthfulness beyond mimicry of falsehoods.","marker":"[40]"}],"fun_headline_variants":["Search beats prompting: tree-scored steps cut LLM hallucinations","MCTS decoding: score each step to tap LLM's known facts","Fast and slow thinking: tree search for less LLM hallucination","HaluSearch: decode by search, not by single pass, to reduce errors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The policy model already holds the facts needed to answer the test questions, so that exploring its own generations can surface the correct path; if the knowledge is absent, tree search cannot create it.","fun_headline_variants_meta":{"raw":{"variants":["Search beats prompting: tree-scored steps cut LLM hallucinations","MCTS decoding: score each step to tap LLM's known facts","Fast and slow thinking: tree search for less LLM hallucination","HaluSearch: decode by search, not by single pass, to reduce errors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000474,"raw_usage":{"total_tokens":2352,"prompt_tokens":942,"completion_tokens":1410,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":1332}},"tokens_in":558,"tokens_out":1410,"duration_ms":11220,"temperature":1.0,"reasoning_tokens":1332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:29:52.049258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a factuality benchmark and split it by whether the policy model can answer a question when given a one-sentence hint: the paper's internal-knowledge story predicts HaluSearch's accuracy gain over direct generation is concentrated in questions the model can answer with a hint but not without. If the gain is just as large on hint-unanswerable questions, the search is not simply exploiting latent knowledge, and the central premise fails.","supporting_citations":[{"cited_title":"Bandit based monte-carlo planning","cited_arxiv_id":null,"evidence_quote":"Supplies the UCT selection rule that balances exploiting high-scored nodes against exploring rarely visited ones, the core of the tree search."},{"cited_title":"Le, Ed H","cited_arxiv_id":null,"evidence_quote":"Provides evidence that hallucinations arise from ineffective generation and supplies the Self-Consistency baseline that HaluSearch must beat."},{"cited_title":"Halueval: A large-scale hallucination evaluation benchmark for large language models","cited_arxiv_id":null,"evidence_quote":"HaluEval-QA is the primary English dataset used for reward-model training and for the main hallucination evaluation."},{"cited_title":"Truthfulqa: Measuring how models mimic human falsehoods","cited_arxiv_id":null,"evidence_quote":"TruthfulQA is the English factuality benchmark used to test whether the method improves truthfulness beyond mimicry of falsehoods."}],"review_version":1}