Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tree-of-Code treats each executed program as a node in a decision tree and votes among successful branches, beating CodeAct on M3ToolEval by 7.2 percentage points.

desk verdict A sensible ToT+CodeAct mashup whose central 7.2% claim is untrustworthy as presented because the comparison is confounded and the paper lacks the basics. read the letter →

arxiv 2412.14212 v1 pith:LGLWHP5Z submitted 2024-12-18 cs.SE cs.AI

Ziyi Ni , Yifan Li , Daxiang Dong This is my paper · ORCID
classification cs.SEcs.AI
keywords Tree-of-CodeLLMagentscode-as-reasoningtreesearchmajorityvotingtaskplanningM3ToolEvalCodeAct
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language model agents often act by writing and executing Python code in a step-by-step loop, an approach the paper calls CodeAct. The paper argues that this incremental repair loop is brittle on complex tasks because errors and hallucinations accumulate across rounds. Tree-of-Code instead generates end-to-end code solutions, treats each executed result as a node in a decision tree, expands failed branches using different prompts, models, and temperatures, and majority-votes over successful nodes. On M3ToolEval the paper reports 81.60% accuracy versus 74.40% for CodeAct's best mode, with average interaction turns falling from 5.5 to 2.3. The method matters because it points to a route to more stable agents that pools diverse LLMs without fine-tuning.

What carries the argument

The carrying object is the execution tree: each node is the pair of generated Python code and its execution result, rather than a thought string as in Tree-of-Thought. Reflection between nodes is modeled as the lines of the tree and comes in two forms: system-level reflection, which samples from different LLMs such as GPT-4, ERNIE-4.0-Turbo, DeepSeek Coder, and Claude 3.5, and operation-level reflection, which varies prompts, temperature, and evaluation strategies. The llm-function tool lets the code itself request an LLM-generated summary, making reasoning explicit inside executable code. The mechanism does its work by combining breadth-first exploration of diverse solution branches with execution feedback, so failed branches get retried from a new angle while successful branches accumulate candidates for a final majority vote.

What would settle it

Run ToC with a single model, such as GPT-4 only, and an execution budget equal to CodeAct's, including matched temperatures, prompt counts, and total code executions per task; if accuracy no longer exceeds CodeAct's 74.40%, the tree structure and voting are not the source of the gain.

Watch

Extended reading notes

Core claim

Tree-of-Code claims that robustness in LLM agents comes from searching over complete code executions rather than repairing code incrementally. In this framework the tree $T=(N, L)$ has nodes $N$ defined as a generated code solution paired with its execution result, and lines $L$ are reflection strategies that expand failed nodes: system-level reflection samples different LLMs, while operation-level reflection varies prompts and evaluation. A special llm-function tool lets generated code call an LLM to produce final answers, so a complete solution is produced end-to-end. Breadth-first expansion continues until all leaves execute successfully or a maximum depth is reached, and successful node outputs are collected and majority-voted. On M3ToolEval the paper reports 81.60% accuracy for ToC versus 74.40% for CodeAct's code_as_reasoning mode, 52.40% for its json_as_reasoning mode, and 53.70% for its text_as_reasoning mode, with average turns reduced from 5.5 to 2.3.

Load-bearing premise

The claimed gain rests on the assumption that the accuracy gap comes from the tree search and voting, not from ToC getting more compute and a stronger set of models than the CodeAct baseline.

Editorial extensions

If this is right

  • ToC raises accuracy on M3ToolEval to 81.60% from 74.40% for CodeAct's code reasoning mode while using 2.3 average turns instead of 5.5.
  • ToC also exceeds CodeAct's JSON and text action modes, which score 52.40% and 53.70%, so the reported advantage holds across action formats, not only code.
  • Because ToC can combine several commercial and open LLMs without fine-tuning, agent builders can pool heterogeneous models behind a single tree-search procedure.
  • Bounded depth plus majority voting keeps exploration finite and produces a single voted answer, supporting stable degradation when some branches fail.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: a controlled ablation that runs ToC with one LLM and the same total execution budget as CodeAct would determine whether the tree and voting, rather than model diversity and extra compute, are responsible for the reported gap.
  • Editorial extension: the same scheme should transfer to any executable action language beyond Python, such as shell scripts or API call sequences, making ToC a general search-over-actions planner.
  • Editorial extension: replacing one-model-one-vote with execution-derived confidence scores or a learned verifier is a natural next step the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Tree-of-Code (ToC), a method that combines Tree-of-Thought-style structured search with CodeAct-style code generation and execution for LLM-based agents. In ToC, each node is a generated code snippet together with its execution result; a breadth-first expansion explores alternative nodes by varying prompts, temperatures, and base LLMs, and a majority vote over successfully executed nodes produces the final answer. The authors evaluate ToC on the M3ToolEval benchmark and report in Table 1 that ToC achieves 81.60% accuracy with an average of 2.3 turns, compared with 74.40% accuracy and 5.5 turns for the best CodeAct (GPT-4) configuration. The paper argues that this demonstrates improved robustness and accuracy from tree-structured exploration and voting. The manuscript includes no ablations, no statistical significance measures, no Tree-of-Thought baseline, and no released code.

Significance. The idea of combining tree-structured exploration with executable code actions is a reasonable and potentially useful direction for LLM agent research. If the claimed gains were rigorously established through controlled comparisons, the method would be a meaningful contribution to the growing literature on code-as-action agents. However, the paper as written does not provide such evidence: the central comparison in Section 4, Table 1 conflates the tree/voting mechanism with a four-model ensemble, so the 7.2% accuracy gap cannot be attributed to the proposed architecture. The paper also makes an unsupported claim in the abstract about superiority over Tree-of-Thought without reporting any ToT baseline. On the positive side, the paper uses a standard benchmark (M3ToolEval) and directly compares against a CodeAct baseline, which gives the empirical claim a concrete reference point; nevertheless, the absence of error bars, compute controls, and reproducibility artifacts prevents the results from being verified. If the authors provide a same-model ablation, a ToT baseline, significance tests, and code, the contribution could become solid, but these are currently missing.

major comments (4)
  1. [Section 4, Table 1] The headline comparison is confounded: ToC is evaluated with 'Mix-modal sampling' across GPT-4, ERNIE-4.0-Turbo, DeepSeek Coder, and Claude 3.5, plus temperature and prompt variation, while the CodeAct baseline is a single GPT-4 model. The treatment and control therefore differ in at least two dimensions: the tree-search/voting mechanism and the model portfolio/inference budget. The reported 7.2% accuracy gain and the reduction in turns could be entirely due to the stronger model mix or the larger number of sampled candidates per task. Please provide a controlled ablation: ToC with GPT-4 only, and CodeAct with an equally strong multi-model ensemble, with matched inference budget and the same number of generated candidates.
  2. [Abstract and Section 4] The abstract claims that ToC provides 'more stable results compared to Tree-of-Thought,' but no Tree-of-Thought baseline appears anywhere in the experiments. Table 1 only compares CodeAct variants. Unless a ToT baseline is run on M3ToolEval and reported with variance or error bars, this claim is unsupported and should be removed.
  3. [Section 4 and NeurIPS checklist item 7] The paper reports no error bars, confidence intervals, or significance tests for the accuracy numbers in Table 1, yet the checklist answers 'Yes' to the question on statistical significance, with the justification that 'the paper reported error bars properly.' This is factually inconsistent with the manuscript. Please provide the number of evaluation tasks, standard errors or bootstrap intervals, and significance tests (e.g., paired tests across tasks) so that the 7.2% gap can be assessed.
  4. [NeurIPS checklist items 4, 5, and 8] The checklist states that code will be open-sourced and that all computational resources are specified, but the manuscript contains no code repository, no prompt templates, no model version identifiers, no branching factor, no exact voting procedure, and no token/API cost breakdown. The phrase 'generation depth was set at 3' is not a compute control. Please provide the missing reproducibility details, including the exact prompts, model APIs and dates, tree-expansion parameters, and total inference cost per method.
minor comments (5)
  1. [Section 3.2, Eq. (1)] Equation (1) writes 'Execution(i) = Code(...)' but the right-hand side appears to be code generation, not execution; the notation should be clarified, for example by distinguishing a generation function from an execution environment.
  2. [Section 3.4] The phrase 'majority vote and tragic summarization process' appears to contain a typo; presumably 'strategic summarization' or 'tragic' is a word-substitution error.
  3. [Figure 1 caption] The caption contains the typo 'CodAct' instead of 'CodeAct'.
  4. [References] Several references are duplicated: [1] and [2] are the same GPT-4 technical report, [8] and [9] are the same DeepSeek-Coder paper, and [22] and [23] appear to be the same paper in different venues. These should be consolidated.
  5. [NeurIPS checklist] Several checklist items contain boilerplate justifications that do not match the paper content; for example, item 2 answers 'Yes' that limitations are discussed, but the manuscript has no Limitations section, and item 8 claims all computational resources are specified, but none are listed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported advantage is an empirical comparison against an external benchmark, not a result that reduces by construction to its inputs.

full rationale

The paper's central claim (ToC 81.60% vs CodeAct 74.40% on M3ToolEval, Table 1) is an empirical evaluation against an externally defined benchmark. The method's components — end-to-end code generation, tree-structured exploration, and majority voting — are operational procedures, and none of the stated equations (e.g., Execution(i) = Code(FC(i), FR(i), Thought(i), Functions), Eq. 1) defines the target accuracy in terms of the method's own outputs. ToC samples from GPT-4, ERNIE-4.0-Turbo, DeepSeek Coder, and Claude 3.5 while the CodeAct baseline is a single GPT-4 model; this is a methodological confound that threatens the attribution of the gain to tree search, but it is not circularity: the comparison is not derived by construction from the chosen inputs. The checklist's claim that error bars were reported contradicts the absence of error bars in Table 1, but this is an evidentiary inconsistency rather than a circular derivation. No self-citation chain, uniqueness theorem, or ansatz-by-citation is load-bearing. Therefore no specific circular step can be exhibited, and the honest finding is score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical hyperparameter choices (tree depth, context length) and on the assumption that LLM voting, execution feedback, and a mixed-model evaluation on M3ToolEval are valid proxies for robustness and accuracy. No new theoretical entities are introduced.

free parameters (3)
  • max_tree_depth = 3
    Set at 3 for consistency in computational costs (Section 4); hand-chosen, no sensitivity analysis provided.
  • context_window = 3000 tokens
    Fixed at 3k tokens in experiments (Section 4); hand-chosen.
  • temperature_range = unspecified
    Paper states adjustments to temperatures within specified bounds (Section 4) but the bounds are not given; a hidden degree of freedom for output diversity.
assumptions (4)
  • domain assumption Majority voting over LLM-generated candidate outputs yields the correct final answer more often than single outputs.
    Adopted as the final result generator (Section 3.4) with no supporting analysis or validation.
  • domain assumption Execution success is a reliable proxy for solution quality, and reflection on execution errors improves future generations.
    Core to the tree expansion and reflection mechanism (Sections 3.1 and 3.3).
  • domain assumption The M3ToolEval benchmark is representative of complex multi-scene tasks and the reported baseline numbers from CodeAct (GPT-4) are directly comparable to ToC's mixed-model numbers.
    Used in Section 4 for the central comparison; the fairness of comparing a four-model ensemble to a single-model baseline is assumed.
  • ad hoc to paper The 'llm-function' tool, where the generator calls an LLM function to produce prompts and summarize outcomes, behaves consistently across all sampling configurations.
    Introduced in Section 3.2; no specification of its implementation or failure modes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution." pith.science (2026). https://pith.science/paper/LGLWHP5Z

@misc{pith2026241214212,
  author       = {Pith},
  title        = {Pith review of: Tree-of-Code: A Hybrid Approach for Robust Complex Task Planning and Execution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LGLWHP5Z}},
  note         = {Machine review of arXiv:2412.14212}
}
read the original abstract

The exceptional capabilities of large language models (LLMs) have substantially accelerated the rapid rise and widespread adoption of agents. Recent studies have demonstrated that generating Python code to consolidate LLM-based agents' actions into a unified action space (CodeAct) is a promising approach for developing real-world LLM agents. However, this step-by-step code generation approach often lacks consistency and robustness, leading to instability in agent applications, particularly for complex reasoning and out-of-domain tasks. In this paper, we propose a novel approach called Tree-of-Code (ToC) to tackle the challenges of complex problem planning and execution with an end-to-end mechanism. By integrating key ideas from both Tree-of-Thought and CodeAct, ToC combines their strengths to enhance solution exploration. In our framework, each final code execution result is treated as a node in the decision tree, with a breadth-first search strategy employed to explore potential solutions. The final outcome is determined through a voting mechanism based on the outputs of the nodes.

Figures

Figures reproduced from arXiv: 2412.14212 by the authors.

Figure 1
Figure 1. An Overview of our method ToC and CodAct comparisons. (a) CodeAct receives input and performs a cycle of execution and correction, but the process is carried out in an iterative, round-by-round manner. (b) ToC applies execution-level reflection in the decision-tree structure. At each layer, different nodes are executed in parallel; if executed correctly, they are stored in the candidate pool for voting, if a node fa… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents

    cs.CL 2026-08 conditional novelty 5.0 of 10

    A survey of 1,547 papers defines the 'horizon gap' and documents that long-horizon agent research is converging on trajectory-level process signals instead of outcome-only scores.

Reference graph

Works this paper leans on

40 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [3]

    Claude 3.5, 2023

    Anthropic. Claude 3.5, 2023. URL https://www.anthropic.com

    work page 2023

  3. [4]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682–17690, 2024

    2024

  4. [5]

    Unleashing the potential of prompt engineering in large language models: a comprehensive review

    Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735, 2023

    arXiv 2023

  5. [6]

    Scaling instruction-finetuned language models

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024

    2024

  6. [7]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024

    arXiv 2024

  7. [9]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024

    arXiv 2024

  8. [10]

    Econet: Effective continual pretraining of language models for event temporal reasoning

    Rujun Han, Xiang Ren, and Nanyun Peng. Econet: Effective continual pretraining of language models for event temporal reasoning. arXiv preprint arXiv:2012.15283, 2020

    arXiv 2012

Show all 40 references
  1. [11]

    L2mac: Large language model automatic com- puter for extensive code generation

    Samuel Holt, Max Ruiz Luyten, and Mihaela van der Schaar. L2mac: Large language model automatic com- puter for extensive code generation. In The Twelfth International Conference on Learning Representations, 2024

    2024

  2. [12]

    Mapcoder: Multi-agent code generation for competitive problem solving

    Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving. arXiv preprint arXiv:2405.11403, 2024. 5

    2024 arXiv

  3. [13]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023

    2023

  4. [14]

    Chatdev: Communicative agents for software development

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1...

    2024

  5. [15]

    Evaluation on agi/gpt based on the dikwp for ernie bot

    Chengxiang Ren, Yingbo Li, and Yucong Duan. Evaluation on agi/gpt based on the dikwp for ernie bot. arXiv preprint, 2023

    2023

  6. [16]

    Algorithm of thoughts: Enhancing exploration of ideas in large language models

    Bilgehan Sel, Ahmad Al-Tawaha, Vanshaj Khattar, Ruoxi Jia, and Ming Jin. Algorithm of thoughts: Enhancing exploration of ideas in large language models. arXiv preprint arXiv:2308.10379, 2023

    2023 arXiv

  7. [17]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    2024 arXiv

  8. [18]

    Reasoning about edits to feature models

    Thomas Thum, Don Batory, and Christian Kastner. Reasoning about edits to feature models. In 2009 IEEE 31st International Conference on Software Engineering, pages 254–264. IEEE, 2009

    2009

  9. [19]

    Leti: Learning to generate from textual interactions

    Xingyao Wang, Hao Peng, Reyhaneh Jabbarvand, and Heng Ji. Leti: Learning to generate from textual interactions. arXiv preprint arXiv:2305.10314, 2023

    2023 arXiv

  10. [20]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. arXiv preprint arXiv:2402.01030, 2024

    2024 arXiv

  11. [21]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  12. [22]

    Learning task decomposition to assist humans in competitive programming

    Jiaxin Wen, Ruiqi Zhong, Pei Ke, Zhihong Shao, Hongning Wang, and Minlie Huang. Learning task decomposition to assist humans in competitive programming. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

    2024

  13. [23]

    Learning task decomposition to assist humans in competitive programming

    Jiaxin Wen, Ruiqi Zhong, Pei Ke, Zhihong Shao, Hongning Wang, and Minlie Huang. Learning task decomposition to assist humans in competitive programming. arXiv preprint arXiv:2406.04604, 2024

    2024 arXiv

  14. [24]

    Wizardlm: Empowering large pre-trained language models to follow complex instructions

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations, 2024

    2024

  15. [25]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  16. [26]

    Quiet-star: Language models can teach themselves to think before speaking

    Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah D Goodman. Quiet-star: Language models can teach themselves to think before speaking. arXiv preprint arXiv:2403.09629, 2024

    2024 arXiv

  17. [27]

    [Yes] " is generally preferable to

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406.11931, 2024. 6 NeurIPS Paper Checklist The chec...

    2024 arXiv

  18. [28]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: The paper claims main contributions and scope in the abstract and introduction. Guidelines: • The answer NA means that...

  19. [29]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Yes, the paper discusses the limitations of the work. 7 Guidelines: • The answer NA means that the paper has no limitation while the answer No means ...

  20. [30]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory Assumptions and Proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: Our paper mainly works on method and experiments and do not include complete proof about the ...

  21. [31]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental Result Reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  22. [32]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: the paper will provide the ...

  23. [33]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental Setting/Details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: Yes, the paper used open sour...

  24. [34]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment Statistical Significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Yes, the paper reported error bars properly. Guide...

  25. [35]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments Compute Resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: Yes, the paper includes...

  26. [36]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code Of Ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: Yes, the research conducted in the paper conform, in every respect, with the Ne...

  27. [37]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: Yes, the paper discussed the issue. Guidelines: • The answer NA means that there is no societal impact of...

  28. [38]

    These models commonly have safeguards in their own license description

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: We mainly use o...

  29. [39]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Yes, our open so...

  30. [40]

    Guidelines: • The answer NA means that the paper does not release new assets

    New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We introduced a new method for the problem and will release our code on github if accepted. Guidelines: • The answer...

  31. [41]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and Research with Human Subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? 1...

  32. [42]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.