Pith. sign in

REVIEW 4 major objections 4 minor 32 references

NSR-Boost patches frozen legacy models with LLM-generated symbolic correction code, outperforming retraining-based baselines without altering the original pipeline.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:17 UTC pith:LS22GU6B

load-bearing objection A clever, mostly honest engineering paper with solid public benchmark results, but the headline 'significant' industrial gains are not yet backed by the evidence. the 4 major comments →

arxiv 2601.10457 v3 pith:LS22GU6B submitted 2026-01-15 cs.AI

NSR-Boost: A Neuro-Symbolic Residual Boosting Framework for Industrial Legacy Models

Ziming Dai , Dabiao Ma , Jinle Tong , Mengyuan Han , Jian Yang , Hongtao Liu , Haojun Fei , Qing Yang This is my paper
classification cs.AI
keywords neuro-symbolic AIresidual boostinglegacy model upgradetabular dataLLM code generationBayesian optimizationhard region identificationinterpretable machine learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a deployed machine-learning model can be improved without retraining or pipeline refactoring. NSR-Boost finds the regions where the frozen model errs, uses a large language model to write small, interpretable code functions that correct those errors, tunes their numeric parameters with Bayesian optimization, and merges them back through a lightweight aggregator. The result, if correct, is a safe, low-cost upgrade path for industrial tabular models: state-of-the-art accuracy, full code-level interpretability, sub-millisecond inference, and measurable improvement in live financial risk control.

Core claim

The central claim is that residual repair, guided by semantic priors from an LLM, can capture long-tail patterns missed by a strong legacy model while leaving that model untouched. Concretely, NSR-Boost trains a shallow CART tree on the legacy model's absolute residuals to carve out disjoint 'hard regions', then for each region an outer LLM loop evolves the symbolic structure of a correction function while an inner Bayesian loop refines its continuous parameters. The final prediction is produced by a trained aggregator that consumes the raw features, the legacy score, and per-expert symbolic interaction vectors. On six public datasets and a private financial dataset, the framework outperform

What carries the argument

The load-bearing mechanism is a three-stage correction pipeline: residual-guided hard-region identification (a shallow CART fitted to |residual|, whose leaves define disjoint subspaces and provide rule-based priors), bi-level expert generation (LLM-driven structure search in an outer loop, Bayesian/TPE parameter fine-tuning in an inner loop, with an incremental freezing strategy and boundary refinement), and context-aware aggregation (a lightweight gradient-boosted model over the concatenation of raw features, legacy output, and per-expert vectors [score, residual delta, ratio]). Each expert is a gated piece of Python code that activates only inside its region, so the deployed system runs as

Load-bearing premise

The entire improvement rests on the assumption that residual patterns in the training data are stable, learnable signal rather than noise, and that the shallow CART's hard-region boundaries remain valid on out-of-time data.

What would settle it

Run NSR-Boost on a task where the legacy model's training labels have been randomly permuted after training, so residuals are provably pure noise; if the pipeline still reports a positive AUC gain on an untouched test set, the gain is an artifact of the aggregation step rather than learnable residual signal.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Industrial GBDT models can be upgraded incrementally on accuracy, interpretability, and calibration without touching the existing feature schema or prediction pipeline.
  • The same frozen-model-plus-symbolic-experts recipe generalizes across backbones: gains appear when the base model is XGBoost, LightGBM, or CatBoost, indicating the mined residual signal is data-driven rather than algorithm-specific.
  • Because inference is pure Python code with sparse expert activation, the method stays under 1 ms latency and fits high-concurrency production constraints without new hardware.
  • Deploying the framework on real financial traffic reduced observed bad rates in two segments, suggesting that small AUC/KS gains translate into substantive business value at scale.
  • Pure residual baselines fail on out-of-time financial data, which the paper attributes to overfitting noise; the symbolic, region-gated experts are what make the correction transferable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework's effectiveness likely hinges on the LLM's semantic priors about feature names and business context; a natural test is to re-run the pipeline with anonymized or scrambled feature names and see whether the gains collapse, which would indicate that the method buys performance from domain knowledge rather than from general residual structure.
  • The hard-region boundaries are fixed by one shallow CART trained on training residuals; under concept drift, those boundaries may become stale, so an extension worth studying is online or periodic re-estimation of the regions without retraining the legacy model.
  • The reported +4.06e-4 AUC gain on the private OOT set, while positive, is close to the noise floor; a practical implication is that the framework's value may be concentrated in application segments where the residual signal is stronger, and practitioners should first verify residual learnability on their own data.
  • The aggregator in Eq. (4) could be applied more broadly as a general 'safety wrapper' over any black-box model, suggesting a testable framework for composing symbolic inspection with arbitrary legacy predictors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes NSR-Boost, a three-stage framework for improving a frozen legacy tabular model without retraining or pipeline refactoring. First, residuals of the legacy model are analyzed and a shallow CART tree partitions the feature space into disjoint 'hard regions' (Section 4.1). Second, for each region an LLM generates symbolic correction functions in a bi-level loop: an outer loop evolves the function structure using prompt feedback, and an inner loop uses Bayesian optimization to tune continuous parameters, including refinement of CART boundary thresholds (Section 4.2). Third, a lightweight XGBoost aggregator combines the raw features, the legacy score, and per-expert interaction vectors to produce the final prediction (Section 4.3, Eq. 4). The paper reports offline results on six public datasets and one private industrial dataset, plus online deployment results at Qfin Holdings, including AUC/KS gains and bad-rate reductions (Section 5).

Significance. The proposed direction is practically relevant: improving an already-deployed model without changing its interface, with interpretable symbolic corrections and sub-millisecond inference, would be valuable to industrial practice. The paper has several strengths: a clearly specified pipeline, ablations that isolate the contribution of each component, experiments across multiple GBDT backbones, and an analysis of LLM failure modes with different backbone models (Appendix B.4). The interpretability example in Figure 3 is a genuine illustration of the intended safety mechanism. However, the statistical evidence for the headline claims is currently weak. The private-dataset gain is at the noise floor, the online bad-rate comparison is not a controlled experiment, and the aggregator design leaves open the possibility that the gain comes from relearning on raw features rather than from symbolic residual repair. These issues are load-bearing for the central claim and need to be addressed with additional experiments and analysis.

major comments (4)
  1. [§5.2, Tables 1/3, Appendix B.5, Table 6] The claim that NSR-Boost 'significantly outperforms' baselines is not supported by the reported statistics. On the adult dataset the improvement over the base XGBoost is 0.873 to 0.875, and Table 3 reports standard deviations of ±0.002 for both; this is within one standard deviation. On the private dataset, Tables 5 and 6 report an AUC gain of only +4.06e-4 over the legacy model, with no confidence interval, bootstrap estimate, or per-seed distribution. More importantly, Table 6 shows that the residual signal on the private OOT set is extremely thin: Res-XGBoost gives -1.02e-3, gplearn gives 0.00 even after 8.3 hours, and NSR-Boost gives +4.06e-4. This does not establish a reliable residual-repair effect. The paper should report paired significance tests across seeds, bootstrap CIs, and ideally a breakdown of the OOT metric per region. Without this, 'significantly enhances' in the abstra
  2. [§5.4, Figures 5/6 and Table 4] The online bad-rate reductions (3.04% to 2.86% and 2.96% to 2.81%) are presented as evidence of real-world benefit, but the experimental design is uncontrolled. The paper states that both legacy and NSR-Boost scores are recorded on the same requests without traffic splitting. A bad-rate difference between two score systems on the same requests is only meaningful if the decision rule, cutoff policy, and any downstream business actions are held fixed; the manuscript does not specify the cutoff, the calibration procedure, or whether the NSR-Boost score actually drove decisions. Month-over-month drift and seasonality could also explain the difference. The authors should provide a randomized traffic split, or at minimum a counterfactual policy evaluation with the same cutoff strategy and confidence intervals, before claiming a significant reduction in bad rate.
  3. [§4.3, Eq. (4), and ablation in Figure 4] The aggregator M_g is an XGBoost model trained on x ⊕ [base_score, expert vectors]. Because the raw features x are included, the observed gain over the legacy model may simply reflect training a new model on the same raw features with the legacy score as an additional feature — a form of stacking rather than residual repair. The ablation in Figure 4 shows that removing raw features from the aggregator drops accuracy from 0.815 to 0.783, which suggests that the raw features contribute substantially. To support the claim that the symbolic experts are the source of improvement, the authors should include a control baseline: the same aggregator trained on x plus base_score but without any expert features, and ideally a second control with the same budget of synthetic/non-expert features. This would isolate the contribution of the symbolic residual functions from the relearning effect of the
  4. [§4.1 and §4.2] The hard-region CART boundaries are fit on training residuals and then used to define the experts, while the paper argues that these corrections transfer out-of-time (OOT). However, no analysis is provided on the stability of the CART regions under drift: feature PSI, region coverage over time, and the sensitivity of the final metric to the CART depth and priority threshold are all absent. The online OOT results in Figures 5/6 and Table 4 show positive AUC deltas, but without region-level diagnostics and without confidence intervals it is unclear whether the OOT gain is robust or concentrated in a few unstable leaves. Given the paper's own Table 6 shows the residual signal is close to the noise floor, the authors should include a stability analysis of the region boundaries and report per-region gains with bootstrap intervals.
minor comments (4)
  1. [Table 1 and Table 3] The table formatting is confusing: 'Res-XGBoost (Ours)' is used in Table 1, but this is not the proposed NSR-Boost; the NSR-Boost results are reported in the same column. The '×' entries are unexplained in the caption. Table 3 also mixes multiple values per cell without clear column headers, making it hard to compare standard deviations.
  2. [§5.1] The query budget description is ambiguous. For baselines, the paper says 'total of 20 samples' while for NSR-Boost it says 'supplies 20 data samples to the LLM during each iteration and terminates after 5 successful iterations.' These are not obviously comparable; the paper should state the total number of LLM calls used by each method and by NSR-Boost.
  3. [Appendix B.5, Table 5] The table reports average AUC improvements but no standard deviations, despite saying 20 independent runs were executed. Reporting the per-run distribution would help assess stability and would strengthen the claim that the method is model-agnostic.
  4. [§5.4] The sentence 'The online deployment processes approximately 2.4 million real user samples' in the Figure 5 caption is fine, but the text does not clarify whether metrics are computed on all decisions or only on approved applicants. This matters for interpreting the bad-rate numbers.

Circularity Check

0 steps flagged

No significant circularity: NSR-Boost's test/OOT results are external to the fitting loop and no self-citation chain is load-bearing.

full rationale

The claimed derivation chain is not circular by construction. The framework fits CART hard regions on training residuals (§4.1), fits symbolic experts on those regions using LLM generation and Bayesian optimization (§4.2), and trains the aggregator M_g on validation data (§4.3, Eq. 4). The reported results, however, are held-out test metrics (Table 1) and strictly out-of-time online deployment metrics (Figures 5-6, Table 4), which are external to that fitting loop. No equation defines the test/OOT quantity as an algebraic consequence of the fitted parameters: Eq. (1) and Eq. (2) are optimization objectives, and Eq. (4) is a model composition whose evaluation data are not the data used to fit it. The residual benchmarks in Table 6 actually provide external falsifiability: alternative residual learners fail on the same OOT set, so NSR-Boost's +4.06e-4 gain is not a tautology. There are no load-bearing self-citations; cited prior work (FunSearch, LLM-SR, CAAFE, etc.) is external and is not used as a uniqueness theorem or to import an ansatz that is itself unverified. The only substantive concern—that the aggregator's inclusion of raw features x could mean gains come from retraining rather than residual repair—is a mechanism-attribution and validity concern, not circularity, because the reported test/OOT values remain external to the fitting procedure. Therefore the paper merits a non-circular finding.

Axiom & Free-Parameter Ledger

9 free parameters · 6 axioms · 0 invented entities

The central claim rests on the assumption that residual error regions are identifiable by CART and that LLM+Bayesian search can fit transferable corrections in those regions. Several hyperparameters (K, T, N, temperature, threshold, annealing schedule, search-space bounds) are hand-set or LLM-generated; no independent evidence fixes them. No new physical entities are introduced.

free parameters (9)
  • CART hard-region priority threshold
    Chosen by hand; determines which leaves are repaired (Section 4.1).
  • CART tree depth = shallow, unspecified
    A shallow CART is used to map residual regions; depth is not precisely specified (Section 4.1).
  • Number of experts K = 5 (knee at 4)
    Set to 5; sensitivity analysis shows knee at K=4 on private data (Section 5.5).
  • LLM iterations T = 5 successful iterations
    Termination criterion for outer loop (Section 5.1).
  • Few-shot sample count N = 20
    Samples per LLM prompt (Section 5.1).
  • Sampling temperature = 0.1
    LLM sampling temperature (Section 5.1).
  • Annealing threshold tau = decaying, value unspecified
    Controls acceptance of seed-function candidates in outer loop (Section 4.2).
  • LLM-generated Bayesian search spaces = per-iteration, no fixed values
    TPE bounds are generated by the LLM; these bounds directly control fitted constants (Section 4.2, Figure 10).
  • Aggregator hyperparameters = default XGBoost
    Lightweight XGBoost M_g trained with default settings (Section 4.3).
axioms (6)
  • domain assumption Residuals in CART-identified hard regions contain learnable systematic signal rather than only noise
    Section 4.1 defines hard regions by absolute residuals; Table 6 shows purely data-driven residual learners fail (Res-XGBoost -1.02e-3, gplearn 0), so this assumption is load-bearing and fragile.
  • domain assumption Hard-region boundaries learned on training data remain valid on OOT/test data
    CART region rules are frozen into expert activation conditions; if drift shifts the error regions, experts may not activate where needed (Sections 4.2, 5.4).
  • domain assumption LLM semantic priors from feature names improve residual functions beyond numeric-only search
    The prompt design injects feature semantics/IV/PSI; ablation shows semantic removal hurts, but this is dataset-dependent and not formally guaranteed (Section 5.3, Appendix A.4).
  • domain assumption Bayesian optimization with LLM-generated search spaces converges to good coefficients
    Inner loop uses TPE with generated bounds; no convergence or coverage guarantee is provided (Section 4.2).
  • standard math Standard tabular ML assumptions (i.i.d. train/test, no leakage from validation into test)
    80/20 split and five-seed averaging are stated; private deployment is OOT but not randomized (Section 5.1).
  • domain assumption The additive residual decomposition y_hat_final = y_hat_base + F is an appropriate inductive bias
    The paper assumes corrections should be additive/logit-scale local patches; alternative global reparameterizations are excluded by design (Eq. 1).

pith-pipeline@v1.3.0-alltime-deepseek · 18589 in / 13874 out tokens · 135533 ms · 2026-08-03T10:17:03.713994+00:00 · methodology

0 comments
read the original abstract

Although the Gradient Boosted Decision Trees (GBDTs) dominate industrial tabular applications, upgrading legacy models in high-concurrency production environments still faces prohibitive retraining costs and systemic risks. To address this problem, we present NSR-Boost, a neuro-symbolic residual boosting framework designed specifically for industrial scenarios. Its core advantage lies in being ``non-intrusive''. It treats the legacy model as a frozen model and performs targeted repairs on "hard regions" where predictions fail. The framework comprises three key stages: First, finding hard regions through residuals, then generating interpretable experts by generating symbolic code structures using Large Language Model (LLM) and fine-tuning parameters using Bayesian optimization, and finally dynamically integrating experts with legacy model output through a lightweight aggregator. Experimental results demonstrate that the framework significantly outperforms state-of-the-art (SOTA) baselines across six public datasets and one private dataset. More importantly, we report the successful deployment of NSR-Boost within the core financial risk control system of Qfin Holdings, where empirical results on real-world online traffic exhibit superior performance improvements and a significant reduction in the bad rate. In conclusion, it effectively captures long-tail risks missed by traditional models and offers a safe, low-cost evolutionary paradigm for industry.

Figures

Figures reproduced from arXiv: 2601.10457 by Dabiao Ma, Haojun Fei, Hongtao Liu, Jian Yang, Jinle Tong, Mengyuan Han, Qing Yang, Ziming Dai.

Figure 1
Figure 1. Figure 1: The framework overview of NSR-Boost. determining optimal constant coefficients constitutes a continu￾ous numerical optimization problem demanding high precision, which typically exposes the limitations of token-based prediction in LLMs [22]. This decoupling effectively mitigates the combinato￾rial explosion challenge associated with mixed search spaces. Outer Loop. We formalize the outer loop as an LLM-dri… view at source ↗
Figure 3
Figure 3. Figure 3: The interpretable logic of a Symbolic Expert. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance evaluation of the Emerging Customer [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance evaluation of the Portfolio Manage [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Inference latency vs. accuracy Pareto frontier for [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: The prompt engineering framework for Symbolic Expert generation. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: The prompt engineering framework for Bayesian optimization. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Visualization of the Symbolic Expert’s decision [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Distribution of failure modes during the symbolic [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 8 linked inside Pith

  1. [1]

    Nikhil Abhyankar, Parshin Shojaee, and Chandan K. Reddy. 2025. LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers. arXiv:2503.14434 [cs.LG] https://arxiv.org/abs/2503.14434

    Pith/arXiv arXiv 2025

  2. [2]

    Sercan Ö Arik and Tomas Pfister. 2021. Tabnet: Attentive interpretable tabular learning. InProceedings of the AAAI conference on artificial intelligence, Vol. 35. 6679–6687

    2021

  3. [3]

    Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (Eds.). ACM, 785–794

    2016

  4. [4]

    DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 [cs.CL] https://arxiv.org/abs/2501. 12948

    Pith/arXiv arXiv 2025

  5. [5]

    Matthias Feurer, Jan N Van Rijn, Arlind Kadra, Pieter Gijsbers, Neeratyoy Mallik, Sahithya Ravi, Andreas Müller, Joaquin Vanschoren, and Frank Hutter. 2021. Openml-python: an extensible python api for openml.Journal of Machine Learn- ing Research22, 100 (2021), 1–5

    2021

  6. [6]

    Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. 2021. Revisiting deep learning models for tabular data.Advances in neural information processing systems34 (2021), 18932–18943

    2021

  7. [7]

    Arya Grayeli, Atharva Sehgal, Omar Costilla Reyes, Miles Cranmer, and Swarat Chaudhuri. 2024. Symbolic regression with a learned concept library.Advances in Neural Information Processing Systems37 (2024), 44678–44709

    2024

  8. [8]

    Sungwon Han, Sungkyu Park, and Seungeon Lee. 2025. Tabular Feature Discovery With Reasoning Type Exploration. arXiv:2506.20357 [cs.AI] https://arxiv.org/ abs/2506.20357

    Pith/arXiv arXiv 2025

  9. [9]

    Sungwon Han, Jinsung Yoon, Sercan O Arik, and Tomas Pfister. 2024. Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning. InInternational Conference on Machine Learning. PMLR, 17454–17479

    2024

  10. [10]

    Noah Hollmann, Samuel Müller, and Frank Hutter. 2023. Large language models for automated data science: Introducing caafe for context-aware automated feature engineering.Advances in Neural Information Processing Systems36 (2023), 44753–44775

    2023

  11. [11]

    Franziska Horn, Robert Pack, and Michael Rieger. 2019. The autofeat python library for automated feature engineering and selection. InJoint European Con- ference on Machine Learning and Knowledge Discovery in Databases. Springer, 111–120

    2019

  12. [12]

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems30 (2017)

    2017

  13. [13]

    Yinhong Li, Hanwen Qu, Chen Chen, Xiaoyi Lv, Enguang Zuo, Kui Wang, and Xulun Cai. 2025. TreeXformer: Extracting tabular feature-context information using tree-structured semantics.Information Processing & Management62, 6 (2025), 104291

    2025

  14. [14]

    Wei-Yin Loh. 2011. Classification and regression trees.Wiley interdisciplinary reviews: data mining and knowledge discovery1, 1 (2011), 14–23

    2011

  15. [15]

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Eureka: Human-Level Reward Design via Coding Large Language Models. InThe Twelfth International Conference on Learning Representations

    2024

  16. [16]

    Jaehyun Nam, Kyuyoung Kim, Seunghyuk Oh, Jihoon Tack, Jaehyung Kim, and Jinwoo Shin. 2024. Optimized feature generation for tabular data via llms with decision tree reasoning.Advances in Neural Information Processing Systems37 (2024), 92352–92380

    2024

  17. [17]

    Mohd Halim Mohd Noor and Ayokunle Olalekan Ige. 2025. A survey on state-of- the-art deep learning applications and challenges.Engineering Applications of Artificial Intelligence159 (2025), 111225

    2025

  18. [18]

    Alexander Novikov, Ngân V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. 2025. AlphaEvolve: A coding agent for scientific an...

    Pith/arXiv arXiv 2025

  19. [19]

    NVIDIA. 2025. NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model. arXiv:2508.14444 [cs.CL] https://arxiv. org/abs/2508.14444

    Pith/arXiv arXiv 2025

  20. [20]

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Doro- gush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features.Advances in neural information processing systems31 (2018)

    2018

  21. [21]

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. 2024. Mathematical discoveries from program search with large language models.Nature625, 7995 (2024), 468–475

    2024

  22. [22]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems36 (2023), 68539–68551

    2023

  23. [23]

    Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K Reddy. 2025. LLM-SR: Scientific Equation Discovery via Program- ming with Large Language Models. InThe Thirteenth International Conference on Learning Representations

    2025

  24. [24]

    Ravid Shwartz-Ziv and Amitai Armon. 2022. Tabular data: Deep learning is not all you need.Information Fusion81 (2022), 84–90

    2022

  25. [25]

    ByteDance Seed Team. 2025. Seed-OSS Open-Source Models. https://github.com/ ByteDance-Seed/seed-oss

    2025

  26. [26]

    Qwen Team. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https: //arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  27. [27]

    Joaquin Vanschoren, Jan N Van Rijn, Bernd Bischl, and Luis Torgo. 2014. OpenML: networked science in machine learning.ACM SIGKDD Explorations Newsletter 15, 2 (2014), 49–60

    2014

  28. [28]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291 [cs.AI] https://arxiv.org/ abs/2305.16291

    Pith/arXiv arXiv 2023

  29. [29]

    Runxiang Wang, Boxiao Wang, Kai Li, Yifan Zhang, and Jian Cheng. 2025. DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience. arXiv:2506.04282 [cs.LG] https://arxiv.org/abs/2506.04282

    Pith/arXiv arXiv 2025

  30. [30]

    Hengzhe Zhang, Qi Chen, Bing Xue, Wolfgang Banzhaf, and Mengjie Zhang

  31. [31]

    Personal Income Prediction Model

    Tianping Zhang, Zheyu Aqa Zhang, Zhiyuan Fan, Haoyan Luo, Fengyuan Liu, Qian Liu, Wei Cao, and Li Jian. 2023. Openfe: Automated feature generation with expert-level performance. InInternational Conference on Machine Learning. PMLR, 41880–41901. A Appendix A.1 LLM Usage Statement During the preparation of this manuscript, the authors utilized an LLM exclus...

    arXiv 2023

  32. [2025]

    arXiv:2505.18602 [cs.NE] https://arxiv.org/abs/2505.18602

    LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression. arXiv:2505.18602 [cs.NE] https://arxiv.org/abs/2505.18602

    arXiv