{"id":"d682ed74-e741-48bb-9baa-822079b158e5","arxiv_id":"2607.27389","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LLM-guided evolutionary search over input-feature programs improves learning-to-optimize pipelines across LP, QP, and MILP tasks, outperforming fixed hand-crafted features in most evaluated settings.","lead":"This paper presents a system that automatically designs the input features used by machine-learning models that guide optimization solvers, by having a language model propose candidate feature programs and selecting them based on measured solver improvements. It demonstrates gains on linear, quadratic, and mixed-integer optimization tasks without changing the underlying learned model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's claim that evolved features 'consistently outperform' across four LLMs is contradicted by the paper's own per-model tables: DeepSeek improves only 6 of 20 shared cells and none of the FSNet tasks, so the headline overstates the evidence.","rationale":"The paper's core framework is well designed: it uses a FunSearch-style loop, semantic contracts, validation-based selection, disjoint test reporting, seed-paired repetitions, and matched-width random controls. These elements make the central contribution plausible. However, the strongest claim is broader than the evidence. The paper's own per-model tables show that DeepSeek-V2—one of the four LLMs explicitly invoked in the abstract—improves only 6 of 20 continuous cells and none of the FSNet or DC3 tasks. The abstract's 'consistently outperform' is therefore not a fair summary: it holds for Claude, GPT-5.5, and Gemini, but not for DeepSeek. The reader's identified weakest assumption—that input-layer resizing is mechanically neutral—is also a legitimate threat to isolating feature quality, especially because matched-width controls exist for only a subset of hosts. Yet that concern is partially addressed by the available controls and is secondary to the direct mismatch between the headline and the tabulated per-LLM results. A conditional verdict remains appropriate: the paper should be accepted only after the abstract and conclusion are tempered to reflect proposal-model dependence, and ideally after the source code is released. My recommendation is UNCHANGED because the reader already assigned CONDITIONAL and cited the overclaim as a reason; this stress-test reinforces that reason rather than altering the verdict.","tokens_in":17652,"tokens_out":6222,"duration_ms":66990,"concrete_test":"From Tables 7–11, enumerate all 20 shared continuous cells per proposal model and count '✓' vs '='; specifically verify DeepSeek-V2's FSNet and DC3 columns. If DeepSeek improves 6/20 and 0/15 FSNet+DC3 cells, then revise the abstract and conclusion to attribute 'consistently outperform' only to the top three LLMs; otherwise the original wording may stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract—'Across continuous and discrete optimization tasks and four LLMs, the evolved features consistently outperform hand-crafted representations'—is not supported by the paper's own per-model results. In the Proposal-model dependence paragraph, the authors report that Claude, GPT-5.5, Gemini, and DeepSeek improve 15, 17, 17, and 6 cells, respectively. The appendix Tables 7–11 allow a direct check: across the 20 continuous cells shared by all four LLMs, DeepSeek-V2 improves 6 and ties 14; it improves none of the 12 FSNet tasks and none of the 3 DC3 tasks. Thus the 'across four LLMs... consistently outperform' claim is true only for the three stronger proposal models; the fourth model mostly returns the handcrafted seed. The conclusion's 'evolved features improve ... without changing the underlying models' is also too broad: in the DeepSeek runs, the evolved features are often identical in effect to handcrafted. The reader's resize concern is real but secondary: matched-width random controls cover only six LP/PDHG settings, so for the other hosts the 'mechanical interface change' is not directly controlled. However, even granting that resize is innocuous, the per-LLM evidence undercuts the headline universality. The paper remains valuable as a framework; the claim should be narrowed to 'FunL2O with capable LLM proposers reliably improves L2O features, while weaker proposers often preserve the seed.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"FunL2O proposes an LLM-driven program-evolution framework that searches over executable feature functions for learning-to-optimize (L2O) pipelines. The feature function is replaced while the model architecture, training procedure, data, solver, and downstream evaluation metric remain fixed; each candidate is validated by a semantic contract, the host pipeline is retrained, and the downstream optimization outcome is used as the fitness signal. The method is evaluated on eight L2O pipelines spanning LP solution prediction, constrained QP/QCQP/SOCP prediction, primal-dual warm-starting, simplex basis and pivot decisions, and MILP Predict-and-Search and learned-backdoor branching, using up to four LLM proposers. The paper claims that evolved features consistently outperform handcrafted representations across all tasks and LLMs, and that the gains are attributable to feature content rather than width expansion or best-of-many selection, supported by equal-budget and matched-width control experiments.","tokens_in":18062,"tokens_out":5527,"duration_ms":57293,"significance":"If the central claim is properly supported, the paper makes a valuable contribution: it introduces a general and automatable approach to a previously fixed component of L2O pipelines, with deployment requiring no LLM. The strengths are the clean separation between LLM proposal and external validation (no circularity), seed-preserving selection that prevents forced replacement, the equal-budget control on PDHG that isolates evolutionary feedback from best-of-48 sampling, and the matched-width random controls for six settings. The paper provides a reproducible protocol and a source artifact. However, the headline claim of 'consistent' improvement across four LLMs is contradicted by the paper's own per-model tables, and the isolation of feature content from width changes is only partially controlled. These issues are fixable by narrowing the claims and strengthening the statistical presentation.","major_comments":[{"comment":"The abstract states: 'Across continuous and discrete optimization tasks and four LLMs, the evolved features consistently outperform hand-crafted representations.' The paper's own per-model results contradict this. The 'Proposal-model dependence' paragraph reports improvements in 15, 17, 17, and 6 cells for Claude, GPT-5.5, Gemini, and DeepSeek, respectively. Appendix Tables 7-11 show that DeepSeek improves none of the 12 FSNet tasks and none of the 3 DC3 tasks, and ties on 14 of 20 shared continuous cells. Even for the stronger proposers, there are many equality cases (e.g., Table 8, Claude on nonconvex QP, nonsmooth QP, and nonsmooth QCQP). The conclusion 'evolved features improve solution prediction ... without changing the underlying models or solvers' is therefore too broad. The claim should be narrowed to 'FunL2O with capable LLM proposers reliably improves L2O features, while weake","section":"Proposal-model dependence; Tables 7-11"},{"comment":"The claim that measured gains are due to feature content rather than the 'mechanical interface change' of resizing the input layer (Eq. 6) is load-bearing. The matched-width random controls in Table 14 cover only six settings (five IPM-MPNN distributions and PDHG-Net). For FSNet, DC3, Smart Initial Basis, Learning to Pivot, and both MILP hosts, no width-matched control is provided. If resizing alters learning dynamics (capacity, gradient flow, effective regularization), then 'without changing the underlying models' is not established for those hosts. The paper should either extend the matched-width controls to all hosts where the interface permits, or explicitly state that the isolation claim is supported only in the controlled settings.","section":"Methodology, Eq. (6); Controlled Analyses 'Matched-Width Features'"},{"comment":"All reported comparisons are means of three seed-paired repetitions, but no variance, confidence intervals, or significance tests are provided. Several improvements are tiny (DC3: 2.0-4.3% gap reduction; Smart Initial Basis: +1.62 percentage points iteration saving) and could be within run-to-run noise. Additionally, the MILP summary in Table 3 and the text state that 'the best FunL2O variant outperforms the handcrafted features on both metrics across all three domains.' This is an oracle selection across the two proposal-model runs; for example, FunL2O-GPT on Predict-and-Search CA has a final primal gap of 1.023% versus 0.423% for handcrafted, i.e., worse. The 'evolved features improve MILP search' conclusion is supported only by cherry-picking the better variant per row, not by either variant individually. Please report per-variant results without best-of-selection and add uncertainty","section":"Metrics and Aggregation; Tables 2, 3, 7-11"}],"minor_comments":[{"comment":"The feasibility column reports values such as '0.957→0.942' (set cover) and '0.988→0.941' (facility location), which are decreases in a higher-is-better metric, yet the table is otherwise framed as improvement-oriented. Clarify the direction or mark these as decreases; the current presentation is ambiguous.","section":"Table 2, Feasibility column"},{"comment":"The phrase 'the first unified framework' is a strong novelty claim. Consider softening to 'a unified framework' unless a systematic prior-art search has been performed; this would not weaken the contribution.","section":"Abstract"},{"comment":"The paper notes that temperature and top-p are not overridden and follow provider defaults. This is a reproducibility concern; please record the actual sampling parameters or state clearly that default values are part of the configuration.","section":"Prompt and Proposal-Model Configuration"},{"comment":"The equal-budget control is described only for PDHG-Net. The text in the appendix says 'The available controls test the two principal alternative explanations' and 'These controls provide convergent evidence,' but the scope of the controls is limited to one host for feedback and six for width. Consider adding a sentence in the main text that these mechanism-level controls are representative, not universal.","section":"Controlled Analyses"}],"recommendation":"major_revision","confidential_remarks":"The paper contains a genuinely useful framework and a substantial experimental effort, but the abstract and conclusion considerably overstate the per-model evidence. The DeepSeek results show that the 'consistency across four LLMs' claim is not supported, and the MILP conclusion relies on oracle selection between proposal-model variants. The matched-width isolation claim is only partially controlled. All of these are fixable by rewriting the claims, adding per-model summaries without cherry-picking, and providing variance/significance information. I recommend major revision rather than rejection, since the core method and the positive results for stronger proposers are sound and the revisions are within scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FunL2O is a well-engineered application of LLM-driven program search to feature engineering for learning-to-optimize. The authors define semantic contracts, evaluate candidates by retraining the host pipeline and measuring downstream solver performance, and wrap it in a clean evolutionary loop. The experiments are unusually careful for this kind of paper: seed-paired repetitions, disjoint test sets, equal-budget independent-sampling control, matched-width random projections, and a seed-preservation safeguard. The evolved programs are readable and deployment doesn't require an LLM. That is real progress, and the eight host pipelines give the contribution breadth.\n\nThe main soft spot is the abstract's 'consistently outperform' across four LLMs. The body's own numbers say otherwise: DeepSeek-Math-V2 improves only 6 of 20 shared continuous cells and ties all FSNet and DC3 tasks; the other three LLMs improve 15–17 cells. So the universal claim is overbroad. The paper even acknowledges this in the proposal-model dependence paragraph, but then the conclusion and abstract keep the stronger wording. That needs to be fixed before publication.\n\nA couple of smaller issues: some gains are marginal (DC3 2–4%, Smart Initial Basis 1.6%), and the 'mechanical interface change' when resizing the input layer is only partly controlled—matched-width random controls cover just six LP/PDHG settings, not all hosts. The reader's concern about resize changing learning dynamics is plausible, but the controls that do exist make it a secondary worry rather than a fatal one. Reproducibility suffers because the source code isn't directly accessible; the appendix refers to an 'accompanying source artifact,' but it's not linked.\n\nI don't think any of this sinks the paper. The framework is solid, the evaluation is more thorough than most in the area, and the empirical gains are real for capable LLM proposers. It deserves a serious referee. The referees should ask for a toned-down claim, a public code release, and ideally additional matched-width controls for the MILP hosts. For me, this is a paper I'd bring to a reading group and probably cite when writing about LLM-driven discovery or L2O.","headline":"A carefully executed L2O feature-search framework with strong controls, but the 'consistently outperform across four LLMs' claim overshoots what the per-model tables show.","tokens_in":18525,"tokens_out":2821,"would_cite":true,"duration_ms":27696,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Feature functions in learning-to-optimize can be discovered by LLM-guided evolution, and the evolved representations consistently outperform hand-crafted ones.","keywords":["learning to optimize","feature function design","LLM-guided program evolution","representation learning","semantic contract","mixed-integer programming","warm-starting","solver guidance"],"falsifier":"An independent replication that retrains every pipeline with a matched-width random-feature function (same seeds, budget, and evaluation protocol) across all eight hosts; if the random-feature baseline matches or beats the evolved function on the held-out test split, the claim that evolved feature content drives the gains is falsified. The paper already runs this control on six of the LP and solver-facing settings; extending it to the remaining tasks would settle the question.","tokens_in":17586,"feed_emoji":"🧬","tokens_out":5067,"duration_ms":44629,"temperature":0.7,"pith_summary":"Every learning-to-optimize pipeline starts with a feature function that turns an optimization instance into the tensors a neural network consumes, and until now those functions have been hand-crafted and treated as fixed. The paper claims that this representation can be searched like a program: an LLM proposes executable feature functions, a validator checks that they respect the pipeline's interface, and the unchanged pipeline is retrained and scored by its real downstream metric to decide which candidates survive. Across eight pipelines—covering LP and QP solution prediction, warm-starting, simplex basis selection, pivot scoring, and mixed-integer search—the evolved features improve objective gaps, feasibility, iteration savings, and wall-clock time relative to the hand-crafted baselines. The point of the claim is that representation design, not just model architecture or training objective, is a removable bottleneck in learned optimization, and that it can be automated in an inspectable, deployment-free way.","feed_headline":"Evolved input features beat handcrafted ones across eight optimizers","feed_subtitle":"An LLM-search loop retrains each optimizer and keeps the best feature programs, improving predictions and solver speed.","key_machinery":"The carrying mechanism is the 'semantic feature contract' plus a measured-outcome evolutionary loop. The contract defines, for each pipeline, the allowed input fields, required output shape and width, invariants (determinism, finiteness, required hand-crafted channels), and an executable validator; it turns the open-ended space of possible feature programs into an admissible set that a constrained LLM can propose within. Every valid candidate is then evaluated the expensive way: the original learning-to-optimize model is retrained with that feature function and scored by the pipeline's native metric, ordered lexicographically (feasibility first, then violation, then objective or solver work)","core_discovery":"The paper's central claim is that the feature function is the overlooked variable in learning-to-optimize, and that an LLM-guided evolutionary loop can find better feature functions than the ones experts write by hand. In FunL2O, a large language model proposes candidate Python feature functions subject to a 'semantic contract' that fixes what information is available and what tensor shape must be returned; valid candidates replace the hand-crafted features only, the original model is retrained with the same data, loss, and schedule, and the resulting optimization outcome—feasibility, gap, iterations, pivots, or wall time—is the fitness signal. The search keeps the best programs as elites an","pith_inferences":["The paper treats resizing the model's input layer to match a new feature width as a 'mechanical interface change'; a testable inference is that some of the measured gains could come from the wider input layer altering optimization dynamics, which the random-width controls only partially rule out.","A natural extension the authors do not explore: the same contract-and-evolve loop could be applied to other pipeline components (loss functions, training schedules, solver interfaces) or to generating entirely new contracts, though feature functions are the most portable component.","Because the search cost is offline and the deployed feature function is LLM-free, the economic case strengthens as the number of future instances grows; in repeated-optimization settings, paying 48 retraining runs may be worthwhile even for modest per-instance gains.","The representative programs suggest transferable design patterns—normalizing by coverage, exposing directed mass, adding scale-invariant views—that could seed hand-designed features in problem classes beyond those tested."],"forward_implications":["Feature-function design becomes a separable, automatable step in learned optimization, so future pipelines can search over representations instead of inheriting fixed ones.","Because the returned feature functions are ordinary executable preprocessing code, they can be audited and reused; they also double as interpretable hypotheses about which problem structure (cost relative to coverage, directed mass asymmetry, normalized scales) matters.","The matched-width random-feature controls suggest that at least part of the gain comes from semantic content rather than simply feeding more numbers into a larger input layer.","The evolutionary-feedback comparison (independent sampling versus elite-guided search) shows that measured feedback itself contributes beyond the best-of-many effect.","Stronger LLMs produce successful feature functions more reliably, but the gain is not tied to a single provider since several different LLMs yield improvements."],"fun_headline_variants":["LLM-evolved features outperform handcrafted ones in optimizers","Evolving feature functions with LLMs improves optimization outcomes","Automated feature design via LLM evolution beats manual methods","FunL2O: LLM search finds better features for learning-to-optimize","LLM-guided feature evolution outperforms handcrafted in tests"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that resizing the fixed model's input layer to accommodate a different feature width is an inert 'mechanical' change; if that resize changes learning dynamics in any meaningful way, the claim that only the feature function changed—and therefore that measured gains are due to feature content—loses force.","fun_headline_variants_meta":{"raw":{"variants":["LLM-evolved features outperform handcrafted ones in optimizers","Evolving feature functions with LLMs improves optimization outcomes","Automated feature design via LLM evolution beats manual methods","FunL2O: LLM search finds better features for learning-to-optimize","LLM-guided feature evolution outperforms handcrafted in tests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1314,"prompt_tokens":730,"completion_tokens":584,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":495}},"tokens_in":474,"tokens_out":584,"duration_ms":5571,"temperature":1.0,"reasoning_tokens":495,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T08:14:55.550593+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent replication that retrains every pipeline with a matched-width random-feature function (same seeds, budget, and evaluation protocol) across all eight hosts; if the random-feature baseline matches or beats the evolved function on the held-out test split, the claim that evolved feature content drives the gains is falsified. The paper already runs this control on six of the LP and solver-facing settings; extending it to the remaining tasks would settle the question.","supporting_citations":[],"review_version":1}