{"id":"2b02ef9d-d350-4d7a-941f-90bfe85451cc","arxiv_id":"2412.14409","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A two-step multi-task training strategy for MILP solving produces embeddings that match specialized models in-distribution and generalize better on larger instances and new tasks.","lead":"This paper proposes a multi-task learning framework for MILP solving, where one shared neural network embedding is trained on several solver-guidance tasks at once and then adapted to new tasks. Experiments on three benchmarks suggest these shared embeddings generalize better across problem sizes and tasks than specialized single-task models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gains attributed to multi-task learning are confounded with pretraining/data volume; a single-task pretraining control is missing.","rationale":"I agree with the reader's weakest assumption. The paper's headline contribution is the multi-task framework and its claimed generalization benefit. The experimental design compares Multi-task-BAPAS (trained on Backdoor+PaS) with Single-task (trained on one task), so the number of tasks and the amount of pretraining data vary simultaneously. Without a single-task pretraining baseline, the results cannot distinguish multi-task representation learning from generic transfer learning or larger training corpora. This is not a question of consensus but of internal experimental control: the paper's own Section 5.2 interprets the gains as due to a 'generalized embedding,' yet no control isolates that mechanism. The anomalous Single-task MIS-L PaS result further suggests the baseline may be undertrained, which would exaggerate the multi-task advantage. Because the missing control undermines the central attribution, the paper should not be accepted without this experiment; the reader's CONDITIONAL verdict is appropriate. I recommend the condition include the single-task pretraining baseline (and, ideally, a data-volume-matched variant).","tokens_in":13119,"tokens_out":4975,"duration_ms":39943,"concrete_test":"Run the following control on Configuration as the new task and on PaS/Backdoor same-task generalization: take the same GAT encoder and pretrain it on 200 Backdoor instances (single task) with the identical contrastive loss and Step-1 protocol, then fine-tune the Configuration head on the 200 Configuration instances, exactly as in Multi-task-BAPAS. Compare on S and L instances. If this Single-task-Pretrain model reaches the same primal integral as Multi-task-BAPAS, the multi-task claim is not supported. To match data volume, also pretrain on 400 Backdoor instances (or 400 instances from the target task if available). If Single-task-Pretrain with 400 instances matches, the effect is data volume, not multi-task.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that shared multi-task representations generalize better to larger instances and new tasks (Sections 5.2, Tables 2 and 3). But Multi-task and Single-task differ in two confounded ways: (1) Multi-task Step 1 pretrains the shared GAT encoder on 200 instances from each of two tasks, i.e., 400 instances, before fine-tuning the target head on 200 target instances; (2) Single-task starts from random initialization and trains only on 200 target instances. Therefore any observed large-instance or cross-task improvement may be due to having seen 400 instead of 200 instances, or to transfer from any related pretraining task, not to the multi-task objective itself. The interpretation in Section 5.2 ('By learning a generalized embedding...') is asserted, not demonstrated. The unusually poor Single-task PaS result on MIS-L (Primal Integral 50.83 vs Gurobi 30.73, while Multi-task achieves 2.23) underscores that the single-task baselines may be under-trained, inflating the gap. A single-task pretraining baseline matching data volume and procedure is required before the paper can attribute the gains to multi-task representation learning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a multi-task representation learning framework for MILP solving. A shared GAT encoder is trained on two tasks (Backdoor, Predict-and-Search, or Solver Configuration) using a two-step procedure: first training the shared architecture with fixed, randomly initialized task-specific heads, then fine-tuning the task-specific heads for a target task. Experiments on Combinatorial Auction, Maximal Independent Set, and Minimum Vertex Cover benchmarks compare multi-task models against single-task models and default solvers. The reported results show that multi-task models are competitive with single-task models on in-distribution small instances and often superior on larger instances and on a new task, leading the authors to claim that multi-task representation learning improves generalization.","tokens_in":13338,"tokens_out":4664,"duration_ms":37351,"significance":"If the central claim is established, the framework could reduce the training cost for new MILP-guided tasks by reusing a shared embedding, and it could improve generalization to larger instances. The paper provides a public code repository and evaluates on genuinely held-out larger instances and cross-task fine-tuning, which are strengths. However, the headline attribution of the observed gains to multi-task learning is not yet supported because the multi-task and single-task comparisons differ in confounding factors, as detailed in the major comments.","major_comments":[{"comment":"The comparison between Multi-task and Single-task confounds multi-task learning with pre-training on additional data. Single-task is trained on 200 instances of the target task only, while Multi-task first trains the shared encoder on 400 instances (200 from each of two tasks) and then fine-tunes on the target task's 200 instances. Thus, the reported gains on large instances and on new tasks could be caused by seeing twice as many training instances, or by any form of pre-training on a related task, rather than by the multi-task objective itself. The interpretation in §5.2 ('By learning a generalized embedding that avoids overfitting to a single distribution…') is asserted without isolating this mechanism. To support the paper's central claim, the authors should add a single-task pre-training control: pre-train the same architecture on one task (e.g., Backdoor only) using the same two-step procedure and the same total data volume (200 + 200 instances), then fine-tune on the target task. If this control performs as well as Multi-task, the multi-task attribution would not be supported; if not, it would substantiate the claim.","section":"§5.1 (Baselines and Approaches) and §5.2 (Same-Task Performance, New-Task Performance)"},{"comment":"The Single-task PaS model achieves a Primal Integral of 50.83, which is substantially worse than the Gurobi baseline (30.73), while Multi-task-BAPAS achieves 2.23. Such a collapse suggests that the Single-task baseline may be under-trained or unstable for this distribution, which would inflate the apparent multi-task advantage. The paper does not report training curves or multiple random seeds for the Single-task baselines. To ensure a fair comparison, the authors should show that Single-task models are trained to convergence and report variance across seeds, or discuss any hyperparameter differences that might explain this failure.","section":"Table 2, MIS-L PaS row"}],"minor_comments":[{"comment":"The column header says 'Gurobi' for all task rows, but the Configuration rows actually compare SCIP, not Gurobi. This mislabeling should be corrected.","section":"Table 3"},{"comment":"The text says 'We use three randomly initialized task-specific layers to enhance the robustness of each task,' which is ambiguous: it is unclear whether three heads are used per task or three heads total across the two tasks. Please clarify.","section":"§4 (Multi-task Representation Learning)"},{"comment":"The contrastive loss formula appears to have a missing closing parenthesis in the denominator: the expression `exp(a′⊤pθ(P)/τ )` should likely be `exp(a′⊤pθ(P)/τ)`. Please check the typesetting.","section":"§4.2 (Network architecture and Contrastive loss)"},{"comment":"The claim 'first multi-task learning framework for ML-guided MILP solving' may be too strong given that multi-task learning is common in other domains; the related work section does acknowledge this, but the introduction should qualify the claim to avoid overstatement.","section":"§1 (Introduction)"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the missing single-task pre-training control. The paper's framework is plausible and the experiments are reproducible, but the current evidence does not establish that multi-task learning, rather than additional pre-training data, drives the observed generalization gains. The Single-task PaS collapse on MIS-L warrants particular scrutiny. I recommend major revision with a request for the control experiment and a more careful analysis of the single-task baselines."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a practical, useful paper for the ML4CO subfield, but the headline attribution — multi-task representation learning, not just extra pretraining data, causes the generalization gains — is not established by the experiments as written.\n\nThe genuinely new pieces are the two-step training recipe (train a shared GAT encoder against frozen, randomly initialized task heads, then fine-tune the heads) and its application to three MILP tasks: backdoors, predict-and-search (PaS), and a new configuration-generation task. The evaluation is mostly honest: large instances are truly unseen, and cross-task fine-tuning on a new task uses the same 200-instance budget as single-task training. The code, data-generation pipeline, and appendices are linked. Same-distribution performance is competitive with single-task models, which is a good sanity check.\n\nThe load-bearing soft spot is exactly the one in the stress-test note. Multi-task Step 1 trains on 200 instances from each of two tasks, i.e., 400 total instances; Single-task trains on 200 instances of one task. So any large-instance or cross-task gain could come from having seen twice as many instances or from any kind of related pretraining, rather than from the multi-task objective itself. The Section 5.2 statement that the model learns 'a generalized embedding that avoids overfitting to a single distribution' is an interpretation, not a demonstrated mechanism. The paper needs a single-task pretraining control (e.g., pretrain on Backdoor data alone, then fine-tune the same PaS head, with matching data volume) before the central claim is supported.\n\nSecond, the Single-task PaS result on MIS-L (Primal Integral 50.83 vs Gurobi's 30.73, while Multi-task achieves 2.23) is anomalously bad and unexplained. That looks like an undertrained or collapsed baseline, and it inflates the multi-task advantage. The authors should either fix the baseline or analyze what went wrong.\n\nOne minor gap: the two-step frozen-head trick is never compared against a more standard alternating multi-task training, so the paper doesn't actually show that the trick is necessary or beneficial. That ablation would strengthen it.\n\nOverall, the paper deserves a serious referee. The empirical direction is worthwhile, the held-out setup is honest, and the new configuration-generation task is a nice contribution. But the central mechanism claim needs the control experiment, and the PaS-MIS-L outlier needs explanation. I'd send it to review with those requests.\n\nBring it to reading group? Yes, it would generate a good discussion about what 'multi-task' means and what baselines are needed. I would cite it as related work on multi-task training for MILP, with a caveat about the missing control.","headline":"Useful multi-task pretraining for MILP with a real confound: gains may come from extra data, not multi-task learning per se.","tokens_in":13817,"tokens_out":4753,"would_cite":true,"duration_ms":38075,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single shared MILP embedding trained on two solver-guidance tasks can be fine-tuned to a third task and to larger instances, beating models trained on that task alone.","keywords":["multi-task learning","mixed integer linear programming","graph attention networks","contrastive learning","backdoors","predict-and-search","solver configuration","generalization"],"falsifier":"Train a single-task model on the combined data of the two pretraining tasks, using the same number of samples and the same encoder architecture, and then fine-tune it on the third task; if that model matches the multi-task model's performance on large instances and cross-task transfer, the claimed benefit of shared task structure is not needed.","tokens_in":12930,"feed_emoji":"🧩","tokens_out":4965,"duration_ms":40613,"temperature":0.7,"pith_summary":"The paper sets out to show that MILP solvers do not need a separate trained model for every decision task. It proposes a multi-task training scheme in which one shared graph-attention encoder learns a MILP embedding by alternating between two tasks, while randomly initialised task-specific heads stay frozen; afterwards the heads are fine-tuned and the encoder is not touched. The claim is that this shared embedding matches task-specific models on training-distribution instances and generalises better to larger instances and to a third, unseen task, across three benchmarks and two solvers. If true, it means a single pretrained MILP representation can be cheaply adapted to new solver-guidance tasks, reducing the data collection and training cost of ML-guided MILP solving.","feed_headline":"Shared embedding makes ML-guided MILP solving generalize across tasks","feed_subtitle":"One graph encoder trained on two MILP tasks beats task-specific models on larger instances and new tasks.","key_machinery":"The core mechanism is a two-step training schedule. In step one, the shared graph-attention encoder processes each MILP as a bipartite graph of variable and constraint nodes and is trained with the InfoNCE contrastive loss on alternating batches from two tasks, while three randomly initialised task heads are kept frozen; in step two, the task heads are fine-tuned with the encoder fixed, so the shared embedding is not distorted by any one task. This separation, together with the permutation-invariant bipartite encoding, is what lets the same embedding feed branching-priority prediction, solution-neighborhood prediction, and solver-configuration generation.","core_discovery":"The paper's central claim is that multi-task representation learning provides a reusable MILP embedding: a graph-attention encoder trained jointly on two tasks yields a representation that, after fine-tuning only a small task-specific head, performs at least as well as a fully trained single-task model on the same task and clearly better when the test instances are larger or the task is new. This is demonstrated on Combinatorial Auction, Maximum Independent Set, and Minimum Vertex Cover, under both Gurobi and SCIP, with the shared model matching or improving solve time, primal integral, and solution quality compared with single-task models.","pith_inferences":["The paper's attribution of the gains to multi-task learning is not fully isolated: no single-task pretraining baseline on the same data volume appears in the experiments, so a reader should treat the mechanism as plausible but not proven.","The fast fine-tuning suggests that if larger pretrained MILP encoders are built, adapting them to new solver tasks could cost far less than training from scratch; this is the direction the paper gestures toward with its mention of foundation models for MILP.","The Configuration head shows a lightweight way to produce solver parameters directly per instance, which could be tested on other solvers and on a broader parameter space than the 15 SCIP parameters used here.","The three tasks all rely on one-time inference from static instance features; extending the same shared embedding to dynamic features such as node selection during branch-and-bound is a natural but untested next step."],"forward_implications":["A model trained on two MILP-guidance tasks can be adapted to a third by training only a small task-specific head, with fine-tuning converging in under an hour in the experiments.","On larger instances, the multi-task model improves average solve time for Backdoors by roughly 15% over Gurobi and reduces PaS primal integral by up to about 92% over Gurobi, while the single-task models gain less or degrade.","The same shared embedding helps a new Configuration task generate per-instance SCIP settings that beat both SMAC and single-task configuration models, especially on large instances.","Cross-task checks, where each possible pair is trained and the third task is fine-tuned, all show improvements over single-task models, suggesting the result is not tied to one pair of tasks.","Because the framework is stated to work with any MILP graph representation, size-invariant network, and loss function, it can be slotted onto other solver-guidance tasks without redesign."],"supporting_citations":[{"why":"Supplies the bipartite graph representation of MILPs that the shared encoder consumes.","marker":"[21]"},{"why":"Provides the graph attention network mechanism used as the shared architecture.","marker":"[6]"},{"why":"Establishes the contrastive-learning backdoor task and data collection approach reused here.","marker":"[7]"},{"why":"Defines the Predict-and-Search task formulation that the paper adopts.","marker":"[23]"},{"why":"Contributes the contrastive predict-and-search model, positive/negative sampling, and hyperparameters for PaS.","marker":"[29]"},{"why":"Provides the contrastive loss with temperature 0.07 used for training and fine-tuning.","marker":"[28]"},{"why":"Supplies the Monte Carlo tree search method used to generate candidate backdoors.","marker":"[35]"},{"why":"Used to generate the training and test instances across small and large sizes.","marker":"[30]"},{"why":"SMAC3 is used to collect candidate solver configurations for the Configuration task.","marker":"[42]"},{"why":"Provides the set of 15 SCIP configuration parameters used by the Configuration head.","marker":"[26]"}],"fun_headline_variants":["Multi-task MILP learning beats single-task on new tasks and sizes","Shared MILP embedding generalizes across solvers and tasks","One model for MILP branching and config outperforms specialists","Multi-task training lifts MILP solver generalization","Graph encoder learns reusable MILP representations for any task"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The decisive assumption is that the gains come from sharing representations between tasks, but the experiments do not compare against a single-task model trained on the same total amount of data, so the observed advantage might simply reflect more training data rather than multi-task learning.","fun_headline_variants_meta":{"raw":{"variants":["Multi-task MILP learning beats single-task on new tasks and sizes","Shared MILP embedding generalizes across solvers and tasks","One model for MILP branching and config outperforms specialists","Multi-task training lifts MILP solver generalization","Graph encoder learns reusable MILP representations for any task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000122,"raw_usage":{"total_tokens":1022,"prompt_tokens":795,"completion_tokens":227,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":148}},"tokens_in":411,"tokens_out":227,"duration_ms":2412,"temperature":1.0,"reasoning_tokens":148,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:16:20.044670+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a single-task model on the combined data of the two pretraining tasks, using the same number of samples and the same encoder architecture, and then fine-tune it on the third task; if that model matches the multi-task model's performance on large instances and cross-task transfer, the claimed benefit of shared task structure is not needed.","supporting_citations":[{"cited_title":"Advances in neural infor- mation processing systems32 (2019)","cited_arxiv_id":null,"evidence_quote":"Supplies the bipartite graph representation of MILPs that the shared encoder consumes."},{"cited_title":"In: Proceedings of the AAAI Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"Supplies the Monte Carlo tree search method used to generate candidate backdoors."},{"cited_title":"Journal of Machine Learning Research 23(54), 1–9 (2022)","cited_arxiv_id":null,"evidence_quote":"SMAC3 is used to collect candidate solver configurations for the Configuration task."},{"cited_title":"Annals of Operations Research339(1), 909–936 (2024)","cited_arxiv_id":null,"evidence_quote":"Provides the set of 15 SCIP configuration parameters used by the Configuration head."}],"review_version":1}