{"id":"04577a1e-9994-4ec4-9810-cc53028310a3","arxiv_id":"2507.05501","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A Julia package, MultiObjectiveAlgorithms.jl, implements ten scalarization-based algorithms for multi-objective optimization on top of JuMP, with benchmarks matching published results.","lead":"MultiObjectiveAlgorithms.jl is a new open-source Julia package that solves optimization problems with multiple objectives at once. It bundles ten published algorithms and connects to many solvers, aiming to make multi-objective optimization easier for researchers and practitioners.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"External validation only compares average cardinality of nondominated sets, not the sets themselves; the claim that solutions match Tamby and Vanderpooten is overstated.","rationale":"The reader's weakest assumption correctly identifies subproblem optimality as a condition for the theoretical solution-set guarantees, and the paper's own report of a non-supported point from Dichotomy with HiGHS is concrete evidence that this condition can fail. However, an even more load-bearing weakness is in the benchmark validation: the paper claims its solutions 'match those reported' in Tamby and Vanderpooten, but the only comparison described is of average cardinalities. Cardinality equality is necessary but not sufficient for solution-set equality; two algorithms can return different nondominated sets of the same size. Since the benchmark match is the primary empirical evidence that the implementations are correct, this gap weakens the central claim more directly than the acknowledged solver-gap issue. The paper does not claim formal verification, and Section 5.2 explicitly disclaims quality guarantees beyond non-domination of returned points, so the package may still be sound; but the validation section overstates what was checked. The verdict remains CONDITIONAL: the package appears usable and likely correct, but the external-validation claim should be either strengthened with pointwise set comparisons or reworded to say that the average number of nondominated points matches. The proposed test—pointwise comparison of objective vectors on the benchmark instances—would settle whether the current wording is justified.","tokens_in":10385,"tokens_out":8102,"duration_ms":87743,"concrete_test":"Use the reproduction archive (Dowson et al. 2026) to run all 30 Tamby-Vanderpooten instances with EpsilonConstraint, Dichotomy, and TambyVanderpooten under both Gurobi and HiGHS. For each instance, extract the full set of objective vectors and compare it pointwise to the nondominated sets from the reference implementation/data of Tamby and Vanderpooten (2021). Report the number of instances where the sets match exactly (up to tolerance). If exact set equality fails beyond the two already-noted Dichotomy/HiGHS anomalies, the paper's verification claim in §3 is not supported. Additionally, for the Dichotomy/HiGHS anomaly, re-run with HiGHS parameter mip_rel_gap=0 (or the solver's tightest gap tolerance) to confirm the extra non-supported point disappears, isolating the cause to subproblem optimality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central evidence for correctness is §3's statement, 'We verified that our solutions match those reported in Tamby and Vanderpooten (2021).' The actual check reported in that paragraph is a match of the average number of non-dominated solutions (405.4, 4090.0, 2240.1) to the averages reported by Tamby and Vanderpooten. Matching aggregate cardinalities does not imply the generated nondominated sets are identical; distinct sets of points can have the same size. The paper does not describe any pointwise comparison of objective vectors, and it does not state that the reference nondominated sets were available for comparison. Thus the benchmark section does not validate that the ten algorithms produce the correct nondominated set per instance; it only validates that the number of returned points is comparable on average. Additionally, the authors themselves report a concrete violation of an algorithmic guarantee: for two (p,n)=(2,200) instances, Dichotomy with HiGHS returns a non-supported point, so the returned set is not a minimum supported set (Table 1). While the authors attribute this to the MIP gap, from a user's standpoint the advertised output type is not produced. Both issues make the correctness claim conditional on subproblem solver optimality and on a stronger form of external validation than was performed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes MultiObjectiveAlgorithms.jl (MOA.jl), a Julia package built on JuMP for solving multi-objective optimization problems. It formulates a multi-objective problem with a vector-valued objective, defines solution concepts (efficient solutions, nondominated points, supported points, minimum complete sets), and presents ten scalarization-based exact algorithms implemented through a modular interface. After a didactic code example, the paper reports benchmarks on binary knapsack instances from Tamby and Vanderpooten (2021), compares MOA.jl with other modeling languages and solvers, and discusses design principles. The central claims are that the library correctly implements the ten algorithms and that it supports a wide range of problem classes, from linear and integer to conic, semi-definite, and general nonlinear, through JuMP and third-party solvers.","tokens_in":10588,"tokens_out":4461,"duration_ms":53183,"significance":"If the claims are correct, MOA.jl is a valuable open-source infrastructure contribution: it gives practitioners a single JuMP-native interface to multiple multi-objective algorithms, makes switching between algorithms and solvers easy, and provides reproducible benchmark code. The explicit design choice to return finite sets of points, together with the honest discussion of its trade-offs in Section 5.2, is a useful contribution to the software-oriented literature. The paper does not claim algorithmic novelty; its value is in software engineering, dissemination, and lowering the barrier to using multi-objective optimization. However, the evidence presented for correctness is weaker than the claims: the external validation compares only aggregate cardinalities, and the benchmark itself documents a violation of a stated algorithmic guarantee. These issues need to be resolved before the correctness claims can be fully accepted.","major_comments":[{"comment":"The statement \"We verified that our solutions match those reported in Tamby and Vanderpooten (2021)\" is not supported by the reported comparison. The verification only compares the average number of nondominated solutions (405.4, 4090.0, 2240.1) with the averages reported in the reference. Distinct nondominated sets can have the same cardinality, so equality of average set sizes does not demonstrate that the generated objective vectors coincide with the reference sets. The paper should either provide a pointwise comparison of the objective vectors on the shared instances, or weaken the claim to, for example, \"the average number of nondominated solutions is comparable.\" This matters because this benchmark is the only external evidence that the algorithms produce the correct nondominated sets.","section":"Section 3"},{"comment":"The Dichotomy algorithm is listed in Table 1 as returning a minimum supported set, but the benchmark in Section 3 reports that for two (p,n)=(2,200) instances HiGHS finds a non-supported point because the scalarized objective is within the MIP gap. The returned output is therefore not a minimum supported set, contrary to the advertised guarantee. This is not merely a theoretical caveat: it is a concrete failure mode that a user can encounter with default settings. The library should either check and disclose solver optimality gaps, qualify the solution type as \"supported up to solver tolerance,\" or revise the algorithm contract so that the returned set is no longer promised to be a minimum supported set.","section":"Section 3 and Table 1"},{"comment":"The paper claims support for conic, semi-definite, and general nonlinear problems, but no benchmark or test in Section 3 exercises these problem classes, and Table 1's \"Solution\" column explicitly states its guarantees only for \"pure linear or pure integer\" problems. For nonconvex nonlinear models, the scalar subproblems require global solvers, and the exactness guarantees described for the algorithms cannot hold without additional assumptions. The paper should either state this limitation explicitly in the abstract and in Section 2, or include at least a small correctness test for a conic or quadratic case if the broad support claim is retained.","section":"Abstract, Section 2.1, Section 6"}],"minor_comments":[{"comment":"The abstract states that MOA.jl is released under the MPL-2 license, while Table 3 lists the license as BSD-3. These should be made consistent.","section":"Abstract and Table 3"},{"comment":"The code example uses `assert_is_solved_and_feasible(model)` without defining or referencing this function; since the code is meant as a didactic example, a brief note about the source of this helper would improve clarity.","section":"Section 2, code listing"},{"comment":"The pseudo-code for the epsilon-constraint algorithm has formatting issues, such as \"opt∗1\" and \"ub1\" without proper subscripts; this makes the algorithm harder to read than necessary.","section":"Algorithm 1"},{"comment":"The sentence \"MOA.jlis a meta-solver. In high-level Julia code it modifies the vectors...\" reads awkwardly; it would be clearer as \"MOA.jl is a meta-solver: at a high level, it modifies the vectors...\".","section":"Section 1.2"},{"comment":"Section 5.2 says the library makes \"no quality assertions about the returned list, other than that each objective vector is non-dominated,\" but Table 1 promises minimum complete sets or minimum supported sets for specific problem classes. These two statements should be reconciled so that users can know what guarantee actually applies to a given algorithm and problem class.","section":"Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the software-and-computing scope of the journal, and the open-source artifact is genuinely useful. The main concern is that the correctness claim rests on an aggregate validation that is weaker than the claim, and the Dichotomy/HiGHS anomaly shows a gap between advertised guarantees and observed behavior. Both are fixable in revision, so I do not recommend rejection. The license inconsistency is minor but should be corrected before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a software paper, and the software is the contribution. MOA.jl is a well-organized Julia package that puts ten published multi-objective algorithms behind a single JuMP interface, with a modular design that makes adding new algorithms genuinely easy. The JuMP extension for vector-valued objectives is real work and will benefit the community. The code and data are archived; that counts for a lot.\n\nThe paper is honest about its limits: it doesn't claim new algorithms, it discloses that solutions are finite sets, and it admits that some guarantees are solver-dependent. That is more than many software papers do.\n\nThe soft spots are real but not fatal. The claim \"we verified that our solutions match those reported in Tamby and Vanderpooten\" is overstated. What they actually verified is that the average number of nondominated points matches: 405.4, 4090.0, 2240.1. Distinct nondominated sets can have the same cardinality, so this is not a pointwise validation. A serious referee should ask for a direct set comparison on at least a few instances. Also unresolved is the Dichotomy/HiGHS case: on two instances the algorithm returns a non-supported point because the scalarized subproblem stopped within the MIP gap. The authors report this openly, but it means the advertised \"minimum supported set\" guarantee is conditional on the subproblem solver, and users need to know that. The claim of supporting conic/SDP/nonlinear classes is untested here; that's a minor concern because the scalarization framework should carry over, but a quick demonstration would help.\n\nNone of these undermine the package. The stress-test note is right that the benchmark section's wording goes beyond the evidence, but the central artifact—the library—is solid, open, and reproducible. This is not a case of fabricated results; it's a case of loose language around an illustrative benchmark.\n\nI'd send this to review. A referee will get value from checking the code, and the authors will get value from the pointwise comparison request. The paper will be a useful reference for anyone building on JuMP or wanting a general-purpose MOO solver.\n\nRecommendation: accept the paper into the review process; ask for a corrected validation claim and a caveat about solver-dependent guarantees in the abstract.","headline":"A genuinely useful MOO library with a paper that overstates its benchmark validation; the software is the contribution, and it should go to review.","tokens_in":11118,"tokens_out":2795,"would_cite":true,"duration_ms":30939,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C29"],"pacs":[],"model":"deepseek-v4-flash","headline":"MultiObjectiveAlgorithms.jl brings ten scalarization-based exact algorithms to JuMP and reproduces published nondominated-set counts on benchmark instances.","keywords":["multi-objective optimization","scalarization","Julia","JuMP","meta-solver","nondominated set","efficient solutions","mathematical programming software"],"falsifier":"Enumerate all feasible solutions of a small binary knapsack instance to compute the true nondominated set, then run the EpsilonConstraint algorithm with $\\varepsilon=1$ and the TambyVanderpooten algorithm; finding a returned point that is dominated by a feasible solution, or a true nondominated point that no returned point attains, would disprove the minimum-complete-set claim.","tokens_in":10172,"feed_emoji":"🧮","tokens_out":10892,"duration_ms":121613,"temperature":0.7,"pith_summary":"This paper presents an open-source Julia library, MultiObjectiveAlgorithms.jl, that solves multi-objective optimization problems by iteratively scalarizing them into sequences of single-objective subproblems. Its central claim is that the ten implemented algorithms are correct: for the published multi-objective binary knapsack benchmarks, the exact algorithms return the same average numbers of nondominated solutions as the original studies, and every returned objective vector is nondominated. To make this possible, the paper extends the JuMP modeling language to accept vector-valued objectives, so one model can be solved by algorithms that return a representative set, a minimum supported set, or a minimum complete set. A sympathetic reader should care because the library puts sophisticated multi-objective methods within reach of practitioners who already know JuMP and lets researchers compare new algorithms against established ones with a few lines of code.","feed_headline":"Julia package adds ten exact multi-objective solvers to JuMP","feed_subtitle":"It scalarizes any multi-objective model into single-objective subproblems and matches published benchmark solution counts.","key_machinery":"The load-bearing object is the scalarization subproblem $\\operatorname{SP}(u,v)$, which minimizes $u^\\top f_0(x)$ subject to the original constraints and the additional bound $f_0(x) \\le v$; changing $u$ and $v$ across solves and filtering dominated candidates is how every algorithm produces its finite solution set. Around this subproblem, the paper builds a modular algorithm interface in which each method subtypes an AbstractAlgorithm and implements a single minimize_multiobjective function, keeping each implementation under 250 lines and making new algorithms drop into JuMP immediately. The same interface handles objective-sense conversion, subproblem caching, and translation of returned vectors into JuMP's multiple-result format.","core_discovery":"The paper's claim is that MultiObjectiveAlgorithms.jl faithfully implements ten exact, preference-free solution algorithms for the multi-objective problem, all built on the same scalarization mechanism. For pure integer linear problems, the TambyVanderpooten and KirlikSayin enumeration algorithms are claimed to return a minimum complete set of efficient solutions, the supported-set Dichotomy algorithm returns a minimum supported set, and the Lexicographic and Hierarchical algorithms return single or representative solutions; for every algorithm the library returns a finite set of points and asserts only that each objective vector is nondominated. The paper validates correctness by reproducing published benchmark results: on the multi-objective binary knapsack instances, the average nondominated-set sizes are 405.4 for $(p,n)=(2,200)$, 4,090.0 for $(3,100)$, and 2,240.1 for $(4,50)$. It also records the main caveat to its own guarantee: one run of Dichotomy with HiGHS returned a valid nondominated but unsupported point because the scalarized objective value lay within the MIP gap, an outcome the algorithm should not produce in theory.","pith_inferences":["If the benchmark ratios hold more broadly, the practical frontier for exact multi-objective optimization is single-objective MIP speed; improvements to scalar solvers and warm-starting should transfer almost directly to these algorithms.","The observed HiGHS behavior suggests that users who rely on a minimum-supported-set guarantee should tighten the subproblem optimality gap, since default tolerances can admit unsupported points even when each solve looks optimal.","The same SP(u,v) abstraction could host preference-based or approximate algorithms without touching the JuMP interface, because the solver interaction is already isolated in one place."],"forward_implications":["A user can switch between representative, supported, and complete-set algorithms by changing one line in a JuMP model, making it straightforward to compare the trade-off between solution-set size and solve time on the same instance.","Because the library builds on JuMP and MathOptInterface, one formulation covers linear, integer, conic, semidefinite, and general nonlinear problems, limited only by what the chosen single-objective solver can handle.","For pure discrete problems, users can obtain the full nondominated set rather than a single compromise point, which distinguishes the library from the multi-objective features of most commercial solvers.","The benchmark timing results indicate that essentially all runtime is spent inside the subproblem solver, so algorithm choice primarily changes the number of subproblems while library overhead stays small.","The finite-set design means that continuous problems are solved by returning a finite list of nondominated points, so users need to know whether their chosen algorithm gives a complete, supported, or merely representative set."],"supporting_citations":[{"why":"It supplies the multi-objective binary knapsack benchmark instances and the published nondominated-set counts that the paper reproduces.","marker":"Tamby and Vanderpooten (2021)"},{"why":"It is the source of the Dichotomy algorithm, which returns a minimum supported set and produces the paper's observed unsupported-point caveat with HiGHS.","marker":"Aneja and Nair (1979)"},{"why":"It is the source of the epsilon-constraint algorithm, one of the three algorithms benchmarked and the basis of Algorithm 1.","marker":"Haimes et al. (1971)"},{"why":"It is the source of the DominguezRios integer-program algorithm, cited as an example of recent algorithms that motivated the modular design.","marker":"Dominguez-Rios et al. (2021)"},{"why":"It is the source of the KirlikSayin algorithm for enumerating all nondominated solutions of multiobjective discrete problems.","marker":"Kirlik and Sayın (2014)"},{"why":"It describes JuMP 1.0, the modeling language the package extends with vector-valued objectives.","marker":"Lubin et al. (2023)"},{"why":"It describes HiGHS, the open-source subproblem solver used in the benchmarks and in the reported MIP-gap caveat.","marker":"Huangfu and Hall (2018)"},{"why":"It defines MathOptInterface, the standard form that lets MultiObjectiveAlgorithms.jl pass subproblems to many solvers.","marker":"Legat et al. (2021)"}],"fun_headline_variants":["Ten exact multi-objective solvers now in Julia for JuMP","MultiObjectiveAlgorithms.jl brings ten exact solvers to JuMP","Julia library solves multi-objective problems via scalarization","Extend JuMP with ten exact multi-objective algorithms","New Julia package: ten exact solvers for multi-objective optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The correctness of every returned set depends on the underlying solver solving each single-objective subproblem to proven optimality, and when that fails—as the paper reports for one HiGHS run with the Dichotomy algorithm—a returned list can contain a point the theory says should not appear.","fun_headline_variants_meta":{"raw":{"variants":["Ten exact multi-objective solvers now in Julia for JuMP","MultiObjectiveAlgorithms.jl brings ten exact solvers to JuMP","Julia library solves multi-objective problems via scalarization","Extend JuMP with ten exact multi-objective algorithms","New Julia package: ten exact solvers for multi-objective optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000709,"raw_usage":{"total_tokens":3182,"prompt_tokens":926,"completion_tokens":2256,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":2169}},"tokens_in":542,"tokens_out":2256,"duration_ms":17960,"temperature":1.0,"reasoning_tokens":2169,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:24:48.677966+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Enumerate all feasible solutions of a small binary knapsack instance to compute the true nondominated set, then run the EpsilonConstraint algorithm with $\\varepsilon=1$ and the TambyVanderpooten algorithm; finding a returned point that is dominated by a feasible solution, or a true nondominated point that no returned point attains, would disprove the minimum-complete-set claim.","supporting_citations":[{"cited_title":"INFORMS Journal on Computing 33(1):72--85","cited_arxiv_id":null,"evidence_quote":"It supplies the multi-objective binary knapsack benchmark instances and the published nondominated-set counts that the paper reproduces."},{"cited_title":"Management Science 25(1):73--78","cited_arxiv_id":null,"evidence_quote":"It is the source of the Dichotomy algorithm, which returns a minimum supported set and produces the paper's observed unsupported-point caveat with HiGHS."},{"cited_title":"IEEE transactions on systems, man, and cybernetics 3:296--297","cited_arxiv_id":null,"evidence_quote":"It is the source of the epsilon-constraint algorithm, one of the three algorithms benchmarked and the basis of Algorithm 1."},{"cited_title":"Information Sciences 565(7):210--228","cited_arxiv_id":null,"evidence_quote":"It is the source of the DominguezRios integer-program algorithm, cited as an example of recent algorithms that motivated the modular design."},{"cited_title":"European Journal of Operational Research 232(3):479--488","cited_arxiv_id":null,"evidence_quote":"It is the source of the KirlikSayin algorithm for enumerating all nondominated solutions of multiobjective discrete problems."},{"cited_title":"Mathematical Programming Computation 15(3):581--589","cited_arxiv_id":null,"evidence_quote":"It describes JuMP 1.0, the modeling language the package extends with vector-valued objectives."},{"cited_title":"Mathematical Programming Computation 10(1):119--142","cited_arxiv_id":null,"evidence_quote":"It describes HiGHS, the open-source subproblem solver used in the benchmarks and in the reported MIP-gap caveat."},{"cited_title":"INFORMS Journal on Computing 34(2):672--689","cited_arxiv_id":null,"evidence_quote":"It defines MathOptInterface, the standard form that lets MultiObjectiveAlgorithms.jl pass subproblems to many solvers."}],"review_version":1}