{"id":"4e13bfd0-3454-461c-b838-f85c92b3a16b","arxiv_id":"2412.09293","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"On one 65-station Polish fuel delivery instance, simulated annealing with random starts beats Gurobi only within the first five seconds of computation.","lead":"This paper tests a Monte Carlo plus simulated annealing routing heuristic against Gurobi's exact solver on fuel deliveries to 65 Polish gas stations. It finds the heuristic matches or beats the solver only in the first few seconds of computation, after which Gurobi pulls ahead.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Gurobi baseline is not trustworthy: Eq (5) breaks flow conservation at non-depot stations and Eqs (10)-(11) fail to upper-bound route demand, so the 5-second comparison may be invalid.","rationale":"The reader's weakest assumption and my own converge: the Gurobi baseline is load-bearing and the written MIP constraints are not a correct CVRP formulation. The paper is otherwise clearly written, uses real data, and makes code available, but those strengths do not repair the comparison if the solver's objective is computed for a model that permits infeasible routes or rejects feasible ones. The flow-conservation defect (Eq 5) alone makes the model infeasible for ordinary routes with intermediate stations, and the capacity constraints (Eqs 10-11) fail to enforce the advertised 39,000 L upper bound. Because the headline claim is empirical and comparative, a corrected baseline can overturn it. I therefore agree with the reader's REJECT verdict; no adjustment is needed.","tokens_in":11995,"tokens_out":4048,"duration_ms":41706,"concrete_test":"Clone https://github.com/naumovvs/MC-SA-routing and locate the MIP construction corresponding to Section 3.3. Add the missing flow-conservation terms (arcs from 0 and to G+1) and replace Eqs (10)-(11) with a single upper bound sum_{i=1}^G sum_{j=1}^{G+1} x_{n,i,j} q_{k,i} <= C for each route. Re-run the 65-station instance with Gurobi for 5 seconds and compare route distance and feasibility to the published values (Gurobi 2438 km at 2s, heuristic 2381 km at ~1s). If the corrected Gurobi solution at 5s is better than 2381 km, or if the uncorrected model admits over-capacity routes, the headline comparison fails.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim ('SA consistently outperforms Gurobi up to 5 seconds') rests on the Gurobi MIP being a correct baseline for the same CVRP. Section 3.3's constraints are internally inconsistent. Eq (5) enforces inflow=outflow at each station i=1..G using only arcs j=1..G, omitting arcs 0->i and i->G+1; hence any route with an intermediate station (0->a->...->b->G+1) violates (5) at the first and last stations. Eq (10) bounds only the demand delivered before the last station by C, and Eq (11) only requires total route demand >= C; there is no constraint that total route demand <= C. A route whose last stop has q=15,500 can carry up to 54,500 L against a 39,000 L tank. If the implemented model mirrors these equations, every reported Gurobi solution is suspect; if it does not, the paper's written model does not match the experiment. Either way the runtime comparison is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage heuristic for the capacitated vehicle routing problem (CVRP) in fuel delivery: a Monte Carlo stage generates many candidate route sets, and a simulated annealing stage improves the best of them. The method is compared against a mixed-integer programming model solved with Gurobi on a single instance built from Polish fuel-delivery data. The central claim is that the simulated-annealing heuristic consistently produces shorter routes than the Gurobi-based model for computation times up to about five seconds, after which Gurobi tends to catch up or surpass it.","tokens_in":12253,"tokens_out":7557,"duration_ms":80460,"significance":"If the comparison were valid, the paper would provide a practically useful result: for short planning budgets, a simple open-source heuristic may beat a commercial MIP solver on a real-world fuel-delivery instance. The authors deserve credit for making source code publicly available and for averaging the stochastic heuristic over 100 repetitions. However, the central claim is not currently established. The written MIP model has serious correctness defects, the presented heuristic algorithm contains a non-terminating loop in its pseudocode, and the quantitative evidence is drawn from one of the 300 claimed instances. The significance of the work is therefore conditional on substantial correction and re-evaluation.","major_comments":[{"comment":"Eq. (5) does not enforce flow conservation for the first and last stations of a route. It writes sum_{j=1..G} x_{n,j,i} - sum_{j=1..G} x_{n,i,j} = 0 for each station i, using only station-to-station arcs. For a route of the form 0 -> a -> ... -> b -> G+1, station a has no incoming station-to-station arc, so the balance is -1, and station b has no outgoing station-to-station arc, so the balance is +1. Thus the constraint is violated by every route with at least two stations. If the implemented Gurobi model reflects this equation, the reported baseline solutions cannot be valid routes; if it does not, the written model does not match the experiment.","section":"Section 3.3, Eq. (5)"},{"comment":"The capacity constraints do not bound total route demand above by C. Eq. (10) only bounds the demand delivered before the last station by C, while Eq. (11) only requires that the total demand of the route be at least C. A route that delivers 39,000 liters before its final stop and 15,500 liters at the final stop has total demand 54,500 liters, exceeding the stated 39,000-liter capacity, yet it satisfies both inequalities. The Gurobi model as written therefore admits infeasible routes, which invalidates the runtime-quality comparison in Section 5.","section":"Section 3.3, Eqs. (10)-(11)"},{"comment":"Even if Eq. (5) were corrected to include depot arcs, the constraint set does not eliminate disconnected subtours. For a fixed route index n, the constraints allow a cycle b -> c -> b among stations to coexist with the path 0 -> a -> G+1, because station-level inflow-outflow balance does not enforce route connectivity. A correct CVRP formulation needs explicit subtour-elimination constraints or an equivalent connected-flow structure. This is a further load-bearing defect in the baseline model.","section":"Section 3.3, Eqs. (3)-(6)"},{"comment":"The inner route-generation loop in Algorithm 2 can fail to terminate. The loop continues while volume < C (line 6), but a station is accepted only when volume + q_j <= C (line 8). Since volume never exceeds C, the loop can exit only if the residual capacity is filled exactly; if no remaining station fits in the residual capacity, the while loop spins forever because no station is removed from Omega. The reported run times therefore imply that the implementation does not match this pseudocode, so the heuristic as presented is not a well-defined algorithm.","section":"Section 4, Algorithm 2"},{"comment":"The paper reports detailed results for only one of the 300 generated instances. The statement that the same regularities were observed in every other instance is not supported by any aggregated data, so the claimed conclusion that the heuristic 'consistently outperforms' Gurobi up to five seconds rests on a single case. In addition, Table 2 gives only means over 100 repetitions, without standard deviations or other variability measures, so the stochastic comparisons are not statistically quantified.","section":"Section 5, Table 2 and Figures 3-5"}],"minor_comments":[{"comment":"The constraint uses the index k in s_{k,i} and u_{k,j} without specifying which day k corresponds to route n; the mapping between routes and days should be made explicit.","section":"Section 3.3, Eq. (7)"},{"comment":"Some constraints are written with n = 1,...,N_k and others with n = 1,...,N; since N = sum_k N_k, the intended indexing should be clarified.","section":"Section 3.3, notation"},{"comment":"There is a typo in line 8 of Algorithm 3: 'cadidate' should be 'candidate'.","section":"Section 4, Algorithm 3"},{"comment":"Reporting the Gurobi MIP optimality gap would help interpret the statement that later improvements yield diminishing returns.","section":"Section 5, Figure 2"},{"comment":"The table header reads 'T able 1'; this formatting error should be corrected.","section":"Table 1"}],"recommendation":"reject","confidential_remarks":"The paper's central comparison is invalid as written: the MIP baseline permits infeasible routes and the heuristic pseudocode has a non-terminating loop. Even if the GitHub code contains corrected versions, the manuscript does not describe the implemented model, and the single-instance evidence cannot support the general conclusion. These are load-bearing issues that would require reworking the model, re-running the experiments, and possibly changing the conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on Naumov's CVRP paper. The genuinely new thing is an empirical result: on one 65-station fuel delivery instance from Samat, Monte Carlo initial solutions plus simulated annealing with 2-opt reversals find shorter routes than Gurobi within the first 5 seconds of solve time. The algorithmic ingredients are standard, and the paper cites many earlier SA applications to CVRP. What it does well is present the heuristic clearly, run 100 repetitions for three MC trial counts, and make the Python code public. That is reproducible work.\n\nThe soft spots land on the load-bearing comparison. The written MIP model in Section 3.3 is internally inconsistent. Eq (5) enforces flow balance using only arcs between stations, so the first and last station on any route violate it. Eqs (10) and (11) bound pre-last-stop demand above by C and total demand below by C, but impose no upper bound on total route demand. A route with a last stop of 15,500 L can carry 54,500 L against a 39,000 L tank. If the implemented Gurobi model matches these equations, its solutions may be infeasible, making the five-second win a comparison to a broken baseline. If the implementation fixes these constraints, the written model does not describe the experiment. Either way, the central comparison is not established.\n\nThe evidence also falls short of the conclusion. The paper mentions 300 instances and says the regularities hold for all, but reports detailed results for only one. No aggregation, no error bars on the 100-run averages. The claim that the heuristic 'consistently outperforms' Gurobi up to 5 seconds is broader than the data shown. A minor implementation concern: Algorithm 2's inner loop may spin indefinitely if no remaining station fits the residual capacity, though it apparently terminated in practice.\n\nThis is for a reader who wants a quick, practical heuristic for fuel dispatch and is willing to treat the comparison carefully. The paper deserves a serious referee because it has real data, clear exposition, and a correctable flaw. If the author fixes the MIP model and re-runs the baseline, it could become a useful case study. I would not cite it in its current form.\n\nRecommendation: engage with it, but expect major revision and a re-run of the Gurobi comparison.","headline":"A routine MC+SA case study whose central Gurobi comparison is undermined by a flawed written MIP baseline; clear exposition and public code, but the headline claim is not established.","tokens_in":12720,"tokens_out":4415,"would_cite":false,"duration_ms":42461,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C59","90B06","90C11"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a Monte Carlo plus simulated annealing heuristic returns shorter fuel-delivery routes than a Gurobi-powered MIP model for computation times up to about five seconds, after which the MIP solver catches up.","keywords":["fuel delivery","CVRP","simulated annealing","Monte Carlo simulation","Gurobi","vehicle routing","time windows"],"falsifier":"A direct check is to take the demand matrix from the reported instance and sum the demands along each Gurobi-generated route; any route whose total demand exceeds the 39,000-liter capacity shows the baseline solutions are infeasible and the runtime comparison collapses. A stronger test is to re-run the comparison with a corrected MIP model that enforces flow conservation at the first and last stops of every route and bounds total route demand above by capacity, then see whether Gurobi still trails the heuristic in the first five seconds.","tokens_in":11806,"feed_emoji":"⛽","tokens_out":12273,"duration_ms":108216,"temperature":0.7,"pith_summary":"Fuel dispatchers often need to rebuild delivery routes within seconds when demand shifts, but commercial mixed-integer solvers can take many seconds to return a good solution. The paper claims that a simple two-stage heuristic — random route construction by Monte Carlo, then route-by-route simulated annealing — consistently gives shorter total distances than a Gurobi-powered MIP model when the planning budget is about five seconds or less. In the reported instance, the heuristic finds a 2381-km plan in roughly one second, while Gurobi needs about five seconds to match that quality. The practical answer to the title question is: use simulated annealing when time is short, and switch to an exact solver when the planner can wait longer.","feed_headline":"For 5-second planning, annealed heuristic beats Gurobi on fuel routes","feed_subtitle":"A Monte Carlo plus simulated annealing heuristic finds shorter fuel routes than the MIP solver in seconds.","key_machinery":"The mechanism is a two-stage stochastic local search. The first stage runs R Monte Carlo constructions: each route randomly adds stations while respecting the 39,000-liter vehicle capacity and the latest-arrival time windows, and the shortest complete plan is kept. The second stage applies simulated annealing independently to each route, generating a candidate by reversing a randomly chosen fragment between two non-neighbor stations (a 2-opt-style move) and accepting it with probability governed by the temperature schedule $T_i = T_0/(i+1)$, with $T_0 = 1$ and $T_{\\mathrm{end}} = 0.001$. The SA stage is what extracts the additional distance reduction at almost no computational cost, about 0.06 seconds per run.","core_discovery":"The paper's central discovery is an empirical answer to when simulated annealing should be used for capacitated fuel-delivery routing. With 1000 Monte Carlo route constructions, the two-stage heuristic averages 2381 km of total route distance in about 1.09 seconds, while the Gurobi solver reaches that same distance only after about five seconds; the paper states that the heuristic consistently outperforms the Gurobi-powered model for up to five seconds of computation time. Beyond that threshold the solver overtakes it: with 10,000 Monte Carlo trials the heuristic takes about ten seconds and yields 2326 km, whereas Gurobi at ten seconds yields 2287 km. The paper therefore positions simulated annealing as the right tool for the seconds-scale replanning window of real dispatch operations, not as a replacement for exact methods when longer computation is available.","pith_inferences":["The paper leaves implicit that the five-second crossover is almost certainly instance- and hardware-specific; on denser networks or with tighter time windows, the relative advantage of the stochastic heuristic could shift, so the decision rule should be re-estimated on each deployment.","A testable extension is to hand the heuristic's solution to an exact solver as a warm start after the first few seconds, which would combine the rapid initial improvement with the solver's longer-horizon convergence.","Because the paper's gains come almost entirely from the 0.06-second SA stage rather than from more Monte Carlo trials, tuning the annealing schedule (end temperature and cooling rate) is likely to yield more improvement per compute second than increasing R."],"forward_implications":["A dispatcher with a five-second replanning budget gets shorter routes from the MC-SA heuristic than from a commercial MIP solver.","The SA stage costs only about 0.06 seconds per run, so almost all of the heuristic's compute budget goes to the Monte Carlo construction phase.","For planning budgets beyond roughly five seconds, the paper's data show the MIP solver producing shorter routes, so the heuristic is not a general replacement for exact methods.","Because the heuristic runs in Python and needs no commercial solver license, it lowers the software cost of short-window route replanning."],"supporting_citations":[],"fun_headline_variants":["Simulated annealing wins if you have only 5 seconds","Why annealing beats Gurobi for quick fuel route fixes","Five-second rule: when annealing outperforms Gurobi on CVRP","Fuel delivery: annealing for quick routes, Gurobi for patient ones","CVRP timing: simulated annealing for seconds, solver for minutes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Gurobi MIP model is a correct and feasible formulation of the same routing problem, so the measured runtime gap is a fair head-to-head comparison.","fun_headline_variants_meta":{"raw":{"variants":["Simulated annealing wins if you have only 5 seconds","Why annealing beats Gurobi for quick fuel route fixes","Five-second rule: when annealing outperforms Gurobi on CVRP","Fuel delivery: annealing for quick routes, Gurobi for patient ones","CVRP timing: simulated annealing for seconds, solver for minutes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000162,"raw_usage":{"total_tokens":1222,"prompt_tokens":911,"completion_tokens":311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":221}},"tokens_in":527,"tokens_out":311,"duration_ms":3428,"temperature":1.0,"reasoning_tokens":221,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:05:55.409961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check is to take the demand matrix from the reported instance and sum the demands along each Gurobi-generated route; any route whose total demand exceeds the 39,000-liter capacity shows the baseline solutions are infeasible and the runtime comparison collapses. A stronger test is to re-run the comparison with a corrected MIP model that enforces flow conservation at the first and last stops of every route and bounds total route demand above by capacity, then see whether Gurobi still trails the heuristic in the first five seconds.","supporting_citations":[],"review_version":1}