Pith. sign in

REVIEW 3 major objections 4 minor 64 references

LRM-1B: Towards Large Routing Model

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 1B-parameter neural routing model beats prior learned solvers on all tested VRP scenarios, and the gains follow a power law with model size.

desk verdict First billion-parameter routing model, but the scaling-law exponent doesn't survive contact with the data it was fitted to. read the letter →

arxiv 2507.03300 v1 pith:LKPQAYYJ submitted 2025-07-04 cs.LG

Han Li , Fei Liu , Zhenkun Wang , Qingfu Zhang This is my paper
classification cs.LG MSC 68T0790B06
keywords vehicleroutingproblemneuralcombinatorialoptimizationmodelscalinglawreinforcementlearningtransformerout-of-distributiongeneralizationlarge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds neural vehicle-routing solvers at four scales, up to 1.1 billion parameters, and trains them by reinforcement learning on a mixture of 16 routing variants, problem sizes from 50 to 200 customers, and eleven node distributions. It claims the largest model, LRM-1B, outperforms existing multi-task routing solvers on every tested scenario, including uniform and six unseen distributions at sizes from 50 to 300 nodes, while solving each instance in milliseconds. It further claims that the performance gap to a strong heuristic solver follows a power law in model size, with an average exponent $\alpha_N=0.066$, so doubling the parameter count yields about 5% relative improvement. The significance is that scaling neural routing models becomes a predictable engineering strategy rather than a set of one-off architecture tricks.

What carries the argument

The load-bearing object is a transformer-based encoder-decoder routing network trained end-to-end with reinforcement learning, using RMSNorm, SwiGLU feed-forward layers, and spectral-norm regularization for stability; the same architecture is instantiated at 1M, 5M, 40M, and 1.1B parameters. The scaling argument rests on the fitted identity $G=(N/N_c)^{-a_N}$ in log-log space, where $G$ is the performance gap to a heuristic solver and $N$ is the parameter count. The identity does the work of converting four training runs into a quantitative prediction: once the line is fit, doubling $N$ is claimed to reduce $G$ by about $2^{-a_N} \approx 5\%$.

What would settle it

Retrain the 1M, 5M, and 40M models under the exact 1B training recipe, or retrain the 1B model under the small-model recipe, and refit $G=(N/N_c)^{-a_N}$; if the fitted $a_N$ collapses toward zero or the $R^2$ drops, the power law is an artifact of the recipe change. Separately, the paper reports that on the X suite of CVRPLib, gaps are 5.05% for the 40M model and 23.4% for the 1B model; a law that cannot predict that inversion from the demand-capacity ratio $R$ is not yet a general scaling law.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the benefit of scaling a neural routing model can be described by the same kind of power law used for language models. Writing $N$ for parameter count and $G$ for the percentage gap between a learned solver and a heuristic solver, the data fit $G=(N/N_c)^{-a_N}$, with fitted exponents from 0.057 to 0.073 across six test sets and an average $a_N=0.066$; the log-log fits have $R^2$ values from 0.899 to 0.941. The same analysis yields power laws in the number of decoding trajectories, $G \propto T^{-a_T}$ with exponents near 0.1 on 100-node problems and 0.058 on OOD200, and in per-instance inference compute, $G \propto C^{-a_C}$ with an average $a_C=0.106$. On top of those curves, the 1.1 billion parameter model reaches gaps of roughly 1.9% on 50-node uniform instances, 2.9% on 100-node uniform instances, and 4.9% on 300-node uniform instances, and it wins on all 16 variants in the 300-node comparison while winning 14 of 16 at 100 nodes.

Load-bearing premise

The scaling law treats the four trained models as if only parameter count changed, but the 1B model was also trained with a different optimizer, mixed precision, a different attention kernel, and a different learning-rate schedule than the smaller models.

Editorial extensions

If this is right

  • If the power law holds beyond 1B parameters, training even larger routing models should keep reducing the performance gap on synthetic and out-of-distribution instances at roughly the same log-linear rate, until data or compute becomes the binding constraint.
  • Because the inference-compute exponent $a_C \approx 0.106$ is larger than the size exponent $a_N \approx 0.066$ on easier problems, the fitted curves imply that adding decoding trajectories or test-time augmentation is often the cheapest way to improve an existing model.
  • On the harder OOD200 setting, the size exponent delivers more relative improvement than doubling trajectories, so deployment guidance shifts toward larger models when distribution shift is expected.
  • A single 1.1B model trained on all 16 variants matches or beats specialized pretrained models on their own test sizes, implying sample-efficient transfer across routing variants.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the fitted exponent is real, one would predict that a 2B or 4B model trained on the same recipe would cut the OOD300 gap by roughly another 5% relative per doubling, an extrapolation the paper does not run.
  • The real-world CVRPLib results suggest the scaling law is conditional on matching the training demand-capacity ratio $R$; a training curriculum that varies $R$ could restore scaling on real benchmarks, which the paper does not test.
  • The confounded training recipe, with the 1B model using Adafactor, mixed precision, FlashAttention, Liger kernels, and a different learning-rate schedule than the smaller models, means the true parameter-count exponent may be lower than 0.066; isolating parameter count would require retraining the small models under the large-model recipe.
  • The same scaling methodology could be applied to other combinatorial optimization families, such as scheduling, packing, and assignment, where the routing-specific decoder would need to be replaced.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces LRM-1B, a 1.1-billion-parameter neural routing model built on the POMO framework, and evaluates it on 16 VRP variants across synthetic uniform and out-of-distribution instances. The authors report that LRM-1B outperforms existing multi-task routing solvers on these benchmarks. The paper also trains smaller models (1M, 5M, 40M) and fits power-law relationships between parameter count, inference trajectories, inference compute cost, and the performance gap relative to HGS-CVRP, claiming an average scaling exponent of approximately 0.066 for model size.

Significance. If the scaling-law claim were supported, the paper would provide useful guidance for allocating model capacity in neural routing solvers, and the 1B-parameter model would be a notable engineering contribution. The authors provide a detailed appendix, a public code link, and self-consistent comparison tables, which are strengths. However, the central quantitative claim is not established because the largest model is confounded with several training-recipe changes, the power law is fit to only four points, and the authors' own real-world benchmarks contradict the claimed universality of the scaling behavior.

major comments (3)
  1. [§3.3 and §5.1, Eq. (2)] The scaling-law fit in Eq. (2) rests on the assumption that the four models differ only in parameter count, but Section 3.3 changes the optimizer (Adam to Adafactor), adds a time-dependent learning-rate schedule, switches to FP32/BF16 mixed precision, and introduces FlashAttention and Liger Kernel exactly at the 1B point. Any of these changes could drive the observed gap reduction, so the exponent aN = 0.066 cannot be attributed to model size. The authors should run a controlled ablation, for example training the 40M model with the same Adafactor/mixed-precision/FlashAttention setup, or training the 1B model with the smaller-model recipe, before claiming a model-size scaling law.
  2. [Table 7 and Figure 4] The power law is fitted to four data points with two free parameters (aN and Nc) and no reported uncertainty or seed averages. More importantly, the 1B point is systematically off the trend implied by the 1M/5M/40M points on every test set: for example, on Uniform100 a log-log fit to the first three rows extrapolates to about a 2.4% gap at 1.1B parameters, while the reported 1B gap is 2.96%, and the same pattern appears on OOD300. The fitted exponent is therefore a compromise among four points with an off-trend largest point, not a stable scaling law. The paper should report confidence intervals, include additional intermediate sizes or repeated seeds, and check whether the residual pattern disappears.
  3. [Appendix D, Table 11] The real-world CVRPLib results contradict the broad scaling claim and the statement that LRM-1B achieves SOTA across all tested scenarios. On set X, LRM-1B has a 23.403% gap for instances with 100-300 nodes, while LRM-40M has 5.050%; LRM-1B is also worse than LRM-40M on sets F and P. The authors attribute this to overfitting to the training-time demand-capacity ratio, but this means the power-law improvement observed on synthetic distributions does not transfer to real-world demand shifts. The central claim needs to be scoped to the tested synthetic distributions, or the discrepancy must be reconciled with a mechanism that still supports a general scaling law.
minor comments (4)
  1. [Section 5.1 title] The heading contains a typo: "Performence" should be "Performance."
  2. [Abstract and Section 1] The abstract says "confirms power-law" and "cross different problem variants"; the latter should be "across," and the former would be better phrased as "provides evidence consistent with a power law" given the fitting methodology.
  3. [Eq. (2) and surrounding text] The exponent is written as aN in Eq. (2) but as αN in the text and in the sentence reporting the average value; please use consistent notation.
  4. [Figures 4-6] The figure captions do not clearly identify which marker shape and color correspond to which model size or augmentation setting; the text below the figures should be integrated into the captions for standalone readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the scaling-law result is an explicitly assumed empirical fit, and the confounded 1B training recipe is a correctness risk rather than a circular step.

full rationale

The central scaling-law claim (Section 5.1, Eq. (2)) is introduced as an assumption ('We assume a power-law relationship between model size N and performance gap G'), then tested by OLS regression on measured gaps. The 'confirmation' is a goodness-of-fit statement (R2), not a derivation of the law from its own conclusion. The statement that doubling N yields roughly 5% gap reduction is the mathematical identity G(2N)/G(N) = 2^{-aN} applied to the fitted exponent, so it is an interpretation of the fit rather than an independent prediction validated by the same data. Section 3.3 changes the 1B model's optimizer, precision, attention kernel, and LR schedule, which is a genuine confound for attributing gains to parameter count, but confounding is an experimental-validity concern, not circularity under the definitions used here. Appendix D reports that LRM-1B underperforms LRM-40M on CVRPLib (e.g., set X: 23.4% vs 5.05%), an honest limitation that weakens the scaling claim's generality but does not make the claim circular. Self-citations (MTPOMO [31], LEHD [32]) appear as baselines and related work; no load-bearing premise, uniqueness theorem, or ansatz is imported from the authors' prior work. The 'power-law confirmed' language is overstrong given four points and the confound, but that is a statistical and correctness criticism, not a circularity finding.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The claims rest on an assumed power-law form, the comparability of HGS gaps, and the attribution of performance to model size alone; the scaling-law constants are fitted, not derived.

free parameters (4)
  • Scaling exponent aN for model size = ~0.066 (average over six test sets; values per set in Figure 4)
    Fitted via OLS in log-log space from four model sizes (1M, 5M, 40M, 1B) on the same test sets used to claim the power law. No error bars or held-out sizes.
  • Normalization constant Nc in G = (N/Nc)^-aN = Varies from 6.2e12 to 6.7e18 depending on test set (Figure 4)
    Fitted intercept from the same OLS regression. The wide range means the fit has little predictive power outside the fitted sizes.
  • Scaling exponent aT for inference trajectories = 0.101, 0.098, 0.058 on Uniform100, OOD100, OOD200 (Figure 5)
    Fitted from a few trajectory counts; reported with +/- values but no procedure or confidence interval description.
  • Scaling exponent aC for compute cost = ~0.106 average (Figure 6)
    Fitted from the same runs as trajectory scaling, with compute cost derived from model size and trajectory count, so not an independent measurement.
assumptions (5)
  • domain assumption The performance gap G relative to HGS is a well-defined and comparable measure of solver quality across variants, sizes, and distributions.
    Used throughout Sections 4 and 5; assumes HGS with fixed time limits is a fair reference for all models and instances.
  • ad hoc to paper G follows a power law in model size N, G = (N/Nc)^-aN.
    Eq. (2) asserts the functional form before fitting; no derivation from architectural or optimization principles.
  • domain assumption The four trained models (1M, 5M, 40M, 1B) differ only in scale, so the performance trend is attributable to model size.
    Section 3.3 changes optimizer, precision, kernels, and LR schedule for the 1B model, confounding the size effect.
  • domain assumption Synthetic OOD distributions generated by Bossek et al. mutation operators represent the distribution shift that matters.
    Appendix C uses these for OOD tests; Appendix D's CVRPLib results show large failures on real-world instances, undermining this assumption.
  • domain assumption Training on sizes 50-200 and 16 variants plus 11 distributions is sufficient for a foundation routing model.
    Section 3.1 defines the training mixture; no evidence that it covers real-world demand-capacity ratios (R), which Appendix D identifies as a failure mode.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LRM-1B: Towards Large Routing Model." pith.science (2026). https://pith.science/paper/LKPQAYYJ

@misc{pith2026250703300,
  author       = {Pith},
  title        = {Pith review of: LRM-1B: Towards Large Routing Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LKPQAYYJ}},
  note         = {Machine review of arXiv:2507.03300}
}
read the original abstract

Vehicle routing problems (VRPs) are central to combinatorial optimization with significant practical implications. Recent advancements in neural combinatorial optimization (NCO) have demonstrated promising results by leveraging neural networks to solve VRPs, yet the exploration of model scaling within this domain remains underexplored. Inspired by the success of model scaling in large language models (LLMs), this study introduces a Large Routing Model with 1 billion parameters (LRM-1B), designed to address diverse VRP scenarios. We present a comprehensive evaluation of LRM-1B across multiple problem variants, distributions, and sizes, establishing state-of-the-art results. Our findings reveal that LRM-1B not only adapts to different VRP challenges but also showcases superior performance, outperforming existing models. Additionally, we explore the scaling behavior of neural routing models from 1M to 1B parameters. Our analysis confirms power-law between multiple model factors and performance, offering critical insights into the optimal configurations for foundation neural routing solvers.

Figures

Figures reproduced from arXiv: 2507.03300 by the authors.

Figure 1
Figure 1. Evolution of model size in neural routing models over time. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Training curves of the 40M-parameter model. (a) Gradient norms displayed in log-scale [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Percentage gap (%) of different models across VRP variants. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Scaling law between model size N and performance gap G across various test sets. represents actual data; represent power-law fits in log-log scale (i.e., log G vs. log N). The average scaling exponent is αN = 0.066, implying that doubling the model size reduces the per…
Figure 5
Figure 5. Figure 5: Scaling law between the number of inference trajectories per instance [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Scaling law between the compute cost per instance [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visualization of VRP instances with different node distributions. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Scatter plot of the performance gap (%) versus average demand–capacity ratio [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 43 canonical work pages

  1. [1]

    Physics of language models: Part 3.3, knowledge capacity scaling laws

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.3, knowledge capacity scaling laws. arXiv preprint arXiv:2404.05405, 2024

    arXiv 2024

  2. [2]

    Machine learning for combinatorial optimization: a methodological tour d’horizon

    Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 290(2):405–421, 2021

    2021

  3. [3]

    Routefinder: Towards foundation models for vehicle routing problems

    Federico Berto, Chuanbo Hua, Nayeli Gast Zepeda, André Hottung, Niels Wouda, Leon Lan, Kevin Tierney, and Jinkyoo Park. Routefinder: Towards foundation models for vehicle routing problems. In ICML 2024 Workshop on Foundation Models in the Wild, 2024. URL https:// openreview.net/forum?id=hCiaiZ6e4G. https://github.com/ai4co/routefinder

    work page 2024

  4. [4]

    Learning generalizable models for vehicle routing problems via knowledge distillation

    Jieyi Bi, Yining Ma, Jiahai Wang, Zhiguang Cao, Jinbiao Chen, Yuan Sun, and Yeow Meng Chee. Learning generalizable models for vehicle routing problems via knowledge distillation. Advances in Neural Information Processing Systems, 35:31226–31238, 2022

    work page 2022

  5. [5]

    Towards deeper deep reinforcement learning with spectral normalization

    Nils Bjorck, Carla P Gomes, and Kilian Q Weinberger. Towards deeper deep reinforcement learning with spectral normalization. Advances in Neural Information Processing Systems, 34: 8242–8255, 2021

    work page 2021

  6. [6]

    Machine learning to solve vehicle routing problems: A survey

    Aigerim Bogyrbayeva, Meraryslan Meraliyev, Taukekhan Mustakhov, and Bissenbay Daulet- bayev. Machine learning to solve vehicle routing problems: A survey. IEEE Transactions on Intelligent Transportation Systems, 2024

    work page 2024

  7. [7]

    Evolving diverse tsp instances by means of novel and creative mutation operators

    Jakob Bossek, Pascal Kerschke, Aneta Neumann, Markus Wagner, Frank Neumann, and Heike Trautmann. Evolving diverse tsp instances by means of novel and creative mutation operators. In Proceedings of the 15th ACM/SIGEVO conference on foundations of genetic algorithms , pages 58–71, 2019

    2019

  8. [8]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020

    1901

Show all 64 references
  1. [9]

    Combining reinforcement learning and constraint programming for combinatorial optimization

    Quentin Cappart, Thierry Moisan, Louis-Martin Rousseau, Isabeau Prémont-Schwarz, and Andre A Cire. Combining reinforcement learning and constraint programming for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 36...

    2021

  2. [10]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023

    2023 arXiv

  3. [11]

    The case for 4-bit precision: k-bit inference scaling laws

    Tim Dettmers and Luke Zettlemoyer. The case for 4-bit precision: k-bit inference scaling laws. In International Conference on Machine Learning, pages 7750–7774. PMLR, 2023. 10

    2023

  4. [12]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  5. [13]

    Learn to design the heuristics for vehicle routing problem

    Lei Gao, Mingxiang Chen, Qichang Chen, Ganzhong Luo, Nuoyi Zhu, and Zhixin Liu. Learn to design the heuristics for vehicle routing problem. arXiv preprint arXiv:2002.08539, 2020

    2002 arXiv

  6. [14]

    Generalization of neural combinatorial solvers through the lens of adversarial robustness

    Simon Geisler, Johanna Sommer, Jan Schuchardt, Aleksandar Bojchevski, and Stephan Günne- mann. Generalization of neural combinatorial solvers through the lens of adversarial robustness. In International Conference on Learning Representations, 2022

    2022

  7. [15]

    Data and parameter scaling laws for neural machine translation

    Mitchell A Gordon, Kevin Duh, and Jared Kaplan. Data and parameter scaling laws for neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5915–5922, 2021

    2021

  8. [16]

    Scaling laws for transfer

    Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.01293, 2021

    2021 arXiv

  9. [17]

    Training compute-optimal large language models

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  10. [18]

    Liger kernel: Efficient triton kernels for llm training

    Pin-Lun Hsu, Yun Dai, Vignesh Kothapalli, Qingquan Song, Shao Tang, Siyu Zhu, Steven Shimizu, Shivam Sahni, Haowen Ning, and Yanning Chen. Liger kernel: Efficient triton kernels for llm training. arXiv preprint arXiv:2410.10989, 2024

    arXiv

  11. [20]

    Learning to solve routing problems via distributionally robust optimization

    Yuan Jiang, Yaoxin Wu, Zhiguang Cao, and Jie Zhang. Learning to solve routing problems via distributionally robust optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9786–9794, 2022

    2022

  12. [21]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  13. [22]

    Learning collaborative policies to solve np-hard routing problems

    Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems, 34:10418–10430, 2021

    2021

  14. [23]

    Sym-nco: Leveraging symmetricity for neural combinatorial optimization

    Minsu Kim, Junyoung Park, and Jinkyoo Park. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. Advances in Neural Information Processing Systems, 35:1936– 1949, 2022

    1936

  15. [24]

    Attention, learn to solve routing problems! In International Conference on Learning Representations, 2019

    Wouter Kool, Herke van Hoof, and Max Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2019

    2019

  16. [25]

    Deep policy dynamic programming for vehicle routing problems

    Wouter Kool, Herke van Hoof, Joaquim Gromicho, and Max Welling. Deep policy dynamic programming for vehicle routing problems. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research , pages 190–213. Springer, 2022

    2022

  17. [26]

    Scaling laws for precision

    Tanishq Kumar, Zachary Ankner, Benjamin F Spector, Blake Bordelon, Niklas Muennighoff, Mansheej Paul, Cengiz Pehlevan, Christopher Ré, and Aditi Raghunathan. Scaling laws for precision. arXiv preprint arXiv:2411.04330, 2024

    2024 arXiv

  18. [27]

    Pomo: Policy optimization with multiple optima for reinforcement learning

    Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems, 33:21188–21198, 2020

    2020

  19. [28]

    An inverse scaling law for clip training

    Xianhang Li, Zeyu Wang, and Cihang Xie. An inverse scaling law for clip training. Advances in Neural Information Processing Systems, 36, 2024. 11

    2024

  20. [29]

    Cross-problem learning for solving vehicle routing problems

    Zhuoyi Lin, Yaoxin Wu, Bangjian Zhou, Zhiguang Cao, Wen Song, Yingqian Zhang, and Jayavelu Senthilnath. Cross-problem learning for solving vehicle routing problems. In The 33rd International Joint Conference on Artificial Intelligence (IJCAI-24), 2024

    2024

  21. [30]

    Why spectral normalization stabilizes gans: Analysis and improvements

    Zinan Lin, Vyas Sekar, and Giulia Fanti. Why spectral normalization stabilizes gans: Analysis and improvements. Advances in Neural Information Processing Systems, 34:9625–9638, 2021

    2021

  22. [31]

    Multi- task learning for routing problem with cross-problem zero-shot generalization

    Fei Liu, Xi Lin, Zhenkun Wang, Qingfu Zhang, Xialiang Tong, and YUAN Mingxuan. Multi- task learning for routing problem with cross-problem zero-shot generalization. InThe 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2024). Association for Computing Ma...

    2024

  23. [32]

    Neural combinatorial optimization with heavy decoder: Toward large scale generalization

    Fu Luo, Xi Lin, Fei Liu, Qingfu Zhang, and Zhenkun Wang. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. Advances in Neural Information Processing Systems, 36:8845–8864, 2023

    2023

  24. [33]

    Spectral normalization for generative adversarial networks

    Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957, 2018

    2018 arXiv

  25. [34]

    A deep reinforcement learning algorithm using dynamic attention model for vehicle routing problems

    Bo Peng, Jiahai Wang, and Zizhen Zhang. A deep reinforcement learning algorithm using dynamic attention model for vehicle routing problems. In Artificial Intelligence Algorithms and Applications: 11th International Symposium, ISICA 2019, Guangzhou, China, November 16–17, 2019,...

    2019

  26. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pa...

    2021

  27. [36]

    Scaling language models: Methods, analysis & insights from training gopher

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021

    2021 arXiv

  28. [37]

    Glu variants improve transformer

    Noam Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020

    2002 arXiv

  29. [38]

    Adafactor: Adaptive learning rates with sublinear memory cost

    Noam Shazeer and Mitchell Stern. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pages 4596–4604. PMLR, 2018

    2018

  30. [39]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    G Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. URL https://goo. gle/GeminiV1-5, 2024

    2024

  31. [40]

    Learning heuristic selection using a time delay neural network for open vehicle routing

    Raras Tyasnurita, Ender Özcan, and Robert John. Learning heuristic selection using a time delay neural network for open vehicle routing. In 2017 IEEE Congress on Evolutionary Computation (CEC), pages 1474–1481. Ieee, 2017

    2017

  32. [41]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017

    2017

  33. [42]

    Hybrid genetic search for the cvrp: Open-source implementation and swap* neighborhood

    Thibaut Vidal. Hybrid genetic search for the cvrp: Open-source implementation and swap* neighborhood. Computers & Operations Research, 140:105643, 2022

    2022

  34. [43]

    Efficient training of multi-task neural solver with multi- armed bandits

    Chenguang Wang and Tianshu Yu. Efficient training of multi-task neural solver with multi- armed bandits. arXiv preprint arXiv:2305.06361, 2023

    2023 arXiv

  35. [44]

    A game-theoretic approach for improving generalization ability of tsp solvers

    Chenguang Wang, Yaodong Yang, Congying Han, Tiande Guo, Haifeng Zhang, and Jun Wang. A game-theoretic approach for improving generalization ability of tsp solvers. In ICLR 2022 Workshop on Gamification and Multiagent Solutions, 2022

    2022

  36. [45]

    Simple statistical gradient-following algorithms for connectionist reinforce- ment learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforce- ment learning. Machine Learning, 8:229–256, 1992

    1992

  37. [46]

    Pyvrp: A high-performance vrp solver package

    Niels A Wouda, Leon Lan, and Wouter Kool. Pyvrp: A high-performance vrp solver package. INFORMS Journal on Computing, 2024. 12

    2024

  38. [47]

    Multi-decoder attention model with embedding glimpse for solving vehicle routing problems

    Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. InProceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12042–12049, 2021

    2021

  39. [48]

    Generative adversarial training for neural combinatorial optimization models, 2022

    Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Generative adversarial training for neural combinatorial optimization models, 2022. URL https://openreview.net/forum? id=9vsRT9mc7U

    2022

  40. [49]

    Spectral norm regularization for improving the generaliz- ability of deep learning

    Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generaliz- ability of deep learning. arXiv preprint arXiv:1705.10941, 2017

    2017 arXiv

  41. [50]

    Stabilizing transformer training by preventing attention entropy collapse

    Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Ramapuram, Yizhe Zhang, Jiatao Gu, and Joshua M Susskind. Stabilizing transformer training by preventing attention entropy collapse. In International Conference on Machine Learning, pages 40770– 40803. PMLR, 2023

    2023

  42. [51]

    Scaling vision transform- ers

    Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transform- ers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12104–12113, 2022

    2022

  43. [52]

    Root mean square layer normalization

    Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  44. [53]

    Examining scaling and transfer of language model architectures for machine translation

    Biao Zhang, Behrooz Ghorbani, Ankur Bapna, Yong Cheng, Xavier Garcia, Jonathan Shen, and Orhan Firat. Examining scaling and transfer of language model architectures for machine translation. In International Conference on Machine Learning, pages 26176–26192. PMLR, 2022

    2022

  45. [54]

    Learning to solve travelling salesman problem with hardness-adaptive curriculum

    Zeyang Zhang, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning to solve travelling salesman problem with hardness-adaptive curriculum. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 9136–9144, 2022

    2022

  46. [55]

    A hybrid of deep reinforcement learning and local search for the vehicle routing problems

    Jiuxia Zhao, Minjia Mao, Xi Zhao, and Jianhua Zou. A hybrid of deep reinforcement learning and local search for the vehicle routing problems. IEEE Transactions on Intelligent Transporta- tion Systems, 22(11):7208–7218, 2021

    2021

  47. [56]

    Towards omni-generalizable neural methods for vehicle routing problems

    Jianan Zhou, Yaoxin Wu, Wen Song, Zhiguang Cao, and Jie Zhang. Towards omni-generalizable neural methods for vehicle routing problems. In International Conference on Machine Learning, pages 42769–42789. PMLR, 2023

    2023

  48. [57]

    Mvmoe: Multi-task vehicle routing solver with mixture-of-experts

    Jianan Zhou, Zhiguang Cao, Yaoxin Wu, Wen Song, Yining Ma, Jie Zhang, and Chi Xu. Mvmoe: Multi-task vehicle routing solver with mixture-of-experts. In International Conference on Machine Learning (ICML), 2024

    2024

  49. [58]

    Scaling law for document neural machine translation

    Zhang Zhuocheng, Shuhao Gu, Min Zhang, and Yang Feng. Scaling law for document neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8290–8303, 2023. 13 A Model Details Each problem instance is represented by a node set V =...

    2023

  50. [59]

    A customer must be served only once, and if the last visited node was the depot, the next move cannot immediately return to the depot (this prevents trivial loops)

    No repeated visits. A customer must be served only once, and if the last visited node was the depot, the next move cannot immediately return to the depot (this prevents trivial loops)

  51. [60]

    In problems without an open-route option, every tour segment must eventually return to the depot within both its time-window and distance limits

    Return requirements for closed routes. In problems without an open-route option, every tour segment must eventually return to the depot within both its time-window and distance limits. If visiting a candidate customer would cause the return trip (including service time) to exc...

  52. [61]

    Whenever time windows apply, a customer cannot be chosen if the earliest possible arrival (plus service) would fall after its window closes

    Individual time windows. Whenever time windows apply, a customer cannot be chosen if the earliest possible arrival (plus service) would fall after its window closes

  53. [62]

    When backhaul visits are required, they are deferred until all linehaul services are completed

    Backhaul ordering. When backhaul visits are required, they are deferred until all linehaul services are completed. Thus, any backhaul customer is masked out as long as there remain unserved linehaul customers

  54. [63]

    A node is only feasible if its demand can be loaded on the vehicle without exceeding the remaining capacity (for pickups) or the available backhaul capacity (for drop-offs)

    Capacity checks. A node is only feasible if its demand can be loaded on the vehicle without exceeding the remaining capacity (for pickups) or the available backhaul capacity (for drop-offs). The above description covers the model architecture without spectral normalization. Wh...

  55. [64]

    Every node coordinate is drawn from the uniform distribution ⃗ xi ∼ U (0, 1)2

    Uniform. Every node coordinate is drawn from the uniform distribution ⃗ xi ∼ U (0, 1)2

  56. [65]

    This distribution is parameterized by the number of clusters m and a scale factor c

    Gaussian Mixture. This distribution is parameterized by the number of clusters m and a scale factor c. For an instance with a Gaussian mixture distribution, the depot node is sampled as ⃗ x0 ∼ U (0, 1)2, and m cluster centers are sampled from U (0, c)2. Then the remaining M − ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.