REVIEW 3 major objections 6 minor 15 references
Primal-Dual Neural Algorithmic Reasoning
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A graph neural network can replicate a primal-dual approximation algorithm exactly, then use optimal solutions from small instances to surpass it on large and out-of-distribution graphs.
desk verdict Solid empirical NAR framework for NP-hard problems; the theoretical replication proof has a real gap but the architecture is a genuine step forward. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the bipartite primal-dual graph together with a message-passing GNN built on it, in an encoder-processor-decoder arrangement. The processor computes $h_T = \min_{e\in N(T)} g_e(h_e, h_{d_e})$ for each dual node, which represents the increment $\delta_T$, and then $h_e = g_u(h_e, \sum_{T\ni e} h_T)$ for each primal node, which represents the residual update $r_e - \sum_{T\ni e}\delta_T$. A virtual node with a min aggregation and broadcast implements the uniform-increase variant. The proof of Theorem 4.1 constructs the MLP weights so that the composition evaluates $\min_{e\in N(T)} r_e/d_e$ and the residual subtraction exactly, in 8 layers, so that the network's per-step outputs match Algorithm 1's hints; training then combines noisy-teacher-forced algorithm hints with a terminal loss against optimal solutions of small instances.
What would settle it
Run the Appendix B 8-layer parameter configuration on a hitting-set input where some active element has residual weight larger than its degree (e.g., $r_e=2$, $d_e=1$, so $\ln(r_e/d_e)>0$) and compare every output with Algorithm 1; any mismatch at that element disproves Theorem 4.1 as written. A complementary check is to train PDNAR without optimal-solution supervision and see whether the trained model still reproduces Algorithm 1 on instances whose residual-to-degree ratios exceed 1.
Extended reading notes
Core claim
PDNAR encodes a minimum hitting set instance as a bipartite graph, with elements (the primal variables) on one side and sets (the dual variables) on the other, and runs a recurrent message-passing processor that mirrors one round of the primal-dual approximation algorithm: a min aggregation over each set's elements computes the dual increment $\delta_T$ from residual weights $r_e$ and degrees $d_e$, and a sum aggregation subtracts the aggregated increments from the residuals. A virtual node connected to all dual nodes implements the optional uniform-increase rule. Theorem 4.1 asserts that some parameter configuration of an 8-layer PDNAR reproduces, at every timestep, the intermediate quantities $(x^{(t)}, r^{(t)}, \delta^{(t)}, \Delta^{(t)})$ of Algorithm 1, so the model inherits the $\alpha$-approximation guarantee for hitting set (Corollary 4.2). The paper's empirical claim is that adding a loss term against optimal solutions from integer-programming solvers on small instances lets the trained model outperform the very approximation algorithm it was designed to simulate, with strong generalization to larger and out-of-distribution graphs.
Load-bearing premise
The load-bearing premise is that the constructed MLPs compute exact arithmetic on log-transformed residual weights and degrees, in particular that $\mathrm{ELU}(\ln(r_e/d_e) + 1) = r_e/d_e$, which holds only when the activation input is non-positive; for active elements with $r_e > d_e$, the identity fails, and if the construction cannot be modified to handle those inputs exactly, Theorem 4.1 collapses as stated.
Editorial extensions
If this is right
- If Theorem 4.1 holds, a PDNAR with the constructed parameters exactly simulates Algorithm 1 on hitting set and thus inherits its $\alpha$-approximation guarantee with $\alpha = \max_{T\in\mathcal{T}}|T|$ and $O(|E|)$ termination rounds.
- The same alignment yields a $2/(1-\epsilon)$-approximation for weighted vertex cover (and hence set cover), so the framework extends neural algorithmic reasoning to NP-hard problems rather than only polynomial-time-solvable ones.
- Training with optimal-solution supervision lets PDNAR consistently beat the approximation algorithm it mimics, with model-to-algorithm weight ratios below 1.0 across graph sizes 16 to 1024 and across out-of-distribution families such as random, star, lobster, and 3-connected planar graphs.
- The resulting solutions warm-start commercial mixed-integer programming solvers faster than both default initialization and algorithm-generated warm starts, at roughly ten times lower compute cost.
- The same pretrained processor, combined with a learned feature encoder, produces embeddings that improve node-classification accuracy on real-world airport networks compared with degree, positional, and random-walk embedding baselines.
Reading between the lines
- Because the architecture only ever sees optimal labels from small instances, a direct test of the paper's mechanism is to vary the hardness of those small instances and measure the performance gap against the approximation algorithm; the paper's own hypothesis is that harder small instances amplify the gain.
- The hitting-set formulation covers many primal-dual problems beyond the three tested, including Steiner tree and feedback vertex set; whether the same bipartite processor transfers to those tasks is an open empirical question the paper does not resolve.
- The proof of exact replication in Appendix B assumes exact arithmetic in the logarithmic domain; if the ELU identity fails for positive arguments, the 8-layer theorem likely needs extra layers or a different activation, which would leave the empirical claims intact while changing the theoretical statement.
- The warm-start experiments suggest a general recipe — train a neural reasoner on small instances, then use its predictions to initialize an exact solver on large ones — whose scope could be tested by measuring end-to-end solve-time improvements on other NP-hard benchmark families.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces Primal-Dual Neural Algorithmic Reasoning (PDNAR), a neural algorithmic reasoning framework for learning primal-dual approximation algorithms for NP-hard problems including minimum vertex cover, minimum set cover, and minimum hitting set. The architecture represents an instance as a bipartite graph between primal variables (elements) and dual variables (sets), with recurrent message passing, an optional virtual node for uniform dual increase, and decoders for the inclusion decision, residual weights, and dual increments. The paper claims that (i) an 8-layer PDNAR can exactly replicate the general primal-dual approximation algorithm and inherit its approximation guarantees (Theorem 4.1, Corollaries 4.2 and 4.3); (ii) adding optimal solutions from small instances lets the trained model outperform the approximation algorithm on larger and out-of-distribution graphs (Tables 1 and 2); and (iii) the pretrained model is practical for airport node-classification embeddings (Table 3) and for warm-starting Gurobi on larger vertex-cover instances (Table 4).
Significance. If the theoretical and empirical claims hold, this is a valuable contribution to neural algorithmic reasoning: it extends NAR from polynomial-time exact algorithms to NP-hard approximation algorithms, introduces a clean bipartite primal-dual inductive bias, and proposes a mechanism—optimal supervision on small instances—by which a neural model can beat the algorithm it mimics while retaining algorithmic generalization. The empirical evaluation is comparatively broad: three tasks, scaling to 1024 nodes, several out-of-distribution graph families, real-world airport datasets, solver warm-starting, and RB benchmarks in the appendix. The paper also ships a public code repository and reports means over 10 seeds. The claim of outperforming the simulated algorithm is substantive and, if reproducible, would distinguish this work from standard NAR simulation.
major comments (3)
- [Appendix B / Theorem 4.1] The proof of Theorem 4.1 relies on two ELU identities that hold only for non-positive arguments: it asserts ELU(ln(r_e^{t-1} - ln d_e^{t-1})) + 1 = r_e^{t-1}/d_e^{t-1}, and similarly ELU(ln r_e^{t-1}) + 1 = r_e^{t-1}. Both require the relevant logarithms to be non-positive, i.e. r_e <= d_e and r_e <= 1. Theorem 4.1 states arbitrary non-negative weights, so an instance with r_e > 1 is within the theorem's domain. On the concrete instance E = {e1, a1, a2, a3}, T_i = {e1, a_i} for i = 1, 2, 3, with w(e1) = 5 and w(a_i) = 1, Algorithm 1 outputs {a1, a2, a3}, while the constructed network's hidden state for e1 updates to (ln 5 + 1) - 3, which is negative, causing the decoder to include e1 as well; the final cover differs. Thus the exact-replication theorem and the inherited alpha-approximation guarantee in Corollary 4.2 are not established as stated. A repair likely requires an explicit bounded-weight and positive-degree assumption, or a substantially different construction that computes exp and log exactly on the full positive range.
- [Corollary 4.3] Corollary 4.3 is stated without proof. The surrounding text says that the MVC guarantee follows from adapting Khuller et al.'s 2/(1 - epsilon)-approximation algorithm, but Appendix B does not contain the adaptation or a proof that the same 8-layer construction works for the MVC-specific algorithm with its epsilon threshold. Since this corollary is one of the paper's advertised theoretical contributions, it needs either a proof or an explicit statement that it is a direct corollary of a repaired Theorem 4.1 under additional assumptions.
- [Section 5.1, Table 1] The central empirical claim that PDNAR outperforms the approximation algorithm is reported only as the ratio w_model/w_algo, not as an approximation ratio relative to optimal solutions. On the 16-node training distribution, an MVC ratio of 0.943 indicates that the algorithm's own solution is on average within about 6 percent of the model's (and presumably of optimal), but the paper never reports the optimality gaps for the test sets. This matters especially for the MHS row, where the ratios are all near or above 0.98 and reach 1.027 at 1024 nodes, meaning PDNAR is at best approximately matching the algorithm at the largest size. Reporting absolute approximation ratios for at least one size per task would make the 'outperforms the algorithm' claim more interpretable and would clarify the practical significance of the improvements.
minor comments (6)
- [Appendix B] The proof conflates the ELU activation with exact exp/log computation and does not account for the positive-argument branch of ELU. The construction should state the intended arithmetic precision and the domain restrictions, or use a different activation that is exactly exp on the needed range.
- [Section 4.1, decoder definition] The text notes that the sigmoid can approximate the threshold function o(x) 'to arbitrary precision by adjusting its temperature,' but the proof then uses o(x) directly. This approximation step is not formalized; the proof should either include the temperature parameter in the construction or specify the approximation error explicitly.
- [Section 4.3] Theorem 4.1 says the construction uses 8 layers, but Appendix B never counts the layers of the encoder, processor, and decoders. Adding this count would make the claim easier to verify.
- [Table 1 caption] The caption says 'Smaller is better for minimization tasks,' but the baselines GIN and GAT are trained only on optimal labels and are not NAR models; the text should clarify that the 'No algo' ablation is the appropriate comparison for the contribution of intermediate algorithmic supervision.
- [Table 4] The warm-start table reports only mean solve times without standard deviations or the number of instances that reached the time limit; reporting these would help assess whether the 5-20 second differences are significant.
- [Appendix H, Table 8] The comparison with EGN and Meta-EGN would benefit from a caption or text stating the baseline training details, since the main text emphasizes that baselines were trained on 4000 RB graphs while PDNAR was trained on 1000 B-A 16-node graphs.
Circularity Check
No significant circularity: Theorem 4.1 is an explicit MLP construction and empirical gains are supervised by external solver labels.
full rationale
PDNAR's central theoretical claim is an existence-by-construction result, not a fitted prediction: Appendix B specifies explicit MLP weights (e.g., W_fr=[1,0,...], W_ge=[1,...,-1,...], W_gu=[1,...,-1,...]) and shows that the recurrent GNN computes delta_T = min_{e in N(T)} r_e/d_e and r_e <- r_e - sum_{T contains e} delta_T, which are exactly Algorithm 1's update rules. The theorem uses the algorithm's recurrences as the target of the simulation, not as an input that is then renamed an output; no parameter is fitted to data and reported as a prediction. The empirical outperformance in Tables 1, 2, 4, and 8 is supervised by optimal solutions from HiGHS/Gurobi on small training instances and evaluated on held-out larger or OOD graphs, so the benchmark is external to the model's own outputs. Self-citations (Hayderi et al. 2024 for B-A bipartite data generation; He et al. 2022 and Xhonneux et al. 2024 cited in related work) are not load-bearing. I did find a correctness gap, but not a circularity: the Appendix B equalities ELU(ln(r/d))+1 = r/d and ELU(ln r)+1 = r require r/d <= 1 and r <= 1, while the theorem allows arbitrary nonnegative weights; Corollary 4.3 is also stated without a proof. These are completeness and soundness issues, not instances of the derivation reducing to its own inputs, and Section F explicitly disclaims worst-case guarantees for the trained model.
Assumptions & free parameters
free parameters (6)
- epsilon =
0.1
- sigmoid decision threshold =
0.5
- noisy teacher forcing probability =
0.5
- training graph size =
16 nodes
- preferential attachment parameter b =
5
- hidden dimension =
32
assumptions (5)
- standard math LP duality and weak duality
- standard math Approximation guarantees of the general hitting-set primal-dual algorithm
- domain assumption MLPs with ELU can compute the required arithmetic exactly
- domain assumption Positive residual weights and degrees for all active elements
- domain assumption Exact sigmoid threshold can implement the step function
invented entities (1)
-
Virtual node z
Cite this review
Pith. "Pith review of Primal-Dual Neural Algorithmic Reasoning." pith.science (2026). https://pith.science/paper/FV67E3QS
@misc{pith2026250524067,
author = {Pith},
title = {Pith review of: Primal-Dual Neural Algorithmic Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/FV67E3QS}},
note = {Machine review of arXiv:2505.24067}
}
read the original abstract
Neural Algorithmic Reasoning (NAR) trains neural networks to simulate classical algorithms, enabling structured and interpretable reasoning over complex data. While prior research has predominantly focused on learning exact algorithms for polynomial-time-solvable problems, extending NAR to harder problems remains an open challenge. In this work, we introduce a general NAR framework grounded in the primal-dual paradigm, a classical method for designing efficient approximation algorithms. By leveraging a bipartite representation between primal and dual variables, we establish an alignment between primal-dual algorithms and Graph Neural Networks. Furthermore, we incorporate optimal solutions from small instances to greatly enhance the model's reasoning capabilities. Our empirical results demonstrate that our model not only simulates but also outperforms approximation algorithms for multiple tasks, exhibiting robust generalization to larger and out-of-distribution graphs. Moreover, we highlight the framework's practical utility by integrating it with commercial solvers and applying it to real-world datasets.
Figures
Reference graph
Works this paper leans on
-
[1]
15 Primal-Dual Neural Algorithmic Reasoning
Base case (t = 0): This is true because the inputs M(0) Θ (T , E, w) = (0, {we : e ∈ E}, 0, 0) = (x(0), r(0), δ(0), ∆(0)). 15 Primal-Dual Neural Algorithmic Reasoning
-
[2]
Induction step ( t >0): To formulate the strong induction hypothesis, let (x(t′), r(t′), δ(t′), ∆(t′)) be the intermediate quantites computed by Algorithm 1 for each timestep t′ ∈ [0, ..., t− 1], assume M(t′) Θ (T , E, w) = (x(t′), r(t′), δ(t′), ∆(t′)). We now prove that M(t) Θ (T , E, w) = (x(t), r(t), δ(t), ∆(t)). The inputs for the tth step of our recu...
work page 2018
-
[4]
URL https://proceedings.mlr.press/ v231/dudzik24a.html. Dwivedi, V . P., Joshi, C. K., Luu, A. T., Laurent, T., Ben- gio, Y ., and Bresson, X. Benchmarking graph neural networks, 2022a. URL https://arxiv.org/abs/ 2003.00982. Dwivedi, V . P., Luu, A. T., Laurent, T., Bengio, Y ., and Bres- son, X. Graph neural networks with learnable structural and positio...
arXiv 2003
-
[5]
Erdos Goes Neural: an Unsupervised Learning Framework for Combinatorial Optimization on Graphs
URL https://proceedings.mlr.press/ v231/jurss24a.html. Karalias, N. and Loukas, A. Erdos Goes Neural: an Unsuper- vised Learning Framework for Combinatorial Optimiza- tion on Graphs, March 2021. URL http://arxiv. org/abs/2006.10643. arXiv:2006.10643. Khot, S. and Regev, O. Vertex cover might be hard to approximate to within 2- ε. Journal of Computer and S...
work page Pith review arXiv 2021
-
[7]
Numeroso, D., Bacciu, D., and Veli ˇckovi´c, P
URL https://openreview.net/forum? id=tRP0Ydz5nN. Numeroso, D., Bacciu, D., and Veli ˇckovi´c, P. Dual algo- rithmic reasoning. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https: //openreview.net/forum?id=hhvkdRdWt1F. Ribeiro, L. F., Saverese, P. H., and Figueiredo, D. R. struc2vec: Learning node representations from s...
arXiv 2023
-
[10]
Xu, K., Zhang, M., Li, J., Du, S
URL https://openreview.net/forum? id=rJxbJeHFPS. Xu, K., Zhang, M., Li, J., Du, S. S., Kawarabayashi, K.-I., and Jegelka, S. How neural networks extrapo- late: From feedforward to graph neural networks. In International Conference on Learning Representations,
-
[11]
URL https://openreview.net/forum? id=UH-cmocLJC. Yau, M., Karalias, N., Lu, E. H., Xu, J., and Jegelka, S. Are graph neural networks optimal approximation algo- rithms? In The Thirty-eighth Annual Conference on Neu- ral Information Processing Systems, 2024. URL https: //openreview.net/forum?id=SxRblm9aMs. Zhou, H., Nova, A., Larochelle, H., Courville, A.,...
arXiv 2024
-
[12]
cc/paper_files/paper/2021/file/ f6a673f09493afcd8b129a0bcf1cd5bc-Paper
URL https://proceedings.neurips. cc/paper_files/paper/2021/file/ f6a673f09493afcd8b129a0bcf1cd5bc-Paper. pdf. Łukasz Kaiser and Sutskever, I. Neural gpus learn al- gorithms, 2016. URL https://arxiv.org/abs/ 1511.08228. 13 Primal-Dual Neural Algorithmic Reasoning A. Additional details of vertex cover and set cover A.1. Primal-dual pair: vertex cover and ed...
arXiv 1994
Show all 15 references
-
[15]
The results are summarized in Table 8
and Meta-EGN (Wang & Li, 2023), two powerful NCO baselines for these benchmarks, as well as two algorithms (the primal-dual approximation algorithm and the greedy algorithm) and Gurobi. The results are summarized in Table 8. Table 8. Approximation ratio of solutions compared w...
2024
-
[2017]
Kruskal, J
URL https://openreview.net/forum? id=SJU4ayYgl. Kruskal, J. B. On the shortest spanning subtree of a graph and the traveling salesman problem. Proceedings of the American Mathematical society, 7(1):48–50, 1956. Kujawa, Z., Poole, J., Georgiev, D., Numeroso, D., and Liò, P. Neu...
1956 arXiv
-
[2018]
Veliˇckovi´c, P., Ying, R., Padovano, M., Hadsell, R., and Blundell, C
URL https://openreview.net/forum? id=rJXMpikCZ. Veliˇckovi´c, P., Ying, R., Padovano, M., Hadsell, R., and Blundell, C. Neural execution of graph algorithms. In International Conference on Learning Representations,
-
[2020]
URL https://openreview.net/forum? id=SkgKO0EtvS. Wang, H. and Li, P. Unsupervised Learning for Com- binatorial Optimization Needs Meta-Learning, Jan- uary 2023. URL http://arxiv.org/abs/2301. 03116. arXiv:2301.03116. Wenkel, F., Cantürk, S., Horoi, S., Perlmutter, M., and Wolf...
2023 arXiv
-
[2021]
cc/paper_files/paper/2021/file/ 82e9e7a12665240d13d0b928be28f230-Paper
URL https://proceedings.neurips. cc/paper_files/paper/2021/file/ 82e9e7a12665240d13d0b928be28f230-Paper. pdf. Dudzik, A. J., von Glehn, T., Pascanu, R., and Veliˇckovi´c, P. Asynchronous algorithmic alignment with cocy- cles. In Villar, S. and Chamberlain, B. (eds.), Pro- ceed...
2021
-
[2023]
Borodin, A., Karavasilis, C., and Pankratov, D
URL https://proceedings.mlr.press/ v202/bevilacqua23a.html. Borodin, A., Karavasilis, C., and Pankratov, D. An exper- imental study of algorithms for online bipartite match- ing, 2018. URL https://arxiv.org/abs/1808. 04863. Bounsi, W., Ibarz, B., Dudzik, A., Hamrick, J. B., Ma...
2018 arXiv
-
[2024]
Deac, A., Bacon, P.-L., and Tang, J
URL https://openreview.net/forum? id=aA2326y3hf. Deac, A., Bacon, P.-L., and Tang, J. Graph neural induction of value iteration, 2020. URL https://arxiv.org/ abs/2009.12604. Deac, A.-I., Veli ˇckovi´c, P., Milinkovic, O., Bacon, P.-L., Tang, J., and Nikolic, M. Neural algorith...
2020 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.