Pith. sign in

REVIEW 2 major objections 1 minor 2 cited by

Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning

T0 review · 2 major / 1 minor · reviewed 2026-05-25 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Placeto learns reinforcement learning policies for device placement that generalize to unseen computation graphs without retraining.

desk verdict Placeto's iterative RL plus label-free graph embeddings is a sensible shift from per-graph optimization, but the generalization results look under-supported from the abstract alone. read the letter →

arxiv 1906.08879 v1 pith:N27DV3RP submitted 2019-06-20 cs.LG cs.DCstat.ML

Ravichandra Addanki , Shaileshh Bojja Venkatakrishnan , Shreyan Gupta , Hongzi Mao , Mohammad Alizadeh This is my paper · ORCID
classification cs.LGcs.DCstat.ML
keywords deviceplacementreinforcementlearningdistributedmachinecomputationgraphsgraphembeddingsgeneralizablepoliciesneuralnetworktraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Device placement decides how to split neural network training across multiple devices, and this choice strongly affects training speed. Prior reinforcement learning methods train a separate policy for each computation graph, which becomes costly when graphs change. Placeto instead trains a single policy that iteratively improves placements and represents graphs via embeddings that ignore specific node labels. This design lets the same policy handle any graph from a given family. Experiments show the policy reaches competitive or better placements after up to 6.1 times fewer training steps than earlier approaches.

What carries the argument

The policy that performs iterative placement improvements guided by graph embeddings capturing structure without node labels.

What would settle it

A new graph from the training family on which the fixed Placeto policy produces placements measurably worse than a policy retrained from scratch on that graph would falsify the generalization claim.

Watch

Extended reading notes

Core claim

Placeto is a reinforcement learning method that represents device placement as a sequence of iterative improvements and encodes computation graphs with label-independent graph embeddings. These choices produce a policy that trains efficiently on one family of graphs and then applies directly to new graphs from the same family, eliminating per-graph retraining while matching or exceeding the placement quality of prior specialized methods after up to 6.1 times fewer training steps.

Load-bearing premise

Graph embeddings capture the structural features needed for good placements even when node labels are ignored.

Editorial extensions

If this is right

  • A single policy trained on a graph family can be reused on any unseen graph from that family.
  • Training cost for placement search drops by up to a factor of 6.1 compared with retraining per graph.
  • The overhead of retraining from scratch for each new graph disappears.
  • Placement quality remains on par with or better than specialized per-graph methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The iterative-improvement framing may transfer to other sequential decision problems on graphs where one-shot prediction is brittle.
  • If graph families can be defined more broadly, the same policy could serve as a starting point for hardware configurations not seen during training.
  • Production pipelines that encounter evolving model architectures could avoid repeated expensive searches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper presents Placeto, a reinforcement learning approach to device placement for distributed neural network training. Unlike prior per-graph methods, Placeto learns generalizable policies by framing placement as iterative improvements and using graph embeddings that avoid reliance on node labels for indexing. The central claims are that this yields up to 6.1x fewer training steps to reach placements on par with or better than prior work, and that a policy trained on one family of graphs can be applied without retraining to unseen graphs from the same family.

Significance. If the generalization results are robust, the work would meaningfully lower the practical cost of RL-based placement by removing the need for per-graph retraining, which is a key limitation of existing methods. The iterative formulation and label-free graph embeddings represent a reasonable technical direction for handling variable computation graphs. The empirical focus means impact hinges on verifiable transfer performance across graph families.

major comments (2)
  1. [Approach] Abstract and approach description: The generalization claim (that embeddings capture placement-relevant structure such as operator sizes, dependencies, and communication volumes independently of node labels, enabling zero-shot transfer) is load-bearing for the no-retraining benefit and the 6.1x step reduction, yet the manuscript supplies no ablation, embedding visualization, or invariance test showing that embeddings encode transferable signals rather than training-graph idiosyncrasies.
  2. [Experiments] Experiments section: The abstract asserts empirical gains and generalization but the provided text contains no experimental details on baselines, number of runs, variance, specific graph families, how 'unseen graphs' are constructed, or statistical verification of transfer; without these the central claims cannot be assessed.
minor comments (1)
  1. [Abstract] The abstract's phrasing 'applicable to any graph' is imprecise given the 'same family' qualifier later in the text; clarify the scope of generalization.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The comments highlight important areas where the manuscript can be strengthened to better support the central claims on generalization and empirical performance. We address each major comment below and commit to revisions that will incorporate additional evidence and details without altering the core technical contributions.

read point-by-point responses

  1. Referee: [Approach] Abstract and approach description: The generalization claim (that embeddings capture placement-relevant structure such as operator sizes, dependencies, and communication volumes independently of node labels, enabling zero-shot transfer) is load-bearing for the no-retraining benefit and the 6.1x step reduction, yet the manuscript supplies no ablation, embedding visualization, or invariance test showing that embeddings encode transferable signals rather than training-graph idiosyncrasies.

    Authors: We agree that explicit evidence for the transferability of the learned embeddings would strengthen the generalization claim. The current manuscript motivates the label-free graph embeddings through the iterative improvement formulation and reports successful zero-shot transfer results, but does not include ablations isolating the embedding component or visualizations. In the revision we will add (i) an ablation comparing policies with and without the graph embedding module on transfer tasks and (ii) t-SNE visualizations of embeddings from training and unseen graphs to illustrate that structural features are captured independently of node identities. These additions will directly address the concern while preserving the existing experimental narrative. revision: yes

  2. Referee: [Experiments] Experiments section: The abstract asserts empirical gains and generalization but the provided text contains no experimental details on baselines, number of runs, variance, specific graph families, how 'unseen graphs' are constructed, or statistical verification of transfer; without these the central claims cannot be assessed.

    Authors: The referee is correct that the submitted manuscript text omits key experimental details required to evaluate the claims. The full experimental section (which was partially truncated in the version reviewed) describes comparisons against prior per-graph RL methods and non-RL baselines on families including Inception, ResNet, and LSTM variants, with unseen graphs drawn from the same distribution. In the revision we will expand the Experiments section to explicitly report: the exact baselines, number of independent runs (5–10 per setting), standard deviations, precise construction of unseen graphs (random sampling from the same family with held-out topologies), and statistical significance tests for the reported speedups and transfer performance. This will make the empirical support fully verifiable. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; empirical RL claims rest on experimental comparisons, not self-referential derivations

full rationale

The paper presents Placeto as an RL method that learns iterative placement policies via graph embeddings (without node labels) and evaluates generalization empirically on unseen graphs from the same family. No equations, predictions, or uniqueness theorems are shown that reduce the claimed 6.1x training-step reduction or generalization benefit to fitted inputs or self-citations by construction. The central claims are supported by direct experimental benchmarks against prior methods rather than internal redefinitions or load-bearing self-citations. This is a standard empirical ML paper with no detectable circular steps.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The method implicitly rests on standard RL assumptions and the effectiveness of graph embeddings for structure capture.

assumptions (1)
  • domain assumption Reinforcement learning can optimize device placement policies for neural network graphs
    Core premise of the approach; invoked by framing the problem as an RL task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning." pith.science (2026). https://pith.science/paper/N27DV3RP

@misc{pith2026190608879,
  author       = {Pith},
  title        = {Pith review of: Placeto: Learning Generalizable Device Placement Algorithms for Distributed Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N27DV3RP}},
  note         = {Machine review of arXiv:1906.08879}
}
read the original abstract

We present Placeto, a reinforcement learning (RL) approach to efficiently find device placements for distributed neural network training. Unlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph. We propose two key ideas in our approach: (1) we represent the policy as performing iterative placement improvements, rather than outputting a placement in one shot; (2) we use graph embeddings to capture relevant information about the structure of the computation graph, without relying on node labels for indexing. These ideas allow Placeto to train efficiently and generalize to unseen graphs. Our experiments show that Placeto requires up to 6.1x fewer training steps to find placements that are on par with or better than the best placements found by prior approaches. Moreover, Placeto is able to learn a generalizable placement policy for any given family of graphs, which can then be used without any retraining to predict optimized placements for unseen graphs from the same family. This eliminates the large overhead incurred by prior RL approaches whose lack of generalizability necessitates re-training from scratch every time a new graph is to be placed.

Figures

Figures reproduced from arXiv: 1906.08879 by the authors.

Figure 1
Figure 1. MDP structure of Placeto’s device placement task. At each step, Placeto updates a placement for a node (shaded) in the computation graph. These incremental improvements amount to the final placement at the end of an MDP episode. Mirhoseini et.al. [9]. For ease of notation, henceforth we will use G(V, E) to denote the graph of op groups. Here V is the set of op groups and E is set of data communication edges between … view at source ↗
Figure 2
Figure 2. Placeto’s RL framework for device placement. The state input to the agent is represented as a DAG with features (such as computation types, current placement) attached to each node. The agent uses a graph neural network to parse the input and uses a policy network to output a probability distribution over devices for the current node. The incremental reward is the difference between runtimes of consecutive placement… view at source ↗
Figure 3
Figure 3. Placeto’s graph embedding approach. It maps raw features associated with each op group to the device placement action. (a) Example computation graph of op groups. The shaded node is taking the current placement action. (b) Two-way message passing scheme applied to all nodes in the graph. (c) Partitioning the message passed (denoted as bold) op groups. (d) Taking a placement action on two candidate devices for the cu… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Optimized placement across 4 GPUs for a 4-layered NMT model with attention found by Placeto. The top LSTM layers correspond to encoder and the bottom layers to decoder. All the layers are unrolled to a maximum sequence length of 32. Each color represents a different GP…
Figure 5
Figure 5. Figure 5: CDFs of runtime of placements found by the different schemes for test graphs from (a), (d) nmt (b), (e) ptb and (c), (f) cifar10 datasets. Top row of figures ((a), (b), (c)) correspond to Placeto and bottom row ((d), (e), (f)) to RNN-based approach. Placeto Zero-Shot p…
Figure 6
Figure 6. Figure 6: Sample graphs from cifar-10 dataset. Each color represents a different GPU in the optimized placement. Size of the node indicates its compute cost and the edge thickness visualizes the communication cost. The above graphs exhibit a wide range of structure and connectiv…
Figure 7
Figure 7. Figure 7: Few of the recurrent cells used to generate the sequence based models in ptb dataset. Each color indicates a different operation from the following: Identity (Id), Sigmoid (Sig.), Tanh (tanh), ReLU (ReLU). x[t] is the input to the cell and the final add operation is it…
Figure 8
Figure 8. Figure 8: (a) Cumulative Episodic rewards when Placeto is trained with and without intermediate rewards on NMT model. Having intermediate rewards within an episode as opposed to a single reward at the end leads to a lower variance in the runtime. (b) Runtime improvement observed…
Figure 9
Figure 9. Figure 9: RNN-based approach exhibits near identical learning curve when reward signal is from a simulator or directly from measurements on real machines. size reaches a desired value N or alternatively until there are no more nodes with cost below a certain threshold C. The mer…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DOPPLER: Dual-Policy Learning for Device Assignment in Asynchronous Dataflow Graphs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    DOPPLER trains two cooperating neural policies, one that orders graph operations and one that maps them to GPUs, to reduce execution time in asynchronous work-conserving multi-GPU systems.

  2. Generalised Rate Control Approach For Stream Processing Applications

    cs.NI 2025-06 conditional novelty 5.0 of 10

    A graph neural network-based reinforcement learning controller for source emission rate improves throughput by up to 13.5% and latency by up to 30% across three simulated stream topologies.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://aws.amazon.com/ec2/instance-types/, 2018

    Ec2 instance types. https://aws.amazon.com/ec2/instance-types/, 2018. Accessed: 2018-10-19

    work page 2018

  2. [2]

    Andrychowicz, F

    M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, O. P. Abbeel, and W. Zaremba. Hindsight experience replay. In Advances in Neural Information Processing Systems, pages 5048–5058, 2017

    work page 2017

  3. [3]

    P. W. Battaglia et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018

    work page Pith review arXiv 2018

  4. [4]

    M. M. Bronstein, J. Bruna, Y . LeCun, A. Szlam, and P. Vandergheynst. Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42, 2017

    work page 2017

  5. [5]

    Y . Gao, L. Chen, and B. Li. Spotlight: Optimizing device placement for training deep neural networks. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning , vol- ume 80 of Proceedings of Machine Learning Research, pages 1676–1684, Stockholmsmässan, Stockholm Sweden, 10–15 Jul 2018. PMLR

    work page 2018

  6. [6]

    Greensmith, P

    E. Greensmith, P. L. Bartlett, and J. Baxter. Variance reduction techniques for gradient estimates in reinforcement learning. Journal of Machine Learning Research, 5(Nov):1471–1530, 2004

    work page 2004

  7. [7]

    Hamilton, Z

    W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, pages 1024–1034, 2017

    work page 2017

  8. [8]

    H. Mao, M. Schwarzkopf, S. B. Venkatakrishnan, Z. Meng, and M. Alizadeh. Learning scheduling algorithms for data processing clusters. arXiv preprint arXiv:1810.01963, 2018

    work page arXiv 2018

Show all 27 references
  1. [9]

    Mirhoseini, A

    A. Mirhoseini, A. Goldie, H. Pham, B. Steiner, Q. V . Le, and J. Dean. A hierarchical model for device placement. In International Conference on Learning Representations , 2018

    2018

  2. [10]

    Mirhoseini, H

    A. Mirhoseini, H. Pham, Q. Le, M. Norouzi, S. Bengio, B. Steiner, Y . Zhou, N. Kumar, R. Larsen, and J. Dean. Device placement optimization with reinforcement learning. 2017

    2017

  3. [11]

    A. Nair, P. Srinivasan, S. Blackwell, C. Alcicek, R. Fearon, A. De Maria, V . Panneershelvam, M. Suleyman, C. Beattie, S. Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296, 2015

    2015 arXiv

  4. [12]

    A. Y . Ng, D. Harada, and S. J. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the Sixteenth International Conference on Machine Learning, ICML ’99, pages 278–287, San Francisco, CA, USA, 1999. Morgan Kaufma...

    1999

  5. [13]

    Onnx model zoo, 2018

    ONNX Developers. Onnx model zoo, 2018

    2018

  6. [14]

    Paliwal, F

    A. Paliwal, F. Gimeno, V . Nair, Y . Li, M. Lubin, P. Kohli, and O. Vinyals. Regal: Transfer learning for fast optimization of computation graphs. arXiv preprint arXiv:1905.02494, 2019

    1905

  7. [15]

    Pellegrini

    F. Pellegrini. A parallelisable multi-level banded diffusion scheme for computing balanced partitions with smooth boundaries. In T. P. A.-M. Kermarrec, L. Bougé, editor, EuroPar, volume 4641 of Lecture Notes in Computer Science, pages 195–204, Rennes, France, Aug. 2007. Springer

    2007

  8. [16]

    H. Pham, M. Y . Guan, B. Zoph, Q. V . Le, and J. Dean. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268, 2018

    2018 arXiv

  9. [17]

    Silver, J

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al. Mastering the game of go without human knowledge. Nature, 550(7676):354, 2017

    2017

  10. [18]

    R. S. Sutton and A. G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998

    1998

  11. [19]

    Szegedy, V

    C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2818–2826, 2016

    2016

  12. [20]

    Tensorflow official repository, 2017

    Tensorflow contributors. Tensorflow official repository, 2017

    2017

  13. [21]

    Perplexity — Wikipedia, the free encyclopedia, 2019

    Wikipedia contributors. Perplexity — Wikipedia, the free encyclopedia, 2019. [Online; accessed 26-April- 2019]

    2019

  14. [22]

    R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992

    1992

  15. [23]

    Y . Wu, M. Schuster, Z. Chen, Q. V . Le, M. Norouzi, W. Macherey, M. Krikun, Y . Cao, Q. Gao, K. Macherey, J. Klingner, A. Shah, M. Johnson, X. Liu, Ł. Kaiser, S. Gouws, Y . Kato, T. Kudo, H. Kazawa, K. Stevens, G. Kurian, N. Patil, W. Wang, C. Young, J. Smith, J. Riesa, A. Ru...

    2016

  16. [24]

    B. Zoph, V . Vasudevan, J. Shlens, and Q. V . Le. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012, 2(6), 2017. 9 Appendices A Implementation Details A.1 REINFORCE Algorithm Placeto is trained using the REINFORCE policy-gradie...

    2017 arXiv

  17. [25]

    Its network consists of a chain of blocks, each of which has multiple branches made up of convolutional and pooling operations

    Inception-V3 [19] is a widely used convolutional neural network which has been successfully applied to a large variety of computer vision tasks. Its network consists of a chain of blocks, each of which has multiple branches made up of convolutional and pooling operations. Whil...

  18. [26]

    NMT [23] Neural Machine Translation with attention is a language translation model that uses an LSTM based encoder-decoder architecture to translate a source sequence into a target sequence. When its computational graph is unrolled to handle input sequences of length up to 32,...

  19. [27]

    Its network consists of a series of cells each of which has multiple branches of computations that are finally reduced at the end to form input for the next cell

    Nasnet [24] is a computer vision model designed for image classification. Its network consists of a series of cells each of which has multiple branches of computations that are finally reduced at the end to form input for the next cell. It’s computational graph consists of 12942...

Pith tools

Reviewed May 25, 2026 · model on record in the stance chip above.