Pith. sign in

REVIEW 3 major objections 4 minor 39 references

This paper claims that a decomposition's quality is relative to the solver that must act on it, and that training the decomposer on feedback from a frozen synthesizer improves programming-by-example performance even when it lowers agreement

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Solver-aware training of a PBE decomposer with a frozen synthesizer's loss outperforms supervised imitation of ground-truth subgoals, solving tasks that a ground-truth decomposition oracle fails.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A genuinely novel training idea undermined by a silent train/serve shift: the synthesizer is trained on full task outputs and tested on predicted subgoals, so the GT-oracle failures are likely an artifact, not evidence against GT decompositions. the 3 major comments →

arxiv 2608.03461 v1 pith:QVJD2BVM submitted 2026-08-04 cs.AI

Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing how to Conquer

Janis Zenkner , Tobias Sesterhenn , Tim Grams , Christian Bartelt This is my paper
classification cs.AI
keywords programming by exampleprogram synthesisdecompositionsolver-aware trainingground-truth subgoalsself-critical sequence trainingcompositional generalizationaccuracy paradox
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Programming-by-example systems that decompose a task into subgoals usually train the decomposer to imitate human-annotated ground-truth decompositions, assuming that structural fidelity is what makes a subgoal good. This paper argues that for a bounded, learned synthesizer this assumption fails: decomposition quality is relative to the solver's search dynamics, not intrinsic to the task. It introduces Solver-Aware Decomposition (SAD), which keeps ground-truth supervision but adds a reinforcement-learning signal from a frozen synthesizer, rewarding subgoals that reduce the synthesizer's loss on the target program. Across two list-manipulation domains SAD improves synthesis and end-to-end accuracy while actually scoring lower on ground-truth agreement, and it solves tasks a ground-truth decomposition oracle fails. On a string-manipulation domain where decomposition order is fixed, the gains vanish, which the paper treats as a falsification check of its mechanism.

Core claim

The central claim is that ground-truth decompositions—the annotator's factorization of a task—are not universally optimal for a bounded synthesizer with fixed inductive biases. A subgoal can be logically valid and still lie outside what the synthesizer's search can act on; conversely, a subgoal that deviates from ground truth can place the synthesizer exactly where the target program becomes probable. To exploit this, SAD trains the decomposer with two signals: supervised imitation of ground-truth subgoals as a structural scaffold, and policy-gradient feedback in which sampled and greedy subgoals are scored by the frozen synthesizer's cross-entropy against the ground-truth program. The rewar

What carries the argument

The load-bearing mechanism is the reward signal: a proposed subgoal is fed to the frozen synthesizer, and the reward is the negative cross-entropy of the synthesizer's predicted program tokens against the ground-truth subprogram. Self-Critical Sequence Training compares the sampled subgoal's reward with the greedy subgoal's reward, so the advantage isolates relative tractability under the same solver. This is combined with the supervised ground-truth decomposition loss, which the paper shows is necessary to keep the policy inside a structured space of decompositions rather than collapsing to degenerate outputs.

Load-bearing premise

The argument stands on the fairness of comparing decompositions through a synthesizer that was trained on true task outputs but is queried on predicted subgoals; if that mismatch, rather than genuine tractability, drives SAD's reward, the ground-truth oracle comparison would not measure what it claims to measure.

What would settle it

Retrain the frozen synthesizer so that its training distribution matches inference—conditioning on subgoals produced by the decomposer—and rerun SAD against the ground-truth oracle. If the oracle then ties or beats SAD on the tasks SAD previously solved alone, the claimed counterexamples to ground-truth optimality are an artifact of the teacher-forcing mismatch.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • SAD yields consistent, statistically significant gains in synthesis and task accuracy over solver-blind supervised decomposition in DeepCoder and LambdaBeam, with larger relative gains under length generalization.
  • Higher agreement with ground-truth decompositions does not improve synthesis success; the accuracy paradox shows structural fidelity is not a proxy for solver utility.
  • A learned decomposer can solve tasks a ground-truth decomposition oracle fails, implying ground-truth decompositions are not a universal ceiling for bounded solvers.
  • SAD's multi-step advantage emerges from single-step training and grows with program length, because solver-aligned subgoals avoid cascading state corruption.
  • When decomposition order is structurally fixed in RobustFill, SAD and the solver-blind baseline are indistinguishable, confirming the mechanism requires decomposition ambiguity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if decomposition quality is genuinely solver-relative, then any change to the synthesizer—architecture, training data, beam size—invalidates an optimized decomposer, so decomposer-synthesizer pairs are coupled artifacts that must be retuned together.
  • Editorial inference: the same structural problem transfers beyond PBE to any system with a learned producer of intermediate representations and a bounded downstream executor, such as LLM agents that plan subtasks for a separate tool or verifier.
  • Editorial inference: a stricter test of the paper's claim would train the synthesizer on predicted subgoals (matching the inference distribution) before comparing SAD to a ground-truth oracle; if the advantage persists, it is about tractable distribution rather than teacher-forcing mismatch.
  • Editorial inference: because SAD's reward is anchored to ground-truth program tokens, a task-success reward could discover even more diverse alternative programs, potentially extending the paper's observed diversity beyond the small subset it currently reports.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies decomposition-based programming-by-example (PBE), where a learned decomposer proposes intermediate subgoals and a frozen learned synthesizer generates subprograms for the induced subtasks. The authors argue that ground-truth (GT) decompositions, being annotator choices, are not necessarily optimal for a bounded solver whose inductive biases may diverge from the GT distribution. They propose Solver-Aware Decomposition (SAD), which augments supervised imitation of GT subgoals with self-critical sequence training (SCST) using a reward based on the frozen synthesizer's cross-entropy to the GT subprogram. Experiments on DeepCoder and LambdaBeam report higher task accuracy than a solver-blind supervised baseline, an 'accuracy paradox' in which lower GT subgoal accuracy accompanies higher synthesis accuracy, and a subset of tasks solved by SAD but not by a GT-decomposition oracle. A RobustFill null result is presented as a designed falsifiability check. The central claim is that decomposition quality is solver-relative, not intrinsic.

Significance. If the central claim is sound, the paper makes a valuable conceptual and practical contribution: it reframes intermediate representation learning in PBE as solver-dependent optimization and provides a training signal that directly rewards solver tractability. The experimental design has notable strengths: the SAD vs. solver-blind comparison holds architecture, synthesizer weights, seeds, and search configuration fixed, and the RobustFill null result is a well-chosen falsification test. The L_sup ablation and beam-oracle analysis are useful diagnostic checks. However, the paper's strongest empirical claims rest on how the frozen synthesizer is conditioned during training, and the manuscript is internally inconsistent on this point. Because the reward definition and the GT-oracle comparison are both affected, the central claim is not yet established.

major comments (3)
  1. [Section 4, Eq. (1); Algorithm 1; Appendix C.1] The synthesizer conditioning is internally inconsistent. Eq. (1) trains the synthesizer on the current task output O^(t), Algorithm 1 calls the synthesizer on the predicted subgoal O_pred, and Appendix C.1 states that the SAD/baseline synthesizer is 'conditioned on the full task specification directly ... without an explicit intermediate subgoal as input.' These statements cannot all be true. If the Appendix C.1 reading is literal, then every synthesizer call at inference and every reward computation in Section 4.1 is out-of-distribution. The GT-oracle failures in Section 6 then reflect the model's reaction to OOD inputs, not the intrinsic tractability of GT decompositions. This directly affects the abstract's strongest claim that SAD solves tasks a GT decomposition oracle fails. The authors must specify the exact training-time and inference-time conditioning; if they differ, the experim
  2. [Section 4.1, reward definition R(O_i)] The reward is R(O_i) = -CE(Synth(I_i, O_i), p*_t), where p*_t is the GT subprogram associated with the GT subgoal for that step. If the proposed subgoal O_i differs from the GT subgoal O*_t, the program p*_t is generally not a valid solution to the induced subtask (I_i, O_i). The cross-entropy against p*_t then does not measure subtask difficulty or solver tractability; it measures how much probability the frozen model assigns to the original GT program under a different conditioning input. This makes the interpretation of SAD as 'trading GT alignment for solver tractability' unwarranted. In particular, the large drop in decomposition accuracy in Table 1 (Deepcoder test: 79.2% to 31.7%) is hard to explain under the stated reward, which should favor subgoals that keep p*_t likely. The authors need to define a valid target for non-GT subgoals—for example, whether the synthesizer can produc
  3. [Section 6, Table 1 and 'accuracy paradox'] Even setting aside the conditioning issue, the accuracy paradox as presented conflates two explanations: (i) GT alignment is an imperfect proxy for solver utility, and (ii) SAD's objective, as written, may itself bias the decomposer away from GT subgoals when the synthesizer is poorly calibrated under off-training-distribution conditioning. The current comparison cannot distinguish these. To support the paradox claim, the experiments should be repeated with a synthesizer trained on the same distribution on which it is queried, and the reward should be replaced by a subgoal-valid target. As written, Table 1 is consistent with a distribution-shift artifact rather than with solver-relative decomposition quality.
minor comments (4)
  1. [Section 1] Typo: 'an distinction' should be 'a distinction'.
  2. [Section 6 and Table 2] The text says 'SAD also improves over Exedec in both domains,' but Table 2 reports SAD vs. ExeDec on Deepcoder test-on-training-distribution with p = 0.177 (not significant). The claim should be qualified or restricted to length generalization and LambdaBeam.
  3. [References] Reference [32] (Shin et al., 'Synthetic datasets for neural program synthesis') has an arXiv identifier '1912.12345' that appears to be a placeholder. Please verify the citation.
  4. [Appendix D.1] The statistical tests use paired t-tests over five seeds. It would be useful to state explicitly whether the 1000 test tasks are identical across seeds or resampled per seed, as this affects the interpretation of the paired test.

Circularity Check

1 steps flagged

GT-oracle evidence is generated by the same CE objective SAD optimizes; with a synthesizer never trained on subgoal inputs, the central claim reduces to the training setup.

specific steps
  1. fitted input called prediction [Sec. 4.1 (reward), Sec. 6 (GT oracle), App. C.1 (synthesizer conditioning)]
    "R(O(t)i) = −Lsyn(Synth(O(t)i)) = −CE(Synth(I(t)i, O(t)i), p∗t) ... The GT decomposition oracle ... replaces predicted decompositions with GT subgoals at inference under the same synthesizer and beam size as SAD ... A consistent subset of tasks is solved by SAD but not the GT oracle ... The SAD and solver-blind baseline synthesizer is instead conditioned on the full task specification directly – it learns to predict the next subprogram given the remaining task, without an explicit intermediate subgoal as input."

    SAD's training reward is exactly the negative CE of the frozen synthesizer on GT program tokens given a proposed subgoal. The GT-oracle comparison judges subgoals by the same synthesizer: oracle failures occur when GT subgoals yield high CE. Since SAD is trained to minimize this CE while GT subgoals are not optimized, the oracle's inferiority is a direct consequence of the optimization objective. Moreover, App. C.1 states the synthesizer was never trained on subgoal inputs (only full task outputs), so GT subgoals are out-of-distribution by construction, guaranteeing higher CE. The conclusion that GT decompositions are not universally optimal is therefore an artifact of the training/reward setup, not an independent discovery.

full rationale

The paper's central claim that GT decompositions are not universally optimal rests on the GT-oracle comparison and the accuracy paradox. Both are compromised by a training/evaluation mismatch that the paper itself documents: Eq. 1 trains the synthesizer on the full task output O^(t), while Algorithm 1 queries it on predicted subgoal O_pred, and Appendix C.1 explicitly states the synthesizer is conditioned on the full task specification without an explicit intermediate subgoal. Consequently, GT subgoals are out-of-distribution inputs by construction; the GT oracle is thus handicapped before any comparison. At the same time, SAD's reward is precisely -CE(Synth(I,O_pred),p*), so SAD is fit to the exact quantity on which the oracle is judged. The oracle failures are therefore not an independent empirical discovery but a consequence of the optimization objective and the chosen synthesizer conditioning. This does not invalidate the entire paper: task accuracy is an external metric, the Robustfill null result is a genuine falsification, and the effect of solver-aware training on task success could persist under a properly subgoal-conditioned synthesizer. But the strongest claim—empirical evidence that GT decompositions are not universally optimal—reduces to the training setup rather than being demonstrated.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claims rely on the reward definition, the synthesizer's train/serve conditioning, and the GT oracle design. No new physical or conceptual entities are introduced beyond the SAD training objective.

free parameters (3)
  • Entropy coefficient lambda = 0.01
    Hand-chosen in the SCST objective L = L_sup + L_RL - lambda H(pi); controls exploration and affects the learned policy.
  • Beam size = 10
    Used for all methods at inference; affects oracle and SAD comparisons.
  • Step limit T = 10 (list domains), 20 (Robustfill)
    Inference budget that constrains the pipeline and affects task accuracy.
axioms (4)
  • domain assumption The synthesizer's CE loss on the GT program, conditioned on a proposed subgoal, is a valid proxy for subtask tractability.
    Section 4.1 defines the reward and calls it 'a signal of subtask difficulty'; this is an assumption, not a proven property.
  • domain assumption The frozen synthesizer, trained on full task outputs, can meaningfully condition on predicted subgoals at inference.
    Algorithm 1 feeds predicted subgoals to the synthesizer, but training (Eq. 1) uses the current target output O^(t); the mismatch is unflagged.
  • domain assumption GT subgoals (intermediate states along the GT program path) are a fair representative of GT-aligned decomposition.
    The GT oracle and decomposition accuracy metric use these subgoals; the paper treats them as the reference.
  • standard math The SCST advantage A = R(sampled) - R(greedy) captures relative tractability differences.
    Standard SCST assumption from Rennie et al. [29]; reasonable but not independently verified here.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing how to Conquer." pith.science (2026). https://pith.science/paper/QVJD2BVM

@misc{pith2026260803461,
  author       = {Pith},
  title        = {Pith review of: Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing how to Conquer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVJD2BVM}},
  note         = {Machine review of arXiv:2608.03461}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Decomposition-based Programming-by-example (PBE) scales performance by splitting tasks into subtasks that a learned synthesizer solves: a decomposer predicts intermediate subgoals, and a synthesizer generates programs conditioned on them. Current approaches train the decomposer to imitate ground-truth ( GT) subgoals, implicitly treating decomposition quality as intrinsic to the task. We challenge this assumption: for bounded solvers with fixed inductive biases, GT decompositions reflect the annotator's factorization choices - not the solver's search dynamics. A decomposer trained to match GT decompositions may therefore propose subgoals that are logically valid yet intractable for the solver. We propose Solver-Aware Decomposition (SAD), a training framework that retains supervised training on GT subgoals as a structural scaffold, while additionally optimizing the decomposer via direct feedback from a frozen synthesizer. Subgoals are rewarded based on the synthesizer's loss on the target program - a signal of subtask difficulty that encourages decompositions the solver can act on. Our experiments reveal an accuracy paradox: higher agreement with GT decompositions does not improve synthesis success - even though the synthesizer was trained on the very same GT data the decomposer is optimized to mimic. SAD instead learns decompositions that trade GT alignment for solver tractability, yielding consistent gains in synthesis and end-to-end task accuracy across two PBE domains. Moreover, SAD solves tasks that a GT decomposition oracle fails - empirical evidence that GT decompositions are not universally optimal for bounded solvers, and that decomposition quality is solver-relative, not intrinsic.

Figures

Figures reproduced from arXiv: 2608.03461 by Christian Bartelt, Janis Zenkner, Tim Grams, Tobias Sesterhenn.

Figure 1
Figure 1. Figure 1: Subgoal similarity to GT is a poor proxy for solver utility. Left: decomposition accuracy (exact match with GT subgoals): solver-blind training scores higher than the solver-aware training. Right: synthesis accuracy given predicted subgoals: solver-blind training scores lower despite its higher GT alignment. This shows that higher decomposition accuracy does not translate into better synthesis performance.… view at source ↗
Figure 2
Figure 2. Figure 2: Solver-aware training loop: the decomposer is trained using feedback from a frozen [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Task accuracy on Deepcoder and Lambdabeam. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Task-level overlap between SAD and the GT decomposition oracle accumulated over in-distribution and length generalization test sets. The GT oracle solves substantially more tasks overall. However, a consistent subset of tasks is solved by SAD but not the oracle, suggesting that GT decompositions are not universally optimal for the synthesizer. Mean and stand. dev. across seeds. the step level, SAD’s decomp… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 26 canonical work pages · 5 internal anchors

  1. [1]

    Towards a generic algorithm for identifying high-quality decompositions of optimization problems

    Andrew Allman, Wentao Tang, and Prodromos Daoutidis. Towards a generic algorithm for identifying high-quality decompositions of optimization problems. InComputer aided chemical engineering, volume 44, pages 943–948. Elsevier, 2018

    work page 2018

  2. [2]

    Syntax- guided synthesis

    Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. Syntax- guided synthesis. In2013 F ormal Methods in Computer-Aided Design, pages 1–8. IEEE, 2013

    work page 2013

  3. [3]

    Deepcoder: Learning to write programs.arXiv preprint arXiv:1611.01989, 2016

    Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. Deepcoder: Learning to write programs.arXiv preprint arXiv:1611.01989, 2016

    Pith/arXiv arXiv 2016

  4. [4]

    Flashfill++: Scaling programming by example by cutting to the chase

    José Cambronero, Sumit Gulwani, Vu Le, Daniel Perelman, Arjun Radhakrishna, Clint Simon, and Ashish Tiwari. Flashfill++: Scaling programming by example by cutting to the chase. Proceedings of the ACM on Programming Languages, 7(POPL):952–981, 2023

    work page 2023

  5. [5]

    Hipo: Instruction hierarchy via constrained reinforcement learning.arXiv preprint arXiv:2603.16152, 2026

    Keru Chen, Jun Luo, Sen Lin, Yingbin Liang, Alvaro Velasquez, Nathaniel Bastian, and Shaofeng Zou. Hipo: Instruction hierarchy via constrained reinforcement learning.arXiv preprint arXiv:2603.16152, 2026

    arXiv 2026

  6. [6]

    Compositional generalization via neural-symbolic stack machines.Advances in Neural Information Processing Systems, 33:1690–1701, 2020

    Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou. Compositional generalization via neural-symbolic stack machines.Advances in Neural Information Processing Systems, 33:1690–1701, 2020

    work page 2020

  7. [7]

    Generating Planning Feedback for Open-Ended Programming Exercises with LLMs

    Mehmet Arif Demirta¸ s, Claire Zheng, Max Fowler, and Kathryn Cunningham. Generat- ing planning feedback for open-ended programming exercises with llms.arXiv preprint arXiv:2504.08958, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    Robustfill: Neural program learning under noisy i/o

    Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. InInternational conference on machine learning, pages 990–998. PMLR, 2017

    work page 2017

  9. [9]

    Synthesizing data structure transformations from input-output examples.ACM SIGPLAN Notices, 50(6):229–239, 2015

    John K Feser, Swarat Chaudhuri, and Isil Dillig. Synthesizing data structure transformations from input-output examples.ACM SIGPLAN Notices, 50(6):229–239, 2015

    work page 2015

  10. [10]

    Au- tomated curriculum learning for neural networks

    Alex Graves, Marc G Bellemare, Jacob Menick, Remi Munos, and Koray Kavukcuoglu. Au- tomated curriculum learning for neural networks. Ininternational conference on machine learning, pages 1311–1320. Pmlr, 2017

    work page 2017

  11. [11]

    Automating string processing in spreadsheets using input-output examples

    Sumit Gulwani. Automating string processing in spreadsheets using input-output examples. ACM Sigplan Notices, 46(1):317–330, 2011

    work page 2011

  12. [12]

    Latent program- mer: Discrete latent codes for program synthesis

    Joey Hong, David Dohan, Rishabh Singh, Charles Sutton, and Manzil Zaheer. Latent program- mer: Discrete latent codes for program synthesis. InInternational Conference on Machine Learning, pages 4308–4318. PMLR, 2021

    work page 2021

  13. [13]

    Compositional Program Generation for Few-Shot Systematic Generalization

    Tim Klinger, Luke Liu, Soham Dan, Maxwell Crouse, Parikshit Ram, and Alexander Gray. Com- positional program generation for systematic generalization.arXiv preprint arXiv:2309.16467, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  14. [14]

    Addison-Wesley Professional, 1998

    Donald E Knuth.The Art of Computer Programming: Sorting and Searching, volume 3. Addison-Wesley Professional, 1998

    work page 1998

  15. [15]

    Coderl: Mastering code generation through pretrained models and deep reinforcement learning

    Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314–21328, 2022. 10

    work page 2022

  16. [16]

    Accelerating search-based program synthesis using learned probabilistic models.ACM SIGPLAN Notices, 53(4):436–449, 2018

    Woosuk Lee, Kihong Heo, Rajeev Alur, and Mayur Naik. Accelerating search-based program synthesis using learned probabilistic models.ACM SIGPLAN Notices, 53(4):436–449, 2018

    work page 2018

  17. [17]

    Is programming by example solved by llms?arXiv preprint arXiv:2406.08316, 2024

    Wen-Ding Li and Kevin Ellis. Is programming by example solved by llms?arXiv preprint arXiv:2406.08316, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Hierarchical programmatic reinforcement learning via learning to compose programs

    Guan-Ting Liu, En-Pei Hu, Pu-Jen Cheng, Hung-Yi Lee, and Shao-Hua Sun. Hierarchical programmatic reinforcement learning via learning to compose programs. InInternational Conference on Machine Learning, pages 21672–21697. PMLR, 2023

    2023

  19. [19]

    Self-refine: Iterative refinement with self-feedback.Advances in neural information processing systems, pages 46534–46594, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback.Advances in neural information processing systems, pages 46534–46594, 2023

    work page 2023

  20. [20]

    Teacher–student curriculum learning.IEEE transactions on neural networks and learning systems, 31(9):3732–3740, 2019

    Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman. Teacher–student curriculum learning.IEEE transactions on neural networks and learning systems, 31(9):3732–3740, 2019

    work page 2019

  21. [21]

    Neural sketch learning for conditional program generation.arXiv preprint arXiv:1703.05698, 2017

    Vijayaraghavan Murali, Letao Qi, Swarat Chaudhuri, and Chris Jermaine. Neural sketch learning for conditional program generation.arXiv preprint arXiv:1703.05698, 2017

    Pith/arXiv arXiv 2017

  22. [22]

    Learning to infer program sketches

    Maxwell Nye, Luke Hewitt, Joshua Tenenbaum, and Armando Solar-Lezama. Learning to infer program sketches. InInternational Conference on Machine Learning, pages 4861–4870. PMLR, 2019

    work page 2019

  23. [23]

    Is self-repair a silver bullet for code generation?arXiv preprint arXiv:2306.09896, 2023

    Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar- Lezama. Is self-repair a silver bullet for code generation?arXiv preprint arXiv:2306.09896, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Doing experiments and revising rules with natural language and probabilistic reasoning.Advances in Neural Information Processing Systems, 37:53102–53137, 2024

    Wasu T Piriyakulkij, Cassidy Langenfeld, Tuan A Le, and Kevin Ellis. Doing experiments and revising rules with natural language and probabilistic reasoning.Advances in Neural Information Processing Systems, 37:53102–53137, 2024

    work page 2024

  25. [25]

    Au- tomatic curriculum learning for deep rl: A short survey.arXiv preprint arXiv:2003.04664, 2020

    Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. Au- tomatic curriculum learning for deep rl: A short survey.arXiv preprint arXiv:2003.04664, 2020

    Pith/arXiv arXiv 2003

  26. [26]

    Adapt: As-needed decomposition and planning with language models

    Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. Adapt: As-needed decomposition and planning with language models. arXiv preprint arXiv:2311.05772, 2023

    Pith/arXiv arXiv 2023

  27. [27]

    Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement.arXiv preprint arXiv:2310.08559, 2023

    Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, et al. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement.arXiv preprint arXiv:2310.08559, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Sequence level training with recurrent neural networks.arXiv preprint arXiv:1511.06732, 2015

    Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks.arXiv preprint arXiv:1511.06732, 2015

    Pith/arXiv arXiv 2015

  29. [29]

    Self- critical sequence training for image captioning

    Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. Self- critical sequence training for image captioning. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 7008–7024, 2017

    work page 2017

  30. [30]

    Lambdabeam: Neural program search with higher-order functions and lambdas.Advances in Neural Information Processing Systems, pages 51327–51346, 2023

    Kensen Shi, Hanjun Dai, Wen-Ding Li, Kevin Ellis, and Charles Sutton. Lambdabeam: Neural program search with higher-order functions and lambdas.Advances in Neural Information Processing Systems, pages 51327–51346, 2023

    work page 2023

  31. [31]

    Exedec: Execution decomposition for compositional generalization in neural program synthesis

    Kensen Shi, Joey Hong, Yinlin Deng, Pengcheng Yin, Manzil Zaheer, and Charles Sutton. Exedec: Execution decomposition for compositional generalization in neural program synthesis. arXiv preprint arXiv:2307.13883, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    Synthetic Datasets for Neural Program Synthesis

    Richard Shin, Neel Kant, Kavi Gupta, Christopher Bender, Brandon Trabucco, Rishabh Singh, and Dawn Song. Synthetic datasets for neural program synthesis.arXiv preprint arXiv:1912.12345, 2019. 11

    work page internal anchor Pith review Pith/arXiv arXiv 1912

  33. [33]

    University of California, Berkeley, 2008

    Armando Solar-Lezama.Program synthesis by sketching. University of California, Berkeley, 2008

    work page 2008

  34. [34]

    Learning to synthesize programs as interpretable and generalizable policies.Advances in neural information processing systems, 34:25146–25163, 2021

    Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J Lim. Learning to synthesize programs as interpretable and generalizable policies.Advances in neural information processing systems, 34:25146–25163, 2021

    work page 2021

  35. [35]

    A Divide-Align-Conquer Strategy for Program Synthesis

    Jonas Witt, Stef Rasing, Sebastijan Duman ˇci´c, Tias Guns, and Claus-Christian Carbon. A divide-align-conquer strategy for program synthesis.arXiv preprint arXiv:2301.03094, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  36. [36]

    A syntactic neural model for general-purpose code generation.arXiv preprint arXiv:1704.01696, 2017

    Pengcheng Yin and Graham Neubig. A syntactic neural model for general-purpose code generation.arXiv preprint arXiv:1704.01696, 2017

    Pith/arXiv arXiv 2017

  37. [37]

    Beyond Either-Or Reasoning: Transduction and Induction as Cooperative Problem-Solving Paradigms

    Janis Zenkner, Tobias Sesterhenn, and Christian Bartelt. Transductively informed inductive program synthesis.arXiv preprint arXiv:2505.14744, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    Limitations

    Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. Planning with large language models for code generation.arXiv preprint arXiv:2303.05510, 2023. 12 A Domains & DSLs We evaluate on three standard PBE domains that differ in their program structure, state update semantics, and–crucially–the degree of decomposition ambig...

    Pith/arXiv arXiv 2023

  39. [39]

    Guidelines: 30 • The answer [N/A] means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or ...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.