Pith. sign in

REVIEW 3 major objections 4 minor 15 references

Committed reasoning stages can earn their cost when a cheap residual check selects which endpoints to repair, and formal transcription plus deterministic execution—not extra model reasoning—drives the measured gain.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 18:36 UTC pith:CF2ZNUWE

load-bearing objection A transparent, well-executed empirical study of LLM reasoning stages; the headline 90% yield is probably right in direction but rests on an uncertified oracle on 18/30 problems—treat the exact number cautiously. the 3 major comments →

arxiv 2607.17240 v1 pith:CF2ZNUWE submitted 2026-07-19 cs.AI

Constrained Path Reasoning: Measuring When Committed Stages Earn Their Cost

classification cs.AI
keywords constrained path reasoningresidual-guided triageLLM reasoning stagescost per usable outputQCQP convexificationerror bound calibrationendpoint concentrationcommitment accounting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks when a committed intermediate stage in an LLM reasoning pipeline earns its cost, and answers with an accounting framework plus a quantitative case study in automated convexification. The central claim is that task-compatible commitments—trusted or verified invariants constraining hard, model-generated states staying soft and revisable—can factor transitions, concentrate candidate mass, and expose feedback, provided their gains exceed propagated error and execution cost. The numbers that carry this claim: a residual-based triage recovers 63.0% of repair-all's extra feasible yield with 17.7% of its repair attempts, and formal transcription with deterministic execution lifts usable yield from 41.1% to 90.0%, while a one-shot convexification commitment drops it to 20.0%. A sympathetic reader cares because the method turns 'think harder vs commit an artifact' into measurable cost-per-usable-output accounting that can be applied to any executable intermediate stage.

Core claim

CPR treats chain-of-thought as constrained path construction: search proposes provisional states; trusted or verified invariants can restrict the path hard, while model-generated proposals are weighed softly and remain revisable until a verifier certifies them. The paper's core empirical discovery is that in a low-friction convexification regime, formal representation plus deterministic execution—not additional LLM reasoning—is the highest-value commitment, with usable yield jumping from 41.1% to 90.0%; adding a committed convex surrogate exposes two failure modes, relaxation leakage (outer semidefinite relaxation infeasible on loose problems) and over-contraction (inner majorization feasibl

What carries the argument

The carrying mechanism is the residual-guided endpoint verifier: for a surrogate solution x_c, compute the continuous constraint residual r(x_c) against the original constraints, and calibrate an empirical power law r-to-distance (here about 0.145 r^1.046 with modulus 0.350) so that a repair-radius budget converts into a residual cutoff. This yields three tiers—near-feasible, repair, reject—that decide whether to spend a bounded local correction. Around it sits the CPR accounting protocol: shared-call nested ablations that reuse upstream LLM calls, stage-local correctness checks, and cost-per-usable-output pricing, so that every committed stage is judged by yield, cost, and the error modes i

Load-bearing premise

The residual–projection power law fitted on synthetic QCQP endpoints (ed about 0.145 r^1.046 with modulus 0.350) is assumed to transfer to LLM-formalized endpoints and held-out instances, so that the accept/repair gate thresholds stay correctly calibrated wherever the pipeline is deployed.

What would settle it

On a fresh set of, say, 50 LLM-formalized QCQPs with coefficients rounded to three decimals, recompute constrained projections and check whether the residual–distance relation still lies near ed = 0.145 r^1.046 and whether the calibrated gate still recovers about 63% of repair-all's additional yield with about 18% of its attempts; a large drift in either would falsify the calibration-transfer premise.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Residual-gated triage can replace most repair attempts: on 360 held-out instances it achieves 28.3% usable yield with 53 repairs versus 35.0% with 299, and the break-even analysis says gating is cheaper whenever a repair attempt costs more than 0.098 retry attempts.
  • Formalization and deterministic execution is the dominant stage in low-friction regimes; committed surrogate transitions need explicit leakage and over-contraction audits before being added.
  • Constraint strength should come from trusted execution or verification: model-generated states should remain soft and reversible, since unverified cross-trajectory commitments concentrate candidate mass while reducing acceptable mass from 30.6% to 2.8%.
  • Feedback-conditioned action selection can underperform a fixed rollback rule: the two-action rule reached 90% usable yield versus 36.7% for the learned-feedback selector, exposing a decision-quality and controller-cost gap.
  • Stage-level accounting with cost per usable output provides a general protocol for pricing intermediate artifacts (tokens, wall-clock, solver time) in LLM reasoning pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: sweep transcription noise (coefficient perturbation levels) and initialization quality to map where formalization-plus-execution stops dominating direct generation; the paper treats these as regime variables but does not chart the boundary.
  • If the power-law calibration transfers beyond QCQPs, the same residual-to-repair-budget mechanism could price repair attempts in other verified-execution settings—code patches, formal proofs, robotics trajectories—where a cheap scalar check predicts repair reachability.
  • The transplant probe suggests a general 'provenance tax': reusing a committed state from another trajectory concentrates candidates but can destroy utility; this is worth testing in multi-agent or retrieval-augmented settings where states are shared across trajectories.
  • An automated regime classifier (transcription difficulty, ordering, initialization) could decide which pipeline package to run per instance; the paper's cost accounting supplies the objective for such a meta-policy but leaves it unimplemented.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Constrained Path Reasoning (CPR), a framework for viewing LLM intermediate reasoning as a constrained path in which trusted/verified artifacts may constrain hard while model-generated proposals remain soft and revisable, together with an accounting protocol for pricing committed stages by cost per usable output. The empirical core is an automated convexification pipeline: on 360 held-out generated QCQPs, residual-guided endpoint triage is reported to recover 63.0% of repair-all's additional feasible yield using 17.7% of its repair attempts; a 30-problem fixed-LLM ablation with 270 shared calls reports usable yield of 41.1% for direct prediction, 90.0% after formalization plus deterministic execution, 20.0% after one-shot convexification, and 21.1% after verification/repair. Additional experiments study feedback-conditioned action selection, cross-trajectory state transplantation, and same-call self-proposal. The paper argues that, in the low-friction regime studied, formalization-plus-execution is the highest-value stage and model-proposed states should remain soft until executor-validated.

Significance. If the empirical claims hold, the residual-triage result is practically useful and the ablation design is a strong example of stage-level accounting: shared calls across nested arms, zero tool events, deterministic executors, per-problem bootstrap intervals, and a public reproducibility release. Proposition 1 is a clean and correct operationalization of classical H\"olderian error bounds, and the paper is unusually transparent about its protocols, amendments, and limitations. The main barrier is that the headline LLM result is measured against an oracle certified on only 12 of 30 problems; the magnitude, and possibly the ordering, of the F-versus-D lift is therefore not yet established. The theoretical framework is broad but the evidence is concentrated in one task family; the case study is nevertheless sufficient to support the framework's accounting claims if the oracle issue is resolved.

major comments (3)
  1. [§6 / App. E, Table 2] The headline LLM lift (F 90.0% vs D 41.1% usable yield) is computed against a 26-start SLSQP oracle certified on only 12/30 problems; stability under 100 starts is not a certificate. F endpoints are produced by the same solver family from the standardized feasible origin, so they lie in the oracle's basin, whereas D endpoints need not. If the reference is suboptimal on some uncertified problems, F passes the gap-to-oracle test more often than D, inflating the abstract's 48.9-point lift. The confounding is explicit in App. E: all 18 uncertified problems have loose SDR, so the certified subset is also the tight-SDR subset. Please make the certified 12-problem comparison primary (52.8% vs 100.0%), certify the remaining problems with a global solver (n≤4 QCQPs are small enough), or provide a sensitivity analysis showing that the F-vs-D ordering and the 90.0% figure are robust under a conserv
  2. [§5, Table 1 and abstract] The central residual-triage result—28.3% vs 35.0% yield, 63.0% recovery of additional yield with 53/299 attempts—is reported without any uncertainty interval. These are binomial counts over 360 held-out instances plus a derivative ratio; the paper gives a Wilson CI for coverage (95.0%) but not for the quantity the abstract advertises. Add exact or bootstrap CIs for the two yields, the recovery ratio, and repair precision, and state the number of instances behind each percentage. App. D's tier counts make this computation straightforward.
  3. [§4.2 / App. D] The calibrated gate constants (κhat=0.350, θhat=1.046) are fit on SDR-relaxed endpoints from the same generator family, and the 95.0% coverage is with respect to the projection proxy ed, not the true distance. The transfer to LLM-formalized endpoints rests on only 30/34 endpoints (88.2%) with no interval. The Table 1 triage claim does not depend on the power-law transfer—at the chosen budget the residual ranking and the gate select the same 53 endpoints—so this is not a circularity in the main result. However, the paper should state explicitly that the 63.0% claim is a property of residual ranking on held-out generated instances, and that the LLM-FCV use of the transferred gate is exploratory until validated on a larger LLM-formalized endpoint bank.
minor comments (4)
  1. [Fig. 4(a)] The 'strict-yield error bars are problem-level bootstrap intervals' should specify the resampling scheme (problems vs samples) and seed in the main text; App. E currently gives only a short statement.
  2. [App. H / §8] With two repeats per cell, endpoint entropy is a collision diagnostic taking only two values; 'unchanged two-repeat collision entropy' should not be read as evidence of no effect. The paper is careful to call it a collision diagnostic, but the §8 summary risks overstatement.
  3. [Abstract and Table 1] Clarify that the 53 attempts comprise 16 near-feasible plus 37 repair attempts, since this decomposition is mentioned only in §4.2 and App. D. Adding it to Table 1's caption would improve interpretability.
  4. [References] Several references are to 2026 preprints (e.g., Peng et al., arXiv:2601.04789); verify availability and update if the cited versions have been superseded.

Circularity Check

0 steps flagged

No significant circularity: the headline triage yield and LLM ablation lifts are measured outcomes from fixed held-out evaluations, with calibration transparently separated from analytical certificates.

full rationale

The paper's derivation chain does not reduce to its own inputs. The residual-triage result is an empirical count on 360 held-out instances (Table 1): the gate thresholds are calibrated in §4.2, but the 63.0% recovery and 17.7% attempt ratios are observed outcomes of applying a fixed cutoff, not algebraic consequences of the fitted power law. The paper even states that 'the gate and residual ranking select the same 53 endpoints,' so the ranking property itself is the empirical finding, not a concealed restatement of the calibration. Proposition 1 is explicitly an 'operationalization of classical bounds' (Hoffman, Pang), and §4.2 separates analytical (κ, θ) from calibrated (κ̂, θ̂), reporting held-out coverage separately from the triage yields. The LLM ablation's F-arm result is measured against a benchmark oracle; the oracle is certified on only 12/30 problems and the paper discloses this in App. E. The concern that an uncertified oracle may favor F is a benchmark-validity/correctness risk, not circularity, because F's yield is not defined as 'close to the oracle' by construction—it is an outcome of running SLSQP. Moreover, on the certified 12-problem subset the paper reports F usable yield 100.0% vs D 52.8%, so the headline lift is not solely an artifact of the uncertified oracle. No self-citation is load-bearing: the author does not rely on prior work of their own to justify the central claims. The four mechanisms are explicitly labeled conjectures/hypotheses rather than derived predictions, and no uniqueness theorem is imported. In sum, no specific reduction of a prediction to its input by construction can be exhibited, so the circularity score is 0.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

No new physical or mathematical entities are introduced. The central tools are standard error bounds, metric regularity, and LLM/optimizer executables, used under explicit domain assumptions. The gate's free parameters are fitted honestly and do the work of turning residual into a repair cutoff.

free parameters (8)
  • global power-law exponent θhat = 1.046 (bootstrap CI [1.019, 1.062])
    Log-log fit of ed on r over 325 SDR-relaxed endpoints; sets the gate exponent (§4.2, App. D).
  • global power-law modulus κhat = 0.350
    95th percentile of ed/r^θhat on the calibration split; sets gate thresholds δ1, δ2 (§4.2).
  • degenerate-family exponent θhat = 0.505 ± 0.017
    Within-instance fit on 40 engineered g = p^2 instances (Fig. 3b); realizes the quadratic-growth regime.
  • acceptance accuracy ε = 0.02R
    Chosen NEAR-FEASIBLE radius; enters δ1 = (ε/κhat)^(1/θhat) (App. D).
  • repair radius budget β = 0.25R
    Chosen repair budget and cumulative repair-path-length cap; sets δ2 and the break-even cost ratio.
  • usable objective gap tolerance = 0.05R
    Defines usable endpoints in LLM arms and probes; arbitrary but fixed and swept near boundary.
  • feasibility tolerance = 10^-6 (loose 10^-3 sensitivity)
    Tight numeric feasibility threshold for all yield definitions.
  • multi-start projection protocol size = 10 starts (checked at 10/50/100)
    The projection proxy ed is computed from 10 starts; sensitivity checks show median ratio 1.00 vs 100 starts.
axioms (6)
  • standard math Classical error bound (EBθ): dist(x,F) ≤ κ r(x)^θ on a residual tube
    Invoked in Proposition 1 and operationalized in §4.2; for generated systems MFCQ or quadratic-growth conditions are assumed (App. C).
  • domain assumption Constraints are C¹ with non-empty compact feasible set containing a bounding ball
    Assumed for Prop 1 and guaranteed by the generator and LLM instance construction (§4, App. D).
  • domain assumption Multi-start projection proxy ed approximates true distance dist, with ed ≥ dist
    The power-law fit uses ed, not true dist; a 10/50/100-start sensitivity check only partially validates this proxy (§4.2, App. D).
  • domain assumption The synthetic QCQP family is representative of LLM-formalized problems
    The calibrated gate from 325 generated endpoints is transferred to LLM endpoints in §6; the loose-SDR subset is a different population.
  • domain assumption Oracle objective values (26-start SLSQP, unchanged at 100 starts; SDR-certified on 12/30) are ground truth for usability
    Defines 'usable' together with the 0.05R gap; not certified on 18/30 problems.
  • domain assumption LLM gpt-5.5 schema-enforced outputs parse and contain no tool events
    Parser checks and JSONL event logs establish 100% parse rate and zero tool events (§§6, F).

pith-pipeline@v1.3.0-alltime-deepseek · 20056 in / 13722 out tokens · 130016 ms · 2026-08-01T18:36:37.686270+00:00 · methodology

0 comments
read the original abstract

When does a committed intermediate stage in an LLM reasoning pipeline earn its cost? Constrained Path Reasoning (CPR) pairs a source-aware path hypothesis with stage-level accounting. Search generates provisional states; trusted or validated invariants can constrain hard, while other proposals remain soft and revisable. CPR predicts that task-compatible commitments can factor transitions, concentrate candidate mass, induce regularity, and expose feedback when their gains exceed propagated error and execution cost. The formalism covers discrete commitments and continuous flows and measures effective branching, endpoint concentration, and cost per usable output. Across 1,180 generated QCQPs and 40 engineered degenerate polynomial instances (2,140 endpoints), residual triage recovers 63.0% of repair-all's additional feasible yield with 17.7% of its attempts. Fixed-LLM accounting (270 unique calls shared across nested arms) finds usable yield of 41.1% direct, 90.0% after formalization and deterministic execution, 20.0% after one-shot convexification, and 21.1% for the full path. In 120 paired-condition calls, a two-action rollback rule reaches 90% usable yield versus 36.7% for the feedback-conditioned selector. Two endpoint probes separate source from validation: a 72-output cross-trajectory transplant reduces entropy and acceptable mass; a 24-output same-call self-proposal pilot gives unchanged two-repeat collision entropy, 25.0% versus 8.3% usable yield, and 1/8 deterministically confirmed endpoint checks. Model-generated states supply hypotheses; trusted execution earns constraint strength.

Figures

Figures reproduced from arXiv: 2607.17240 by Honglin Li (ShanghaiTech University).

Figure 1
Figure 1. Figure 1: Constrained Path Reasoning. Left: direct prediction crosses a broad candidate interface and receives endpoint feedback. Right: trajectory-generated states or trusted invariants factor the transition, may concentrate candidate mass, induce regularity, and expose correction signals. Provisional states retain rollback. The fans visualize the candidate￾level hypothesis; arrows may be discrete operators or chec… view at source ↗
Figure 2
Figure 2. Figure 2: Automated convexification as a constrained path with heterogeneous operators. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Residual–projection-distance scaling on generic SDR-relaxed QCQPs and engi [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Stage-package pricing. (a) Usable yield at two feasibility tolerances; strict-yield error bars are problem-level bootstrap intervals, and the diamond marks F with seeded random initialization. (b) Complete LLM input and output tokens per usable solution. (c) LLM plus executor wall-clock per usable solution. Arm calls usable % [CI] ret./prec. % in/use (k) out/use (k) wall/use (s) Direct (D) 1 41.1 [29,52] 1… view at source ↗
Figure 5
Figure 5. Figure 5: Residual-ranking frontier with the operational repair-path cap fixed at [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example natural-language problem (abridged; problem 0 of the bank). [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-problem stage-package outcomes. Each cell is the number of usable outputs [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Concentration paired with task utility. (a) For the fixed action interface, many reported allocation distributions concentrate while usable allocation mass decreases; marker shape records the executed utility of the selected action. (b) The transplanted endpoint state concentrates several problem-level endpoint distributions while acceptable mass falls beyond the pre-specified five-point tolerance. Shading… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 9 linked inside Pith

  1. [4]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,

  2. [11]

    Scaling LLM test-time com- pute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314,

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314,

  3. [12]

    Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419,

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models.arXiv preprint arXiv:2503.16419,

  4. [14]

    Thus there are a neighborhood V¯x and modulus κ ¯x for which dist(x, F)≤κ ¯x dist(0, M(x)) on V¯x

    of theset-valuedconstraint mapping M(x) = g(x), h(x) − Rm − × {0} (Robinson, 1976). Thus there are a neighborhood V¯x and modulus κ ¯x for which dist(x, F)≤κ ¯x dist(0, M(x)) on V¯x. Finite-dimensional norm equivalence lets the max- residual (11) control dist(0, M(x)) . Compactness of F provides a finite subcover V1, . . ., VN; taking the largest of the c...

  5. [18]

    SCA” as one SCA-style inner majorization (OSM) anchored at x0 = 0 by dropping each quadratic’s concave part; results target the single committed surrogate. “Already convex

    is SDR-certified. A planner chooses 2 decision variables x1..x2. They want to MINIMIZE the cost f(x) = 0.186*x1 - 0.983*x2. The choice must satisfy ALL of the following constraints: (C1) -0.967*x1ˆ2 - 6.875*x1*x2 - 3.995*x2ˆ2 + 0.846*x1 + 0.063*x2 - 1.424 <= 0 . . . (five more quadratic constraints) . . . (C7) the Euclidean norm bound sqrt(x1ˆ2 + ... + x2...

  6. [1997]

    NC2C: Automated convexification of generic non-convex optimization problems.arXiv preprint arXiv:2601.04789,

    Xinyue Peng, Yanming Liu, Yihan Cang, Yuwei Zhang, Xinyi Wang, Songhang Deng, and Jiannan Cao. NC2C: Automated convexification of generic non-convex optimization problems.arXiv preprint arXiv:2601.04789,

  7. [1998]

    PICARD: Parsing incrementally for constrained auto-regressive decoding from language models

    Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. PICARD: Parsing incrementally for constrained auto-regressive decoding from language models. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,

  8. [2016]

    NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints

    Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics,

  9. [2017]

    Do NOT think that much for 2+3=? On the overthinking of o1-like LLMs.arXiv preprint arXiv:2412.21187,

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do NOT think that much for 2+3=? On the overthinking of o1-like LLMs.arXiv preprint arXiv:2412.21187,

  10. [2018]

    Stream of search (SoS): Learning to search in language.arXiv preprint arXiv:2404.03683,

    Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D Goodman. Stream of search (SoS): Learning to search in language.arXiv preprint arXiv:2404.03683,

  11. [2021]

    Under review

    11 Preprint. Under review. Linhao Luo, Zicheng Zhao, Gholamreza Haffari, Yuan-Fang Li, Chen Gong, and Shirui Pan. Graph-constrained reasoning: Faithful reasoning on knowledge graphs with large language models.arXiv preprint arXiv:2410.13080,

  12. [2022]

    Chain of thought empowers trans- formers to solve inherently serial problems.arXiv preprint arXiv:2402.12875,

    Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma. Chain of thought empowers trans- formers to solve inherently serial problems.arXiv preprint arXiv:2402.12875,

  13. [2023]

    Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769,

    Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuan- dong Tian. Training large language models to reason in a continuous latent space.arXiv preprint arXiv:2412.06769,

  14. [2024]

    Constraints-of-thought: A framework for constrained reasoning in language-model-guided search.arXiv preprint arXiv:2510.08992,

    Kamel Alrashedy, Vriksha Srihari, Zulfiqar Zaidi, Ridam Srivastava, Pradyumna Tamb- wekar, and Matthew Gombolay. Constraints-of-thought: A framework for constrained reasoning in language-model-guided search.arXiv preprint arXiv:2510.08992,

  15. [2025]

    Solving math word problems with process- and outcome-based feedback.arXiv preprint arXiv:2211.14275,

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback.arXiv preprint arXiv:2211.14275,