Pith. sign in

REVIEW 5 major objections 5 minor 31 references

A 4-billion-parameter model can answer educational questions with verifiable deductions by writing programs for a symbolic solver, and the paper's error analysis says the hard part is choosing which premises to encode, not the deduction its

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 01:10 UTC pith:AAPPHLVT

load-bearing objection A competent, honest competition report whose two headline claims—'correct, verifiable deductions at the 4B scale' and 'premise selection is the residual difficulty'—outrun the evidence; referee-worthy, but needs a faithfulness analysis and released code. the 5 major comments →

arxiv 2607.14735 v1 pith:AAPPHLVT submitted 2026-07-16 cs.CL

CoTu at EXACT 2026: Neuro-Symbolic Reasoning for Transparent Educational QA

Quoc-Khang Tran , Minh-Thien Nguyen , Phu-An Thai , Xuan-Tung Bui , Truong-Thanh Ma , Nguyen-Khang Pham This is my paper
classification cs.CL
keywords educational question answeringneuro-symbolic reasoningprogram-of-thoughtZ3 solverfirst-order logic entailmentpremise selectionsmall language modelsphysics problem solving
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a small, open-weight model can do reliable and transparent educational QA if it is used as a code writer rather than an answer machine. For regulation questions, the model emits a Z3 program whose satisfiability verdict determines the answer; for physics questions, it emits numerical Python that is executed directly. On the EXACT 2026 competition, this approach scored perfectly on all 50 physics items across the two automated rounds and earned the highest final-round technical score (13.44/15), though the equally weighted presentation score moved the overall ranking to 3rd place. The authors conclude that symbolic grounding removes the deduction bottleneck at the 4-billion-parameter scale, and that the remaining errors are predominantly failures of premise selection—identifying and encoding the right supporting premises—not of the logic. If true, this shifts the design target for small-model reasoning systems from making the model smarter at deducting to making it more faithful at translating premises into formal constraints.

Core claim

Central claim: grounding answers in a symbolic solver yields correct, verifiable deductions at the 4-billion-parameter scale, and the residual difficulty is premise selection, not deduction. The system turns regulation queries into Z3 programs: the entailment check treats a query as entailed when the premises plus its negation are unsatisfiable, contradicted when the premises plus the query are unsatisfiable, and uncertain otherwise. Physics queries become numerical Python executed in a sandbox. The paper reports a perfect 25/25 physics score in both automated rounds and the highest final technical score (13.44/15), while its error analysis attributes most Task 1 failures to premise selectio

What carries the argument

The load-bearing mechanism is the SMT-based entailment check (Definition 2): given premises P and query C, Status(P,C) is decided by two satisfiability calls—Sat(P∧¬C) and Sat(P∧C)—mapping UNSAT to Entailed/Contradicted and SAT to Uncertain. Z3 is the SMT solver that executes these calls. The paper's Proposition 1 states this verdict scheme is sound with respect to first-order logic semantics provided the natural-language premises and query are faithfully translated into FOL. The same program-of-thought pattern runs on the physics branch with Python execution instead of Z3. A shared self-correction loop regenerates programs up to two times after parse or execution failures, and a latency fal

Load-bearing premise

The load-bearing premise is that the 4B model translates every natural-language premise and query faithfully into first-order logic; Proposition 1 is stated without proof of that faithfulness, and the paper's own footnote notes that a single mis-encoded premise can change the solver verdict.

What would settle it

Take a Task 1 item the system answered correctly, replace one supporting premise with its logical negation without changing the question text, and re-run the system. If the emitted Z3 program or the final verdict does not change, the encoding is not tracking the premise content, and the soundness proposition's assumption is violated. A more direct check: have a domain expert hand-translate the same premises to first-order logic and compare solver verdicts with the system's; mismatches would quantify translation unfaithfulness.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • 4-billion-parameter models can deliver answers whose deductions are externally verifiable, because the verdict is computed by a solver rather than sampled from the model.
  • For multi-step quantitative tasks, emitting and executing numerical code moves arithmetic and unit handling into the interpreter; the physics task was solved perfectly in both automated rounds.
  • The dominant failure mode on regulation queries becomes premise selection—citing and encoding the right supporting premises—which is scored equally with answer correctness under the competition metric.
  • Distilling reasoning traces from a larger teacher into task-specific fine-tunes improved both accuracy and speed, reducing mean Task 1 response time from 32.5s to 12.1s and Task 2 from 17.2s to 4.2s.
  • A fast self-consistency fallback can keep a slow reasoning-heavy solver within hard time limits; it was needed on 20% of Task 1 items in the base round and on none after fine-tuning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own error split suggests a falsifiable experiment: perturb premise wording or swap premises and measure how often the emitted Z3 encoding changes; if encodings are stable across wording but sensitive to logical content, then translation has a tractable, local failure profile.
  • The same program-of-thought recipe should transfer to other codified domains—tax rules, organizational policies, clinical guidelines—where the knowledge can be expressed as logical constraints and the user wants an auditable chain.
  • Since the physics branch needed no retrieval, a likely corollary is that small models know enough physics formulas; the bottleneck is applying them numerically. Adding unit and dimension checking to the executed code would carry the Task 1 verifiability guarantee into the physics branch.
  • The manuscript itself still contains two 'TODO: insert link to the model' placeholders, so the released SFT checkpoints are not independently locatable from the text as written; that is a reproducibility gap, not a reasoning gap.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper describes CoTu, team Can Tho University's submission to EXACT 2026, a transparent educational QA challenge restricted to open-weight models of at most 8B parameters. The system uses a 4B backbone (GRM-2.5) with a Program-of-Thought architecture: for Task 1 (logical reasoning over university regulations) the backbone emits a Python/Z3 encoding of natural-language premises and query, and the solver's entailment verdict determines the answer; for Task 2 (physics) it emits numerical Python evaluated in a sandbox. Both branches share a self-correction loop, a latency-aware serving stack (SGLang with DFLASH speculative decoding), and a unified JSON output schema. The authors report a perfect Task 2 score in both automated rounds, an improvement in Task 1 P1+P2 from 21.24/25 to 23.10/25 after swapping in SFT checkpoints, and the highest final-round technical score (13.44/15), leading to 3rd place overall. The central claim is that grounding answers in a symbolic solver yields correct, verifiable deductions at the 4B scale, with residual difficulty lying in premise selection rather than deduction.

Significance. If the performance claims are reproducible, this is a useful demonstration that a small open-weight model, combined with symbolic execution, can achieve strong competition results while emitting structured rationales. The paper has several genuine strengths: it formalizes the entailment status and the soundness condition (Definition 2, Proposition 1), is unusually honest about the conditional nature of the SFT attribution, discloses the non-competing teachers and the latency-fallback behavior, and provides a clear split of where the remaining errors concentrate. The perfect Task 2 result across 50 automated items is a striking empirical signal. However, the central scientific claim is currently supported only conditionally: the faithfulness of the neural NL-to-FOL translation is assumed rather than tested, the error analysis does not separate translation failures from deduction failures, and the formal treatment does not account for Z3's UNKNOWN path. These gaps are fixable in a revision, but they are load-bearing for the abstract's conclusions. The manuscript also has reproducibility gaps: model checkpoints are marked TODO and no run logs or code are released, so the official lead

major comments (5)
  1. [§3.2, Proposition 1; §4.1 footnote] The soundness proposition is conditional on faithful translation of natural-language premises into FOL formulas P and C, but this translation is performed by the 4B backbone with no verification. The self-correction loop checks only parse/execution success, not semantic faithfulness. The footnote in §4.1 concedes that a single mis-encoded premise can change the solver verdict. Therefore the abstract's claim that grounding in a symbolic solver yields "correct, verifiable deductions" does not transfer end-to-end; it is contingent on an untested neural encoding step. Please add a faithfulness evaluation against the provided gold FOL translations (which the paper states exist in the training set), or explicitly restrict the soundness claim to the solver-internal step.
  2. [§5.3, first paragraph] The error analysis splits Task 1 failures into 'wrong answer' and 'wrong premises' without specifying the classification procedure. Because a mis-encoded premise can produce a wrong solver verdict, the 'wrong answer' category conflates translation errors with deduction errors. The conclusion that 'premise selection remains the dominant Task 1 failure mode' and the broader claim that residual difficulty lies in premise selection rather than deduction are therefore not supported by the reported split. Please describe how the two categories were assigned and, ideally, redo the analysis while controlling for translation fidelity (e.g., by comparing the emitted FOL encoding to gold FOL translations on the failure cases).
  3. [Definition 2, Eq. (5); Figure 1] Sat(Φ) is defined as a Boolean function returning True or False, but Z3 is an SMT solver that can return UNKNOWN for formulas outside decidable fragments. Figure 1 explicitly includes an UNK path, yet Proposition 1 does not cover it. If the Task 1 encoding is restricted to a decidable fragment, that restriction should be stated and proved to make Status total; otherwise the definition and proposition should be amended to map UNKNOWN to the Uncertain branch and the soundness argument adjusted accordingly. This is a formal gap in the paper's main correctness claim, even if it does not affect the empirical scores.
  4. [§3.4, final sentence; §5.2] The manuscript states that replacing the base backbone with the SFT checkpoints between Round 1 and Round 2 'raises accuracy and cuts response time,' but Section 5.2 itself acknowledges that the two rounds use different item sets and that Round 2 also refined the surrounding pipeline. This is not a controlled comparison, so the causal attribution to SFT is confounded. Since RQ3 is framed as a central research question, please either present the round-over-round change only as suggestive evidence or add an ablation that varies the checkpoint while holding items and pipeline fixed.
  5. [§5.1, Tables 3 and 4] The central empirical results—perfect Task 2 scores, the Task 1 improvement, and the final-round technical score—are self-reported from official leaderboards, but the model checkpoints are marked TODO and no code or run logs are provided. This makes the claims impossible to verify from the manuscript. Please release the checkpoints, code, and, if permitted, the official evaluation artifacts or a detailed per-item log, so reviewers and readers can reproduce the reported scores.
minor comments (5)
  1. [Title and abstract] The title renders inconsistently as 'CoT u' / 'CoTu' / '\cotu{}'; please standardize to CoTu throughout.
  2. [Table 5] The column header 'Fallback Bonus Task 1 B' is confusing. The fallback count and the time bonus are different quantities; separate them or relabel for clarity.
  3. [§4.2] The 54s fallback threshold and the fixed retry budget of two iterations are stated, but the self-consistency sample count M is not given a concrete value in the main text. The exact M should be reported here rather than only implied from Section 3.2.
  4. [§5.2, Table 5] Mean response times are computed over different numbers of items across rounds (25 vs 6 Task 1 items in Round 3). State the number of items in every cell or separately note that Round 3's means are based on only 6 Task 1 items.
  5. [References] Reference [10] is cited for rank-stabilized LoRA, but the citation format is inconsistent with the rest of the bibliography (author name only). Please align it with the reference style used elsewhere.

Circularity Check

0 steps flagged

No significant circularity: reported scores are external competition outcomes, and the only conditional result rests on an unverified faithfulness assumption, which is a soundness risk rather than a circular derivation.

full rationale

None of the enumerated circularity patterns is present. The paper's central empirical claims — perfect Task 2 scores in both automated rounds and the highest final-round technical score (13.44/15) — are official leaderboard outcomes, not predictions produced by fitting a parameter to the evaluation data. The Task 1 deduction chain rests on Definition 2 and Proposition 1: Status(P,C) is defined via satisfiability checks, and Proposition 1 states that an Entailed verdict implies classical entailment. This proposition is true by construction (classical entailment is exactly unsatisfiability of P ∧ ¬C), but it is presented as a soundness lemma rather than as a derived empirical result, and no fitted quantity is renamed as a prediction from it. The actual weakness is that Proposition 1 assumes faithful NL→FOL translation, while the 4B backbone performs that translation and the paper's own footnote concedes that 'a single mis-encoded premise can change the solver verdict'; no faithfulness test is reported, and the UNKNOWN path of Z3 is not addressed. That is an end-to-end soundness gap, not circularity: it makes the central claim less supported, but it does not make the derivation equivalent to its inputs. Similarly, the §5.3 error analysis classifies P2 misses as 'wrong premises' and P1 misses as 'wrong answer, where the deduction itself is incorrect' without a method for separating translation failures from deduction failures; this is an unsupported interpretation, not a by-construction reduction. The paper's citations are to external tools and standard methods (Z3, DSPy, LoRA, SGLang, RRF), and no self-citation or imported uniqueness theorem is load-bearing for the central claim. Accordingly, the derivation chain is not circular.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The central claim rests on two external correctness assumptions: faithful NL-to-FOL translation and Z3's coverage of the encoded logic. No free parameters are fitted to produce the reported scores; the listed engineering constants are hand-chosen and do not affect the bottleneck claim. The paper introduces no new physical or logical entities.

free parameters (4)
  • Self-consistency sample count M = 3
    Section 4.2: number of non-thinking candidate paths in the Task 1 fallback runner. Hand-chosen; not fitted to data in the paper.
  • Latency fallback threshold t_fallback = 54s
    Section 4.2: safety timeout before the fast self-consistency runner replaces the main thinking solver. Hand-chosen engineering constant.
  • RRF smoothing constant k = 60
    Equation (4), Section 3.2: constant in Reciprocal Rank Fusion. Hand-set; retrieval is only used in Round 1 and is not part of the final system.
  • LoRA rank and alpha = r=16, alpha=32
    Table 6: LoRA hyperparameters. Chosen by standard practice; not shown to be optimized.
axioms (3)
  • domain assumption Natural-language premises and query are faithfully translated into FOL formulas P and C
    Proposition 1 states the soundness of symbolic grounding is conditional on this. The 4B backbone performs the translation; no mechanism or evaluation ensures faithfulness. If translation is wrong, the Z3 verdict is sound for the encoded formulas but not for the actual question.
  • domain assumption Z3 verdicts are sound for full classical first-order logic
    Definition 2 and Proposition 1 present the entailment test over FOL formulas, but Z3 is an SMT solver for decidable fragments, not a complete FOL theorem prover. The paper does not restrict the encoding to a decidable fragment or formally handle the UNK outcome shown in Figure 1.
  • domain assumption Physics laws and constants are correctly recovered from parametric memory
    Section 3.3: Task 2 provides only the raw question text; the model must recall the correct physical laws and formulas. There is no verifier for law selection, only for arithmetic execution.

pith-pipeline@v1.3.0-alltime-deepseek · 14862 in / 10980 out tokens · 98085 ms · 2026-08-02T01:10:19.285633+00:00 · methodology

0 comments
read the original abstract

Transparent educational question answering asks for answers that are not only correct but explainable, and doing so with small models rules out the reasoning power of the largest proprietary systems. The EXACT 2026 competition poses this problem concretely: open-weight language models of at most 8B parameters, self-hosted, with a natural-language explanation for every answer. It pairs two tasks: logical reasoning over university regulations, and multi-step physics problem solving. We describe the system that team \cotu{} developed to address both, a neuro-symbolic Program-of-Thought pipeline in which a 4B backbone writes a program rather than stating an answer directly: for regulation queries it emits a Z3 encoding whose entailment verdict grounds the deduction, and for physics it emits numerical Python, both wrapped in a shared self-correction loop and a unified explained-JSON output. Answer-type routing, distillation-based task fine-tuning, and a latency-aware serving stack -- SGLang with speculative decoding -- keep the system within the 60-second per-query limit. The system achieved a \textbf{perfect score} on the physics task in both automated selection rounds and obtained the \textbf{highest final-round technical score} of any team -- $13.44/15$, combining automated answer evaluation with expert-judged reasoning depth -- with the equally weighted presentation score included, \cotu{} placed 3rd overall. Grounding answers in a symbolic solver yields correct, verifiable deductions at the 4B scale, and the residual difficulty lies in premise selection rather than the deduction itself.

Figures

Figures reproduced from arXiv: 2607.14735 by Minh-Thien Nguyen, Nguyen-Khang Pham, Phu-An Thai, Quoc-Khang Tran, Truong-Thanh Ma, Xuan-Tung Bui.

Figure 1
Figure 1. Figure 1: The CoTu dual-task pipeline. A query, tagged with its task identifier, first re￾ceives task-specific rich prompting: translation and symbolic-error guidance for Task 1, and units, constants, and formula guidance for Task 2. The Task 1 branch routes the query by answer type, then generates a PoT that runs Z3 through Python, whereas the Task 2 branch generates PoT Python that is evaluated directly. Both bran… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 13 linked inside Pith

  1. [1]

    In: Advances in Neural Information Processing Sys- tems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020

    Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Win- ter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford...

    2020

  2. [2]

    ACM Transactions on Intelligent Systems and Technology15(3), 1–45 (2024)

    Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., Ye, W., Zhang, Y., Chang, Y., Yu, P.S., Yang, Q., Xie, X.: A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology15(3), 1–45 (2024). https://doi.org/10.1145/3641289

    doi:10.1145/3641289 2024

  3. [3]

    arXiv preprint arXiv:2602.06036 (2026), https://arxiv.org/abs/2602.06036

    Chen, J., Liang, Y., Liu, Z.: DFlash: Block diffusion for flash speculative decoding. arXiv preprint arXiv:2602.06036 (2026), https://arxiv.org/abs/2602.06036

    Pith/arXiv arXiv 2026

  4. [4]

    In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval

    Cormack, G.V., Clarke, C.L.A., Buettcher, S.: Reciprocal rank fusion outper- forms condorcet and individual rank learning methods. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 758–759. SIGIR ’09, Association for Computing Ma- chinery, New York, NY, USA (2009). https://doi.org/10.11...

    arXiv 2009

  5. [5]

    Nature645, 633–638 (2025)

    DeepSeek-AI: DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. Nature645, 633–638 (2025). https://doi.org/10.1038/ s41586-025-09422-z, https://arxiv.org/abs/2501.12948

    Pith/arXiv arXiv 2025

  6. [6]

    Arti- ficial Intelligence Review56, 12387–12406 (2023)

    d’Avila Garcez, A., Lamb, L.C.: Neurosymbolic AI: the 3rd wave. Arti- ficial Intelligence Review56, 12387–12406 (2023). https://doi.org/10.1007/ s10462-023-10448-w

    2023

  7. [7]

    In: International Conference on Learning Representations (ICLR) (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (ICLR) (2022)

    2022

  8. [8]

    els: Principles, taxonomy, challenges, and open questions

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., Liu, T.: A survey on hallucination in large language mod- 18 Quoc-Khang et al. els: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems43(2), 1–55 (2025). https://doi.org/10.1145/3703155

    doi:10.1145/3703155 2025

  9. [9]

    ACM Computing Surveys55(12), 1–38 (2023)

    Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM Computing Surveys55(12), 1–38 (2023). https://doi.org/10.1145/3571730

    doi:10.1145/3571730 2023

  10. [10]

    arXiv preprint arXiv:2312.03732 (2023)

    Kalajdzievski, D.: A rank stabilization scaling factor for fine-tuning with LoRA. arXiv preprint arXiv:2312.03732 (2023)

    Pith/arXiv arXiv 2023

  11. [11]

    In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers)

    Kamalloo, E., Dziri, N., Clarke, C., Rafiei, D.: Evaluating open-domain question answering in the era of large language models. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers). pp. 5591–5606. Association for Computational Linguistics, Toronto, Canada (Jul 2023). https://doi.org/10.18653...

    doi:10.18653/v1/2023.acl-long.307 2023

  12. [12]

    arXiv preprint arXiv:2310.03714 (2023), https://arxiv.org/ abs/2310.03714

    Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Vard- hamanan, S., Haq, S., Sharma, A., Joshi, T.T., Moazam, H., Miller, H., Za- haria, M., Potts, C.: DSPy: Compiling declarative language model calls into self- improving pipelines. arXiv preprint arXiv:2310.03714 (2023), https://arxiv.org/ abs/2310.03714

    Pith/arXiv arXiv 2023

  13. [13]

    In: Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS

    Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. In: Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS

    2022

  14. [14]

    arXiv preprint arXiv:2402.06196 (2024), https://arxiv.org/abs/2402.06196

    Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., Gao, J.: Large language models: A survey. arXiv preprint arXiv:2402.06196 (2024), https://arxiv.org/abs/2402.06196

    Pith/arXiv arXiv 2024

  15. [15]

    In: Tools and Algorithms for Construction and Analysis of Systems

    de Moura, L., Bjørner, N.: Z3: an efficient smt solver. In: Tools and Algorithms for Construction and Analysis of Systems. pp. 337–340. Springer, Berlin, Hei- delberg (March 2008), https://www.microsoft.com/en-us/research/publication/ z3-an-efficient-smt-solver/

    2008

  16. [16]

    In: ITADATA (2025), https://ceur-ws.org/Vol-4152/paper98.pdf

    Nguyen, L.S.T., Vo, K.H.N., Nguyen, T.H.A., Bui, T.C., Nguyen, D.Q., Tran, T.T., Nguyen, A.D., Nguyen, M.L., Baldacci, F., Bui, T.H., Nardo, E.D., Ciaramella, A., Le, S.H., Ullah, I., Rocco, L.D., Quan, T.T.: Bridging llms and symbolic reason- ing in educational qa systems: Insights from the xai challenge at ijcnn 2025. In: ITADATA (2025), https://ceur-ws...

    2025

  17. [17]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P.F., Leike, J., Lowe, R.: Training language models to follow instructions with human feedback. In: Advances in Neural Information Process...

    2022

  18. [18]

    Transactions on Machine Learning Research (2024), https: //arxiv.org/abs/2312.06585

    Singh, A., Co-Reyes, J.D., Agarwal, R., Anand, A., Patil, P., Garcia, X., Liu, P.J., Harrison, J., Lee, J., Xu, K., Parisi, A., Kumar, A., Alemi, A., Rizkowsky, A., Nova, A., et al.: Beyond human data: Scaling self-training for problem-solving with language models. Transactions on Machine Learning Research (2024), https: //arxiv.org/abs/2312.06585

    Pith/arXiv arXiv 2024

  19. [19]

    arXiv preprint arXiv:2302.13971 (2023), https://arxiv.org/abs/2302.13971

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, CoTu at EXACT 202619 E., Lample, G.: LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023), https://arxiv.org/abs/2302.13971

    Pith/arXiv arXiv 2023

  20. [20]

    Nature Methods17, 261–272 (2020)

    Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Courna- peau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, İ., Feng, Y., Moore, E.W., VanderPlas, J., Lax- alde, D., Perktold, J., Cim...

    2020

  21. [21]

    IEEE Signal Processing Magazine42(6), 51–63 (2025)

    Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P.S., Wen, Q.: Large language models for education: A survey and outlook. IEEE Signal Processing Magazine42(6), 51–63 (2025). https://doi.org/10.1109/MSP.2025.3594309, https: //doi.org/10.1109/MSP.2025.3594309

    arXiv 2025

  22. [22]

    Advances in neural information processing systems35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, 24824–24837 (2022)

    2022

  23. [23]

    In: Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024 (2024), https://arxiv

    Wei, Y., Cassano, F., Liu, J., Ding, Y., Jain, N., Mueller, Z., de Vries, H., von Werra, L., Guha, A., Zhang, L.: SelfCodeAlign: Self-alignment for code generation. In: Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024 (2024), https://arxiv. org/abs/2410.24198

    Pith/arXiv arXiv 2024

  24. [24]

    In: Proceedings of the Thirty-Fourth Inter- national Joint Conference on Artificial Intelligence, IJCAI-25

    Yang, X.W., Shao, J.J., Guo, L.Z., Zhang, B.W., Zhou, Z., Jia, L.H., Dai, W.Z., Li, Y.F.: Neuro-symbolic artificial intelligence: Towards improving the reasoning abilities of large language models. In: Proceedings of the Thirty-Fourth Inter- national Joint Conference on Artificial Intelligence, IJCAI-25. pp. 10770–10778. International Joint Conferences on...

    doi:10.24963/ijcai.2025/1195 2025

  25. [25]

    arXiv preprint arXiv:2503.14476 (2025), https://arxiv.org/abs/2503.14476

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Fan, T., Liu, G., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, W., Zhu, H., Chen, J., Chen, J., Wang, C., Yu, H., Song, Y., Wei, X., Zhou, H., Liu, J., Ma, W.Y., Zhang, Y.Q., Yan, L., Qiao, M., Wu, Y., Wang, M.: DAPO: An open-source LLM reinforcement l...

    Pith/arXiv arXiv 2025

  26. [26]

    arXiv preprint arXiv:2308.01825 (2023), https://arxiv.org/abs/2308.01825

    Yuan, Z., Yuan, H., Li, C., Dong, G., Lu, K., Tan, C., Zhou, C., Zhou, J.: Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825 (2023), https://arxiv.org/abs/2308.01825

    Pith/arXiv arXiv 2023

  27. [27]

    Zelikman, E., Wu, Y., Mu, J., Goodman, N.D.: STaR: Bootstrapping rea- soning with reasoning. In: Advances in Neural Information Processing Sys- tems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022 (2022), https://proceedings.neurips.cc/paper_files/paper/2022/ hash/639a9a172c044fbb64175b5fad42e9a5-Abstract-Conference.html

    2022

  28. [28]

    In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Zeng, H., Jiang, D., Wang, H., Nie, P., Chen, X., Chen, W.: AceCoder: Acing coder RL via automated test-case synthesis. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Asso- ciation for Computational Linguistics (2025), https://arxiv.org/abs/2502.01718

    Pith/arXiv arXiv 2025

  29. [29]

    arXiv preprint arXiv:2303.18223 (2023), https://arxiv.org/abs/2303.18223 20 Quoc-Khang et al

    Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J.Y., Wen, J.R.: A survey of large language models. arXiv preprint arXiv:2303.18223 (2023), https://arxiv.org/abs/2303.18223 20 Quoc-Khang et al

    Pith/arXiv arXiv 2023

  30. [30]

    Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C.H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J.E., Barrett, C., Sheng, Y.: SGLang: Efficient execution of structured language model programs. arXiv preprint arXiv:2312.07104 (2024), https://arxiv.org/abs/2312.07104 CoTu at EXACT 202621 Table 6.LoRA configuration used to produce the Task 1 and Tas...

    Pith/arXiv arXiv 2024

  31. [2022]

    22199–22213 (2022), https://proceedings.neurips.cc/paper_files/paper/ 2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html

    pp. 22199–22213 (2022), https://proceedings.neurips.cc/paper_files/paper/ 2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html

    2022