Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Syntactic Control of Language Models by Posterior Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Posterior inference with a syntactic tagger can enforce target constituency structures at generation time, lifting F1 from 12–35 to about 93 on GPT-2-large and Llama-3-8B.

desk verdict Strong empirical F1 gains for syntactic control, but the stated SMC proposal is not fully specified, so the paper's posterior-inference framing outruns its algorithm. read the letter →

arxiv 2506.07154 v1 pith:XVRJXO5H submitted 2025-06-08 cs.CL cs.AIcs.LG

Vicky Xefteri , Tim Vieira , Ryan Cotterell , Afra Amini This is my paper
classification cs.CLcs.AIcs.LG
keywords syntacticcontrolposteriorinferencesequentialMonteCarlocontrolledtextgenerationconstituencyparsingtetrataggershapingfunctionlanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that syntactic control of language-model generation should be treated as posterior inference: condition the model's prior over strings on the event that the generated string has the user-specified constituency tree. Since the posterior is intractable, the authors run sequential Monte Carlo with an autoregressive Tetratagger as a shaping function that scores partial sentences for their syntactic promise. With a well-chosen proposal distribution that mixes the language model with a part-of-speech bigram, the method lifts bracketing F1 from 12.31 (GPT-2-large) and 35.33 (Llama-3-8B) to about 93 in both cases while keeping fluency roughly intact. A sympathetic reader would care because it suggests that small open models can rival much larger closed models at structure-following tasks, without fine-tuning.

What carries the argument

The load-bearing object is the shaping function ψ, an autoregressive Tetratagger: a constituency parser-as-tagger that assigns each word a pair of tags encoding the word's position in the binarized tree, but conditioned only on the prefix generated so far rather than the full sentence. In SMC, ψ enters every weight update as the ratio ψ(y·y')/ψ(y), which approximates the posterior predictive probability of the next token; the second ingredient is the informed proposal Q(y') ∝ p(y'_1|y<n)·q(y'|z_n z_{n+1}), a product of the language model's first-token probability with a part-of-speech bigram, which makes long-horizon syntactic success likely enough that only a handful of particles are needed. SMC resampling, triggered by the effective sample size, then reallocates compute toward promising partial strings.

What would settle it

Re-run the SMC experiments with the exact normalizer of Q in Eq. (11) computed and included in the weight updates; if the F1 scores or particle weights change materially, or if an M=1 run with Q∝pq diverges from the posterior, the reported ~93 F1 is not an unbiased posterior sample. A simpler check: compare SMC (M=1) with Q∝pq to SMC (M=1) with Q=p on the same trees and confirm the weights target the same posterior up to the shaping approximation.

Watch

Extended reading notes

Core claim

Controlled generation under a target constituency tree t is cast as sampling from the posterior P(y|t) ∝ p(y)φ(t|y), where p is the language model prior and φ is a Tetratagger scoring the tree–string fit. Because the normalizing constant and posterior are intractable, the paper uses SMC: M particles drawn from a proposal Q are extended token-by-token, weighted by the ratio of the shaping function ψ (an autoregressive Tetratagger that sees only the prefix) to the proposal probability, and resampled when the effective sample size drops. The central empirical discovery is that the choice of proposal is decisive: with Q = p, SMC improves F1 but remains far from perfect, whereas with Q ∝ p·q, where q is a bigram model over part-of-speech tags, the F1 reaches roughly 93 for both GPT-2-large and Llama-3-8B, with the log-likelihood under the syntax tagger near zero while the model's log-prior barely degrades. The same method makes smaller open models competitive with GPT-4, which, the paper shows, fails to follow such structures even with gold exemplars in the prompt.

Load-bearing premise

The proposal in Eq. (11) is defined only up to a constant, yet the algorithm divides by Q in every weight update without computing or cancelling that constant; if this prefix-dependent normalizer is not accounted for, the SMC weights are biased and the sampler no longer targets the stated posterior.

Editorial extensions

If this is right

  • Smaller open models become competitive with much larger closed ones: GPT-2-large reaches ~93 F1 with Q∝pq, a score GPT-4 does not approach even with gold exemplars in the prompt.
  • The proposal distribution is the decisive component: SMC with Q=p roughly doubles F1 (from 12.31 to 28.26 for GPT-2-large), while Q∝pq is what reaches ~93.
  • Fluency is not traded away: with Q∝pq, −logφ falls to near zero while −logp stays close to or better than baseline under SMC.
  • A handful of particles suffices: with the informed proposal, M=6 reaches near-peak F1 and gains from more particles are marginal.
  • The cost of the informed proposal is lexical diversity, which drops markedly for GPT-2-large under Q∝pq.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The recipe is generic: any left-to-right scoring function that estimates how well a prefix will end up satisfying a global constraint can replace the Tetratagger, suggesting transfers to meter, rhyme, or grammar-constrained code generation.
  • The unstated normalization of Eq. (11) is a testable weak point: if the prefix-dependent normalizer must be divided out for unbiasedness, ignoring it could shift the reported F1 slightly; computing it for a few prefixes would settle whether it matters.
  • Interpolating Q∝pq with the prior (or adding a diversity penalty at resampling) is a natural extension that could recover the lost lexical diversity while keeping F1 high.
  • Because the method requires a tagger whose tokenizer matches the sampling LM, multilingual use hinges on training autoregressive Tetrataggers per language; the single-language experiments leave that as an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an inference-time method for controlling the constituency structure of language-model output. The target syntax is represented as a Penn Treebank tree with leaves replaced by question marks, and a tetratagger likelihood φ(t|y) is combined with an LM prior p(y) to define a posterior over strings. The authors use sequential importance sampling and sequential Monte Carlo with an autoregressive tetratagger as a shaping function, and experiment with two proposal distributions: the prior Q=p and a syntax-informed proposal Q ∝ p q, where q is a POS-conditioned bigram model. On GPT2-large and Llama3-8B (base, 0-shot, and 5-shot), the method raises bracketing F1 from values around 12–35 to roughly 93 in the best configuration. The paper also reports GPT4 prompting baselines showing that instruction-tuned models struggle with this task. The central claim is that posterior inference with an appropriate proposal can enforce a target syntactic structure without compromising fluency.

Significance. If the reported results are correct, this is a practically useful and non-tautological demonstration of inference-time syntactic control: the optimization objective is a BERT-tetratagger likelihood, while the evaluation metric is Berkeley-parser bracketing F1, so improving the objective is not trivially identical to improving the metric. The paper ships a code repository, reports mean and standard deviation over multiple runs, and includes a parameter study over the number of SMC particles. The method is also applied to instruction-tuned Llama3-8B, where existing prompting baselines fail. These strengths support the promise of the approach, provided the algorithmic details are made precise.

major comments (4)
  1. [§5.3, Eq. (11), and Algorithm 2] The proposal in Eq. (11) is defined only up to a prefix- and tag-dependent normalizer: Q(y'|y_{<n}) ∝ p(y'_1|y_{<n}) q(y'|z_n z_{n+1}). Algorithm 2, however, treats Q as a properly normalized conditional distribution when it divides by Q(y'|y^{(m)}) in lines 9 and 11. The omitted normalizer Z_n(y_{<n}; z_n,z_{n+1}) = Σ_{y'} p(y'_1|y_{<n}) q(y'|z_n,z_{n+1}) depends on the prefix and on the target POS tags, so it is not a global constant. If this normalizer is not computed, each incremental weight is off by a step-dependent factor and the self-normalized particle weights no longer approximate the stated posterior P(y|t). The paper must specify the exact normalized form of Q, explain how Z_n is computed efficiently, or justify that omitting it induces only negligible bias. As written, the central 'posterior inference' interpretation of the reported gains is not supported.
  2. [§4 and Algorithm 2, token-level vs. word-level proposal] Eq. (11) defines a distribution over words y', while Algorithm 2 samples token-level symbols y' and updates weights at every token. The 'Multi-token words' paragraph says that when the sampled token is not the last token of a word, the method 'keep[s] sampling from the LM until we hit the last token,' but it does not specify the proposal probability Q(y'|·) for intermediate tokens of a multi-token word, nor does it state whether shaping ratios are applied only at word-final tokens or at every token. Without this specification, the per-token weight updates in Algorithm 2 are not well defined for the actual decoding procedure, and the correctness argument in §2.2 does not cover the implemented algorithm. The authors should provide a precise token-level formulation of Q and the corresponding weight recursion.
  3. [Abstract and §5.4, Table 3] The abstract's claim that the method improves syntactic accuracy 'without compromising the language model's fluency' is contradicted by the authors' own logp results for the Q=p proposal. In Table 3, GPT2-large logp degrades from −52.64 to −56.23 under SIS and to −58.56 under SMC; for Llama3-8B (not instruction-tuned) the degradation is much larger, from −62.47 to −79.79 (SIS) and −117.88 (SMC). The text acknowledges a 5-point degradation for GPT2-large but calls it 'slight,' and it does not address the 55-nat degradation for Llama3-8B. The fluency claim is defensible only for the Q∝pq proposal, where logp improves relative to the same-proposal baseline; the authors should either restrict the claim to that setting or provide a task-level measure of fluency that supports the stronger statement.
  4. [§2.2, admissibility of the shaping function] The correctness of SMC with shaping relies on the shaping function being admissible: p(y)ψ(y)=0 must imply φ(y·y')=0 for every continuation y'. The paper states that the autoregressive tetratagger is used as ψ but does not show that it satisfies this condition, e.g., that it never assigns exactly zero probability to a prefix that could still be completed to a string with positive tetratagger likelihood. If softmax outputs are used, ψ may be positive everywhere, but the paper should state this explicitly; otherwise the 'same guarantees as importance sampling' claim in §2.2 is not justified for the implemented system.
minor comments (5)
  1. [Algorithm 2, lines 17–25] In the RESAMPLE procedure, the 'temporary copy' in lines 21–24 appears to be an implementation artifact; the pseudocode as printed does not make clear which variables are being updated in place. This is confusing but easily fixed.
  2. [§5.4 and Figure 3] Figure 3 is captioned '−logφ and −logp' but the axis labels are unclear; please label the two panels explicitly so it is obvious which metric each histogram refers to.
  3. [§4, footnote 20] The statement that the shaping ratio ψ(y·y')/ψ(y) 'simplifies to' ψ(t_{2n−1}|y·y')ψ(t_{2n}|y·y') appears to omit the denominator terms from the factorization in Eq. (9). Please clarify the exact expression for a prefix of arbitrary length.
  4. [§A.2] Please report the number of trainable parameters and the exact LoRA configuration used for the Llama3-8B tetratagger, since the paper currently says only that 1.06% of parameters are trainable.
  5. [§5.3, footnote 23] The sentence 'This choice was made for efficiency, as evaluating the probability of all possible words is prohibitive' is vague; please clarify whether the difficulty is in evaluating p(y') for all words, in normalizing Eq. (11), or in sampling from q.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the posterior target, proposal, shaping function, and evaluation metric are distinct quantities; the Eq. (11) normalizer issue is a correctness gap, not a circular reduction.

full rationale

The paper's derivation chain is self-contained. The stated target is the posterior P(y|t) ∝ p(y) φ(t|y) (Eq. 2), where p is the LM prior and φ(t|y) is the Kitaev–Klein tetratagger likelihood (Eq. 7), an external pre-trained parser. The proposals are either Q = p (Eq. 10) or the product Q(y'|y<n) ∝ p(y'_1|y<n) q(y'|z_n z_{n+1}) (Eq. 11); neither is defined in terms of the evaluation metric. The shaping function ψ is an autoregressive tetratagger (Eq. 9) trained on the Penn Treebank, not fitted to the evaluation targets, and the SMC weight updates (Alg. 2) are the standard importance ratios p·φ/(Q·ψ) with the ψ factors telescoping, so the procedure targets the stated posterior. The evaluation F1 is computed with the Berkeley Neural Parser, a different model from the tetratagger φ, so maximizing φ is not identical to maximizing the reported F1 by construction; the reported gains are empirical. The self-citations (parsing-as-tagging; Loula et al. 2025) provide background and are not load-bearing, and no uniqueness theorem is imported to force the choice of tetratagger. The unnormalized proportionality in Eq. (11) is a legitimate algorithmic concern: if the prefix- and tag-dependent normalizer is not computed, the proposal is improper and the particle weights are biased, so the method would no longer be unbiased posterior inference. That is a correctness risk, not a circular reduction, because no equation in the paper defines the prediction in terms of the input or renames a fitted parameter as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or theoretical entities. Its free parameters are the SMC hyperparameters and the two learned models (the POS-bigram proposal and the autoregressive tetratagger), whose weights are fitted to Penn Treebank and not shared. The key unstated assumptions are the normalization of the proposal and the admissibility of the shaping function.

free parameters (4)
  • M (particle count) = 20 for Q=p, 6 for Q∝pq
    Chosen by hand; Figure 5 shows F1 plateaus beyond 6 particles, so M is a tuning decision that affects the results.
  • τ (resampling threshold) = 0.25
    Chosen by hand; controls when resampling triggers and thus the bias-variance trade-off of SMC.
  • POS-bigram model q = trained on PTB, distribution not reported
    The strong proposal depends on this model; its parameters are fitted to Penn Treebank and not disclosed.
  • autoregressive tetratagger ψ = trained on PTB, 2 epochs, LoRA, 1.06% trainable params
    The shaping function is learned and its weights are not released; the method's behavior depends on it.
assumptions (4)
  • standard math Language model p defines a proper distribution over strings (Eq. 1)
    Defines the prior; standard autoregressive sampling assumption.
  • domain assumption φ(t|y), the BERT tetratagger, is a faithful likelihood for 'y has syntax t'
    The posterior and all experiments treat φ as the objective; its own parser errors (95.4 F1) are inherited by the method.
  • ad hoc to paper The learned shaping function ψ is admissible (Section 2.2)
    The paper states the admissibility condition but does not verify it for the trained autoregressive tetratagger.
  • ad hoc to paper The proposal Q in Eq. (11) is correctly normalized in the SMC weights
    Q is defined up to proportionality; Algorithm 2 divides by Q and no normalizer computation is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Syntactic Control of Language Models by Posterior Inference." pith.science (2026). https://pith.science/paper/XVRJXO5H

@misc{pith2026250607154,
  author       = {Pith},
  title        = {Pith review of: Syntactic Control of Language Models by Posterior Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XVRJXO5H}},
  note         = {Machine review of arXiv:2506.07154}
}
abstract

Controlling the syntactic structure of text generated by language models is valuable for applications requiring clarity, stylistic consistency, or interpretability, yet it remains a challenging task. In this paper, we argue that sampling algorithms based on the posterior inference can effectively enforce a target constituency structure during generation. Our approach combines sequential Monte Carlo, which estimates the posterior distribution by sampling from a proposal distribution, with a syntactic tagger that ensures that each generated token aligns with the desired syntactic structure. Our experiments with GPT2 and Llama3-8B models show that with an appropriate proposal distribution, we can improve syntactic accuracy, increasing the F1 score from $12.31$ (GPT2-large) and $35.33$ (Llama3-8B) to about $93$ in both cases without compromising the language model's fluency. These results underscore both the complexity of syntactic control and the effectiveness of sampling algorithms, offering a promising approach for applications where precise control over syntax is essential.

Figures

Figures reproduced from arXiv: 2506.07154 by the authors.

Figure 1
Figure 1. Example user–system interaction. The user [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. F1 score across all models and methods using [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. − log φ and − log p for different methods, pro￾posal distributions Q = p and Q = pq and GPT2-large and Llama3-Instruct (0 shots) models. Our approach does not compromise fluency for syntactic consistency. 5.4 Controlled Generation Results25 We repeat each experiment five times for GPT2-large and two times for Llama3-8B models. The experiments were conducted using τ = 0.25 (for SMC), M = 20 when Q = p and M = 6when Q… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Diversity metric across methods and GPT2-large and Llama3-Instruct (0 shots) models. Q ∝ p q proposal decreases diversity of generated sen￾tences, especially in the case of GPT2-large model. up to 93% F1-score, making smaller models com￾petitive to larger ones. However…
Figure 6
Figure 6. Figure 6: Example of a tag sequence given a tree generated by the deterministic algorithm of [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: The prompt used for generating text with [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 15 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Afra Amini and Ryan Cotterell. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.607 On parsing as tagging . In Proceedings of the Conference on Empirical Methods in Natural Language Processing

    doi:10.18653/v1/2022.emnlp-main.607 2022

  4. [4]

    Afra Amini, Li Du, and Ryan Cotterell. 2023. https://openreview.net/forum?id=vf77fTbgG3 Structured Voronoi sampling . In Proceedings of the Conference on Neural Information Processing Systems

    work page 2023

  5. [5]

    Dhananjay Ashok and Barnabas Poczos. 2024. https://arxiv.org/abs/2405.01490 Controllable text generation in the instruction-tuning era . Computing Research Repository, arXiv:2405.01490

    work page Pith review arXiv 2024

  6. [6]

    Britton, Shawn M

    Bruce K. Britton, Shawn M. Glynn, Bonnie J Meyer, and M. J. Penland. 1982. https://psycnet.apa.org/record/1982-20298-001 Effects of text structure on use of cognitive capacity during reading . Journal of Educational Psychology, 74(1)

    work page 1982

  7. [7]

    Sourav Chatterjee and Persi Diaconis. 2017. https://arxiv.org/abs/1511.01437 The sample size required in importance sampling . Preprint, arXiv:1511.01437

    arXiv 2017

  8. [8]

    Mingda Chen, Qingming Tang, Sam Wiseman, and Kevin Gimpel. 2019. https://doi.org/10.18653/v1/P19-1599 Controllable paraphrase generation with a syntactic exemplar . In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    work page doi:10.18653/v1/p19-1599 2019

Show all 42 references
  1. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://arxiv.org/abs/1810.04805 Bert: Pre-training of deep bidirectional transformers for language understanding . Preprint, arXiv:1810.04805

    2019 arXiv

  2. [10]

    Arnaud Doucet, Nando De Freitas, and Neil James Gordon. 2001. https://link.springer.com/book/10.1007/978-1-4757-3437-9 Sequential Monte Carlo Methods in Practice . Springer

    2001 doi

  3. [11]

    Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. https://doi.org/10.18653/v1/N19-1004 Neural language models as psycholinguistic subjects: Representations of syntactic state . In Proceedings of the Conference of the North A me...

    2019 doi

  4. [12]

    Carlos G \'o mez-Rodr \' guez and David Vilares. 2018. https://doi.org/10.18653/v1/D18-1162 Constituent parsing as sequence labeling . In Proceedings of the Conference on Empirical Methods in Natural Language Processing

    2018 doi

  5. [13]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  6. [14]

    David Kauchak. 2013. https://aclanthology.org/P13-1151/ Improving text simplification language modeling using unsimplified text data . In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1537--1546, Sofia, B...

    2013

  7. [15]

    Nikita Kitaev, Steven Cao, and Dan Klein. 2019. https://doi.org/10.18653/v1/P19-1340 Multilingual constituency parsing with self-attention and pre-training . In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    2019 doi

  8. [16]

    Nikita Kitaev and Dan Klein. 2018. https://doi.org/10.18653/v1/P18-1249 Constituency parsing with a self-attentive encoder . In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    2018 doi

  9. [17]

    Nikita Kitaev and Dan Klein. 2020. https://doi.org/10.18653/v1/2020.acl-main.557 Tetra-tagging: Word-synchronous parsing with linear-time inference . In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    2020 doi

  10. [18]

    Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.424 G e D i: Generative discriminator guided sequence generation . In Findings of the Associa...

    2021 doi

  11. [19]

    Lew, Tan Zhi-Xuan, Gabriel Grand, and Vikash K

    Alexander K. Lew, Tan Zhi-Xuan, Gabriel Grand, and Vikash K. Mansinghka. 2023. https://arxiv.org/abs/2306.03081 Sequential Monte Carlo steering of large language models using probabilistic programs . Preprint, arXiv:2306.03081

    2023 arXiv

  12. [20]

    Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. https://doi.org/10.1162/tacl_a_00115 Assessing the ability of LSTM s to learn syntax-sensitive dependencies . Transactions of the Association for Computational Linguistics, 4

    2016 doi

  13. [21]

    Smith, and Yejin Choi

    Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. https://doi.org/10.18653/v1/2021.acl-long.522 DE xperts: Decoding-time controlled text generation with experts and anti-experts . In Proceedings of the Annual Meeti...

    2021 doi

  14. [22]

    Llama Team . 2024. https://arxiv.org/abs/2407.21783 The Llama 3 herd of models . Preprint, arXiv:2407.21783

    2024 arXiv

  15. [23]

    Lew, Tim Vieira, and Timothy J

    João Loula, Benjamin LeBrun, Li Du, Ben Lipkin, Clemente Pasti, Gabriel Grand, Tianyu Liu, Yahya Emara, Marjorie Freedman, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Alexander K. Lew, Tim Vieira, and Timothy J. O'Donnell. 2025. https://openreview.net/forum?id=xoXn62FzD0 ...

    2025

  16. [24]

    Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz

    Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. https://aclanthology.org/J93-2004 Building a large annotated corpus of E nglish: The P enn T reebank . Computational Linguistics, 19(2)

    1993

  17. [25]

    Luca Martino, Víctor Elvira, and Francisco Louzada. 2017. https://doi.org/10.1016/j.sigpro.2016.08.025 Effective sample size for importance sampling based on discrepancy measures . Signal Processing, 131

    2017 doi

  18. [26]

    OpenAI. 2024. https://arxiv.org/abs/2303.08774 GPT-4 technical report . Preprint, arXiv:2303.08774

    2024 arXiv

  19. [27]

    Judea Pearl. 1984. https://mat.uab.cat/ alseda/MasterOpt/Judea_Pearl-Heuristics_Intelligent_Search_Strategies_for_Computer_Problem_Solving.pdf Heuristics - intelligent search strategies for computer problem solving . Addison-Wesley series in artificial intelligence. Addison-Wesley

    1984

  20. [28]

    Gabriel Poesia, Alex Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. https://openreview.net/forum?id=KmtVD97J43e Synchromesh: Reliable code generation from pre-trained language models . In International Conference on Learning Representations

    2022

  21. [29]

    Luan, Dario Amodei, and Ilya Sutskever

    Alec Radford, Jeff Wu, Rewon Child, D. Luan, Dario Amodei, and Ilya Sutskever. 2019. https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/9405cc0d6169988371b2755e573cc28650d14dfe Language models are unsupervised multitask learners

    2019

  22. [30]

    Adithya Renduchintala, Rebecca Knowles, Philipp Koehn, and Jason Eisner. 2016. https://doi.org/10.18653/v1/P16-4023 Creating interactive macaronic interfaces for language learning . In Proceedings of Association for Computational Linguistics System Demonstrations

    2016 doi

  23. [31]

    Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. https://doi.org/10.1162/tacl_a_00434 Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP . Transactions of the Association for Computational Linguistics, 9

    2021 doi

  24. [32]

    Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. 2021. https://aclanthology.org/2021.emnlp-main.779/ PICARD : Parsing incrementally for constrained auto-regressive decoding from language models . In Proceedings of the Conference on Empirical Methods in Natural Language ...

    2021

  25. [33]

    Richard Shin, Christopher Lin, Sam Thomson, Charles Chen Jr, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021. https://aclanthology.org/2021.emnlp-main.608/ Constrained language models yield few-shot semantic parsers ....

    2021

  26. [34]

    Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, and Xuezhe Ma. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.190 Evaluating large language models on controlled generation tasks . In Proceedings of the Conference on Empirical...

    2023 doi

  27. [35]

    Shubham Ugare, Tarun Suresh, Hangoo Kang, Sasa Misailovic, and Gagandeep Singh. 2024. https://arxiv.org/abs/2403.01632 SynCode : LLM generation with grammar augmentation . Preprint, arXiv:2403.01632

    2024 arXiv

  28. [36]

    Valenzuela-Esc \'a rcega, and Mihai Surdeanu

    Robert Vacareanu, George Caique Gouveia Barbosa, Marco A. Valenzuela-Esc \'a rcega, and Mihai Surdeanu. 2020. https://aclanthology.org/2020.lrec-1.643 Parsing as tagging . In Proceedings of the Language Resources and Evaluation Conference

    2020

  29. [37]

    Vos, Thomas C

    Sandra H. Vos, Thomas C. Gunter, Herbert Schriefers, and Angela D. Friederici. 2001. https://doi.org/10.1080/01690960042000085 Syntactic parsing and working memory: The effects of syntactic complexity, reading span, and concurrent load . Language and Cognitive Processes, 16(1)

    2001 doi

  30. [38]

    John Wieting and Kevin Gimpel. 2018. https://doi.org/10.18653/v1/P18-1042 P ara NMT -50 M : Pushing the limits of paraphrastic sentence embeddings with millions of machine translations . In Proceedings of the Annual Meeting of the Association for Computational Linguistics

    2018 doi

  31. [39]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  32. [40]

    Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015. https://doi.org/10.1162/tacl_a_00139 Problems in current text simplification research: New data can help . Transactions of the Association for Computational Linguistics, 3:283--297

    2015 doi

  33. [41]

    Kevin Yang and Dan Klein. 2021. https://doi.org/10.18653/v1/2021.naacl-main.276 FUDGE : Controlled text generation with future discriminators . In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

    2021 doi

  34. [42]

    Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Baker Grosse. 2024. https://proceedings.mlr.press/v235/zhao24c.html Probabilistic inference in language models via twisted sequential M onte C arlo . In Proceedings of the International Conference on Machine Learning

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.