On the (Intuitionistic) Logic of Next-Token Prediction

Paul Tarau (University of North Texas)

REVIEW 4 major objections 4 minor 12 references

On the (Intuitionistic) Logic of Next-Token Prediction

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that next-token prediction is constructive proof extension in intuitionistic implicational logic, and that this reading gives rise naturally to a recurrent, operator-based neural architecture.

desk verdict A clean logic-first framing of next-token prediction whose neural 'derivation' is really a stipulation; worth a read for the logic, but the architecture claims don't hold up. read the letter →

arxiv 2608.08145 v1 pith:EN7EAQVL submitted 2026-08-08 cs.LO

Paul Tarau (University of North Texas) This is my paper · ORCID

classification cs.LO MSC 03B4003B5568N1768T50

keywords logic-basedderivationofneuralarchitecturesintuitionisticimplicationallogictoken-as-operatormodelsleft-nestedimplicationmultiplicativeRNNCurry-Howardcorrespondencenext-tokenprediction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Predicting the next token from a prefix, the operation that powers modern generative language models, can be modeled as constructive proof extension in intuitionistic implicational logic. In this reading, a sequence is a left-nested implication chain, each token acts as an implication, and producing the next word is an application of modus ponens. The paper argues that this proof-theoretic interpretation leads naturally to a recurrent neural architecture, the Arrow language model, in which each token is a low-rank linear operator and the hidden state is a compact proof context. Because matrix multiplication is non-commutative, word order is encoded by the algebra of composition rather than by explicit positional encodings. If the claim is right, the empirical success of next-token prediction gains a logical explanation, and sequence models can be designed from proof-theoretic first principles rather than from attention or convolution.

What carries the argument

The load-bearing objects are the left-nested implication chain $L_n = ((((w_1\rightarrow w_2)\rightarrow w_3)\rightarrow\cdots)\rightarrow w_n)$ and the token operator $M_t = I + U\operatorname{diag}(s_t)V^\top$. The chain is order-sensitive in intuitionistic logic: permuting its atoms changes its set of provable inhabitants, which is what lets the representation carry word order. The operator updates the hidden state as $h_{t+1} = \operatorname{LayerNorm}(h_t + U((V^\top h_t)\odot s_t))$, and the paper reads this update as one modus ponens step. Composing such operators over a prefix realizes the left-nested chain, and non-commutativity of matrix multiplication is what removes the need for explicit positional information.

What would settle it

Train the Arrow model on a corpus where word order determines the next token, and compare it to a variant in which token operators are replaced by commutative token additions; if the proof-theoretic claim is right, the commutative variant should fail on order-sensitive predictions while the operator version succeeds, and the gap should grow with the depth of the left-nested chain.

Watch

Extended reading notes

Core claim

The central discovery is a formal correspondence between next-token prediction and intuitionistic implicational logic. A prefix $w_1,\ldots,w_n$ is represented by the left-nested implication $L_n = ((((w_1\rightarrow w_2)\rightarrow w_3)\rightarrow\cdots)\rightarrow w_n)$, and predicting the next token is the modus ponens step that completes $L_n$ to a proof. Under the Curry-Howard correspondence this makes a sentence prefix a partially applied $\lambda$-term and each prediction a further function application. The paper then derives a neural realization in which the hidden state is a proof context, each token is the operator $M_t = I + U\operatorname{diag}(s_t)V^\top$ with shared low-rank $U,V$ and a token-specific gate $s_t$, and the recurrence $h_{t+1} = \operatorname{LayerNorm}(h_t + U((V^\top h_t)\odot s_t))$ is implication elimination. The resulting Arrow model is equivalent to a multiplicative RNN and is contrasted with transformers, which attach positional encodings to a similarity-based attention mechanism rather than making order intrinsic to the update.

Load-bearing premise

The whole derivation rests on the assumption that a token can be faithfully represented as a low-rank linear operator on a fixed-dimensional hidden state, so that iterating these operators genuinely implements the left-nested implication chain rather than merely approximating it.

Editorial extensions

If this is right

Order information is carried by the non-commutative composition of token operators, so the baseline Arrow model needs no positional encodings.
Multi-token prediction can be framed as iterated implication elimination, giving a provable link between single-step and multi-step completion.
The architecture is parameter-efficient (low-rank operators) and trains with a streaming loss on commodity hardware, making it a practical alternative to attention-based models.
The same left-nested representation supports an exact, interpretable retrieval mechanism, where a contiguous token span is a subformula of a stored sentence.
The model moves the inductive bias of sequence modeling from similarity-based retrieval to transformation: each token permanently modifies the proof context that determines later predictions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: if the proof-theoretic account is right, learning difficulty should track the proof complexity of the left-nested chain; probing with sequences of known logical structure could test this.
Editorial extension: the derivation does not force the low-rank linear form, so the same logic-first reading could support nonlinear or full-rank operator families, and comparing them would separate logical content from linear-algebra convenience.
Editorial extension: the exact-match retrieval limitation suggests soft unification or approximate proof matching as a natural bridge to robust language modeling, which the paper mentions as future work.
Editorial extension: because the Arrow model is equivalent to a multiplicative RNN, known training and expressivity results for that family may transfer to scaled versions of the model.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the framing: next-token prediction as left-nested intuitionistic implication, with order sensitivity explained by the non-commutativity of that nesting. The Prolog LJT prover, the verified examples, and the contrast between left-nested and right-nested chains are all solid and clearly presented. The paper is also honest about its limitations and ships code. Credit where it is due: the logical content checks out as far as it goes, and the observation that right-nested implication collapses to permutation-invariant conjunction is a nice, crisp way to say why order matters.

The soft spot is the bridge to the neural model. The abstract says the Arrow architecture 'arises naturally from a proof-theoretic interpretation,' but the low-rank operator family in Section 6.2 is stipulated, not derived. Nothing in the LJT rules forces a linear operator, and Section 6.4 merely asserts that the hidden state 'corresponds to' the left-nested implication chain, with no theorem or error bound on what is lost when arbitrary proof contexts are crushed into a d-dimensional vector. The modus ponens reading is also content-free as a theory of token choice: p -> (p->q) -> q is provable for every q, so the logic does not constrain which next token is preferred. That preference comes from trained parameters. In the end, the 'derivation' reduces to the observation that non-commutative matrix products are order-sensitive, which is true of any operator recurrence.

The paper itself hedges in Section 6.1, saying the goal is not to propose a novel architecture but to show a proof-theoretic interpretation 'leads naturally' to a known multiplicative RNN. That is a fairer statement, and the related work correctly names [7] as the closest model. The experiments are overfitting sanity checks with no baselines or accuracy metrics, so they cannot validate the neural claims; they only show the thing trains.

Who should read this? People interested in neuro-symbolic connections and logic programming might enjoy the conceptual mapping and might find the Prolog tooling useful to play with. But I would not cite it as a derivation of an architecture, and the empirical section does not support the abstract's strength. A serious referee could help the author reframe the contribution as 'logic-inspired' rather than 'derived' and push for real comparisons against multiplicative RNN and transformer baselines. The logical part is sound and the paper is clearly written, so it merits peer review rather than desk rejection, but as it stands the central architectural claim needs major revision.

Referee Report

4 major / 4 minor

Summary. The paper proposes a proof-theoretic interpretation of next-token prediction in autoregressive language models, modeling prefixes as left-nested intuitionistic implications and next-token prediction as modus ponens. It implements an LJT-based Prolog prover, validates several implicational formulas, and uses these formulas to motivate a recurrent architecture (the Arrow model) in which each token acts as a low-rank linear operator on a hidden state. A subset of the experiments demonstrates that the Prolog retrieval mechanism and a Python implementation of a similar substring-based retrieval work on Gutenberg texts. The central claim is that the Arrow architecture arises naturally, or is 'derived', from the proof calculus.

Significance. If the claimed derivation were established, the paper would provide a novel conceptual bridge between intuitionistic proof theory and recurrent neural architectures, potentially explaining order sensitivity without positional encodings and offering a new design principle for sequence models. The paper has concrete strengths: the LJT prover is machine-checkable and the validity examples in Sections 2-4 are reproducible from the supplied code; the order-sensitivity claims about left-nested implication chains are concrete and falsifiable; and the authors are transparent about the experimental limitations. However, the significance hinges on whether the proof-theoretic interpretation genuinely constrains the architecture, which is precisely the point that the paper does not establish.

major comments (4)

[§6.2-§6.4] The claimed derivation of the Arrow architecture is a stipulation rather than a derivation. In §6.2 the low-rank operator family M_t = I + U diag(s_t) V^T is posited, and in §6.4 it is asserted that the hidden state 'corresponds to' the left-nested implication ((((x1->x2)->x3)...)->xt). No rule from the LJT calculus in §2 forces a linear, low-rank operator form, and no theorem states that a d-dimensional hidden state can faithfully encode proof terms of unbounded depth, nor is any error bound given as a function of d, r, t. The abstract's claim that the architecture 'arises naturally from a proof-theoretic interpretation' is therefore unsupported.
[§3.3 and §6.5] The modus ponens interpretation is content-free as a theory of next-token choice. The formula p -> (p->q) -> q is provable for every q, so the logic alone imposes no preference among candidate next tokens. The preference comes entirely from the trained parameters W_out, U, V, and s_t, whose values are not constrained by the proof calculus. Consequently, the 'logic of next-token prediction' is at most a labeling of the mechanics of the recurrence, not a constraint on the predicted distribution.
[§6.3 and §9] The claimed explanatory power of non-commutativity is not specific to intuitionistic logic. The paper itself acknowledges in §9 that the update is 'mathematically closest' to the multiplicative RNN of [7]. Any recurrent model with token-dependent multiplicative state transitions will be order-sensitive because matrix products are non-commutative. The left-nested implication chain therefore provides no additional explanatory content beyond the already-known fact that operator-valued recurrences encode order; this weakens the paper's central novelty claim.
[§7] The experiments are sanity checks and do not validate the central architectural claim. There are no comparisons against Transformer, standard RNN, or MRNN baselines, no perplexity or accuracy numbers, no ablations of rank r or hidden dimension d, and the deliberate overfitting regime demonstrates only that the model can memorize a small corpus. As such, the empirical results cannot distinguish the Arrow model from any other sufficiently expressive recurrent network, and they do not provide evidence that the proof-theoretic interpretation is the reason the model works.

minor comments (4)

[§4] The sentence 'A formula is called valid if if is an axiom or a theorem' contains a typo: 'if if' should be 'if it'.
[§3.1] The statement 'Any permutation alters the functional type of the antecedent' is too strong as written; a particular permutation of atoms may preserve provability for some instances. The intended claim is that the left-nested chain is not generally invariant under permutation, which the counterexample in Example 1 supports, but the phrasing should be softened.
[§5.1] The predicate names isufpref and iprefsuf are easy to confuse, and the comment in Example 7 does not explain why the two DCG orderings give the same solutions; a brief explanation would improve clarity.
[§6.2] The notation distinguishes M_t as a token-specific operator but the definition uses shared U and V and a token-specific gate s_t; this is clear, yet the text says 'each token induces a linear operator' without stating that the operator is parameterized by the shared low-rank factors, which could mislead readers about parameter sharing.

Circularity Check

2 steps flagged · score 6.0 of 10

The Arrow architecture is asserted, not derived, and the experimental 'predictions' are memorized training fragments.

self definitional [Section 6.2–6.4 (Arrow Model)]
"To make this feasible for large vocabularies, the operator is parameterized in low-rank form: Mt = I + U diag(st)V ⊤ ... After consuming tokens x1, . . . ,xt, the hidden state corresponds to the left-nested implication ((((x1 → x2) → x3) → . . .) → xt) applied to the initial seed h0. Note that each recurrence step directly corresponds to one implication elimination."

The low-rank operator family and the LayerNorm update are introduced as design choices, then Section 6.4 asserts that the resulting hidden state 'corresponds to' the left-nested implication chain. No rule of the LJT calculus in Section 2 forces proof contexts to be d-dimensional vectors or token embeddings to be low-rank matrices, and no theorem bounds the proof structure lost by this compression. The correspondence is therefore true by construction: the recurrence was written to mimic modus ponens, and the 'derivation' labels that choice with proof-theoretic vocabulary. Consequently, the claim that Arrow 'arises naturally' from intuitionistic logic is not derived from the logic; it is a post-hoc interpretation of an independently chosen operator recurrence.
fitted input called prediction [Section 7 (Experiments)]
"we use a deliberately overfitted training regime that turns the training corpus into a “memorized” store of sentences and sentence fragments. This mirrors the Prolog dynamic database of assumptions used by the logic retrieval procedure."

The neural retrieval pipeline scans 'the stored sentences (the same sentence set used for training)' for exact contiguous subsequences and ranks candidate completions with the trained model. Because queries and completions all come from the training set, the continuation scores are memorized values, not out-of-sample predictions. Calling this 'neural next-token prediction via operator composition' reports as a prediction what is, by construction, retrieval from the fitted training store. No held-out evaluation is presented, and the paper's own limitation section concedes that the controlled regime relies on exact contiguous substructure matching.

full rationale

The paper's logic provers and the implicational theorems are independently checked by a Prolog implementation of LJT, so those parts are not circular. The central problem is the claimed derivation of the Arrow architecture from proof theory. Section 6.2 stipulates a low-rank linear operator family, Section 6.3 writes the recurrence to resemble modus ponens, and Section 6.4 simply asserts that the hidden state 'corresponds to' the left-nested implication chain. That last assertion is the load-bearing step, and it is not forced by any theorem of intuitionistic logic; it is a definitional mapping from a chosen neural update to a proof-theoretic reading. The paper itself states that Arrow is 'mathematically closest' to the multiplicative RNN of Sutskever et al. [7], so the architecture was not derived from the logic as a novel consequence. The experimental section amplifies the circularity by measuring only a deliberately overfitted, memorized training regime where queries are exact substrings of the training data, so the reported neural 'predictions' are fitted continuations rather than independent predictions. There is no self-citation chain carrying the argument, and the symbolic logic content has independent value, but the advertised proof-theoretic derivation of the neural model and the experimental validation both reduce to construction-level choices rather than tested consequences.

Assumptions & free parameters 5 free parameters · 6 assumptions · 1 invented entities

The standard logic results are reasonable inputs, but the correspondence between proof contexts and fixed-dimensional hidden states, and the specific low-rank operator family, are chosen by hand. The paper's central neural claims rest on these unproved modeling choices; the learned parameters (h0, U, V, s_t, W_out) and the hyperparameter r are not quantified, so the ledger contains no fitted numerical values.

free parameters (5)

initial hidden state h0 = not reported
Learned vector seeding the recurrence (Section 6.1); no value or initialization given.
low-rank factors U, V = not reported
Shared trainable matrices in the token operator (Section 6.2); dimensions and rank r are hyperparameters not given in the paper.
token gate vectors s_t = not reported
Embedding-based, tanh-bounded gates per token (Section 6.2); learned from data.
output projection W_out = not reported
Linear readout to vocabulary logits (Section 6.5); trained with cross-entropy, no details of vocabulary size or dimensions in the text.
rank hyperparameter r = not reported
Chosen by hand in the low-rank operator M_t = I + U diag(s_t) V^T (Section 6.2); affects expressivity and is not derived from logic.

assumptions (6)

standard math LJT is sound and complete for the implicational fragment of intuitionistic propositional logic (Dyckhoff 1992).
Section 2 relies on this result to justify the Prolog provers.
standard math Propositions correspond to types and proofs to lambda terms under the Curry-Howard isomorphism.
Section 3.4 uses this to translate proof steps into function application and state updates.
domain assumption Next-token prediction is faithfully represented as proof extension by modus ponens.
Introduced in Section 3.3 as the paper's central modeling premise; no empirical evidence establishes that neural next-token distributions behave like intuitionistic proofs.
ad hoc to paper A fixed-dimensional hidden state can represent the full proof context of a left-nested implication chain.
Sections 6.1 and 6.4 assert ht is a compact encoding of the proof context; no bound on information loss is given.
ad hoc to paper The low-rank linear operator family M_t = I + U diag(s_t) V^T is the correct numeric realization of an implicational token operator.
Section 6.2 introduces this form for parameter efficiency; the proof calculus does not imply it.
domain assumption Exact contiguous substring match is an adequate analogue of logical subformula retrieval.
Section 7 uses substring enumeration to mimic isufpref; Section 8 limits the claim to exact matching, excluding noisy and paraphrased queries.

invented entities (1)

Arrow Language Model (token-as-operator recurrent update)
purpose: Neural realization of left-nested implicational proof completion, with tokens as non-commutative operators.
The architecture is introduced to instantiate modus ponens, but its behavior is only sanity-checked on overfit corpora and no independent benchmark establishes that it works as a proof-state machine; the update resembles multiplicative RNN [7].

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the (Intuitionistic) Logic of Next-Token Prediction." pith.science (2026). https://pith.science/paper/EN7EAQVL

@misc{pith2026260808145,
  author       = {Pith},
  title        = {Pith review of: On the (Intuitionistic) Logic of Next-Token Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EN7EAQVL}},
  note         = {Machine review of arXiv:2608.08145}
}

read the original abstract

We model in intuitionistic implicational logic the key enabler of today's GenerativeAI: the next-token prediction in autoregressive causal neural networks. In our framework, next-token prediction corresponds to modus ponens, and sequence processing becomes constructive proof extension under the Curry-Howard correspondence. Our Prolog-based specialized theorem provers validate fundamental properties of the neural models, among which relations between commutative vs. non-commutative sequencing and single-token vs. multi-token prediction choices. We derive a neural architecture equivalent to multiplicative RNNs that arises naturally from a proof-theoretic interpretation of next-token prediction as nested intuitionistic implication and position the model relative to transformers, state-space models and recursive LLMs.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 3 canonical work pages

[7]

In: Proceedings of the 28th International Conference on Machine Learning (ICML) , pp

Ilya Sutskever, James Martens & Geoffrey Hinton (2011): Generating Text with Recurrent Neural Networks. In: Proceedings of the 28th International Conference on Machine Learning (ICML) , pp. 1017–1024. Avail- able at https://icml.cc/2011/papers/524_icmlpaper.pdf

work page 2011
[1]

Journal of Symbolic Logic 57(3), pp

Roy Dyckhoff (1992): Contraction-free sequent calculi for intuitionistic logic . Journal of Symbolic Logic 57(3), pp. 795–807, doi:10.2307/2275431

doi:10.2307/2275431 1992
[2]

arXiv preprint 2312.00752

Albert Gu & Tri Dao (2024): Mamba: Linear-Time Sequence Modeling with Selective State Spaces . arXiv preprint 2312.00752. arXiv:2312.00752

arXiv 2024
[3]

In: Proceedings of the 20th International Conference on Principles of Knowledge Represen- tation and Reasoning, pp

Adam Ishay, Zhun Yang & Joohyung Lee (2023): Leveraging Large Language Models to Generate Answer Set Programs. In: Proceedings of the 20th International Conference on Principles of Knowledge Represen- tation and Reasoning, pp. 374–383, doi:10.24963/kr.2023/37

doi:10.24963/kr.2023/37 2023
[4]

In: Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, Association for Computational Linguistics, pp

Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum & Roger Levy (2023): LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, Association for Computational Linguist...

doi:10.18653/v1/2023.emnlp- 2023
[5]

Smith & Mike Lewis (2022): Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Ofir Press, Noah A. Smith & Mike Lewis (2022): Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 , OpenReview.net. Available at https://openreview.net/ forum?id=R8sQPpGCv0

work page 2022
[6]

Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo & Yunfeng Liu (2024): Ro- Former: Enhanced transformer with Rotary Position Embedding . Neurocomputing 568, p. 127063, doi:10.1016/J.NEUCOM.2023.127063

arXiv 2024
[8]

arXiv preprint, arXiv.2306.14077, doi:10.48550/arXiv.2306.14077

Paul Tarau (2023): Full Automation of Goal-driven LLM Dialog Threads with And-Or Recursors and Refiner Oracles. arXiv preprint, arXiv.2306.14077, doi:10.48550/arXiv.2306.14077

arXiv doi:10.48550/arxiv.2306.14077 2023

Show all 12 references

[9]

In Jeremy Gibbons & Dale Miller, editors: Functional and Logic Programming , Springer Nature Singapore, Singapore, pp

Paul Tarau (2024): System Description: DeepLLM, Casting Dialog Threads into Logic Programs. In Jeremy Gibbons & Dale Miller, editors: Functional and Logic Programming , Springer Nature Singapore, Singapore, pp. 117–134, doi:10.1007/978-981-97-2300-3 7. 66 On the (Intuitionisti...

2024 doi
[10]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser & Illia Polosukhin (2017): Attention is All you Need . In I. Guyon, U. V on Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan & R. Garnett, editors: Advances in Ne...

2017
[11]

In: Findings of the Association for Computational Linguistics: ACL 2023 , Association for Computational Linguistics, pp

Zhun Yang, Adam Ishay & Joohyung Lee (2023): Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text . In: Findings of the Association for Computational Linguistics: ACL 2023 , Association for Computational Linguistics, pp. 5186–5219, d...

2023 doi
[12]

Zhang, Tim Kraska & Omar Khattab (2025): Recursive Language Models

Alex L. Zhang, Tim Kraska & Omar Khattab (2025): Recursive Language Models . arXiv preprint 2512.24601, doi:10.48550/arXiv.2512.24601

2025 doi arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [7]

In: Proceedings of the 28th International Conference on Machine Learning (ICML) , pp

Ilya Sutskever, James Martens & Geoffrey Hinton (2011): Generating Text with Recurrent Neural Networks. In: Proceedings of the 28th International Conference on Machine Learning (ICML) , pp. 1017–1024. Avail- able at https://icml.cc/2011/papers/524_icmlpaper.pdf

work page 2011

[2] [1]

Journal of Symbolic Logic 57(3), pp

Roy Dyckhoff (1992): Contraction-free sequent calculi for intuitionistic logic . Journal of Symbolic Logic 57(3), pp. 795–807, doi:10.2307/2275431

doi:10.2307/2275431 1992

[3] [2]

arXiv preprint 2312.00752

Albert Gu & Tri Dao (2024): Mamba: Linear-Time Sequence Modeling with Selective State Spaces . arXiv preprint 2312.00752. arXiv:2312.00752

arXiv 2024

[4] [3]

In: Proceedings of the 20th International Conference on Principles of Knowledge Represen- tation and Reasoning, pp

Adam Ishay, Zhun Yang & Joohyung Lee (2023): Leveraging Large Language Models to Generate Answer Set Programs. In: Proceedings of the 20th International Conference on Principles of Knowledge Represen- tation and Reasoning, pp. 374–383, doi:10.24963/kr.2023/37

doi:10.24963/kr.2023/37 2023

[5] [4]

In: Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, Association for Computational Linguistics, pp

Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum & Roger Levy (2023): LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, Association for Computational Linguist...

doi:10.18653/v1/2023.emnlp- 2023

[6] [5]

Smith & Mike Lewis (2022): Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation

Ofir Press, Noah A. Smith & Mike Lewis (2022): Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 , OpenReview.net. Available at https://openreview.net/ forum?id=R8sQPpGCv0

work page 2022

[7] [6]

Jianlin Su, Murtadha H. M. Ahmed, Yu Lu, Shengfeng Pan, Wen Bo & Yunfeng Liu (2024): Ro- Former: Enhanced transformer with Rotary Position Embedding . Neurocomputing 568, p. 127063, doi:10.1016/J.NEUCOM.2023.127063

arXiv 2024

[8] [8]

arXiv preprint, arXiv.2306.14077, doi:10.48550/arXiv.2306.14077

Paul Tarau (2023): Full Automation of Goal-driven LLM Dialog Threads with And-Or Recursors and Refiner Oracles. arXiv preprint, arXiv.2306.14077, doi:10.48550/arXiv.2306.14077

arXiv doi:10.48550/arxiv.2306.14077 2023