Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning

Sridhar Mahadevan

REVIEW 3 major objections 6 minor 1 cited by

A Transformer layer is a weighted structured extension over chosen neighborhoods; attention, geometric mixing, and the new KET are special cases, and predict-detach is the boundary that lets noncausal neighborhoods be used safely.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 12:58 UTC pith:MR63CYQZ

load-bearing objection Useful categorical lens, but the headline predict-detach results are almost certainly future-token leakage; the paper's own abstract states the condition the experiments don't enforce. the 3 major comments →

arxiv 2605.27259 v2 pith:MR63CYQZ submitted 2026-05-26 cs.LG

Kan Extension Transformers: A Categorical Unification of Attention, Diffusion, and Predict-Detach Self-Conditioning

Sridhar Mahadevan This is my paper

classification cs.LG

keywords Kan extensionsTransformersattentionpredict-detachself-conditioningsimplicial neighborhoodsdiffusionstructured completion

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes that a Transformer layer is best understood as a weighted structured extension operator: it takes values on a chosen neighborhood system and extends them to target token positions. Standard attention is the singleton-neighborhood case; geometric incidence mixing is a sparse edge-restricted case; and the new Kan Extension Transformer (KET) is the higher-order simplicial case. The paper argues that the decisive enabler for using noncausal neighborhoods in language modeling is not the graph shape but the values transported: detached predicted carriers, not teacher-forced hidden states, turn a noncausal update into a valid self-conditioning mechanism. If this holds, attention, geometric mixing, and diffusion-style completion share one design language, and the dominant empirical lever is the information regime (what values flow) rather than the neighborhood family.

Core claim

The central claim is that a Transformer layer can be viewed as a weighted left-Kan-style extension operator. Concretely, the update h'_t ≈ Σ_σ W(t,σ) V(σ) is a coend-style aggregation over a source category of neighborhoods: tokens for attention, learned geometric adjacency for TopoCoend, and simplices (tokens, edges, faces) for KET. The operator is an actual enriched left Kan extension only when source values are functorial, weights are representable hom-objects (or the specified profunctor action), and aggregation realizes the corresponding coend; otherwise 'Kan-style' denotes an interpretation. A second claim is that predict-detach, computing ê_t = detach(softmax(ℓ_t/T)E) from causal logi

What carries the argument

The central object is the weighted structured extension operator h'_t ≈ ∫^{σ∈N} W(t,σ) ⊗ X(σ), interpreted computationally as Σ_σ w(t,σ) V(σ). Different neighborhood categories N give attention (tokens), TopoCoend (learned geometric adjacency), and KET (simplices). The predict-detach carrier ê_t = detach(softmax(ℓ_t/T)E) is the second load-bearing piece: it is a prefix-derived prediction frozen before transport, and it is what makes noncausal aggregation causally legitimate.

Load-bearing premise

The validity of the predict-detach regime rests on the assumption that every predictive carrier consumed at target position t is measurable from the prefix available at t; the abstract states this requirement explicitly, but the main-text experiments do not demonstrate that the released runs enforce it.

What would settle it

Run the predict-detach configurations with a strict prefix-measurability constraint (e.g., only carriers from positions ≤ t−1 or with an explicit mask) and compare perplexities to the reported values. If the strict run loses most of the reported improvement, the central causal-validity claim is falsified. Alternatively, apply the paper's shuffled-target leakage test: if a predict-detach model still collapses to near-1 perplexity, an illicit gold-future path is present.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If the unification holds, attention, geometric mixing, and simplicial aggregation can be engineered within one design space; the incidence-restricted KET becomes a linear-time sparse approximation to the quadratic KET.
The information-regime distinction means architecture comparisons are only fair within the same regime (strict-causal, self-conditioned, augmented-context), and cross-regime gains should be attributed to the information budget.
The paper's hierarchy places Geometric Transformers as an efficient incidence-restricted special case of KET, so improvements to geometric mixers can transfer to KET and vice versa.
Denoising-style completion (predicting a corrupted future block) is dramatically easier than direct block prediction, supporting a right-Kan/compatibility view of language modeling as structured completion rather than one-shot generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The paper leaves implicit that the 'Kan-style' label is an interpretation for most practical implementations; constructing a true enriched left Kan extension (with functorial source values and representable hom-objects) could yield a different, more principled model, and the difference is testable.
The abstract's prefix-measurability requirement for predict-detach carriers is not verified in the reported experiments; a strict run that enforces it (e.g., masking future carriers) would clarify whether the large predict-detach gains are genuinely leakage-free.
The right-Kan/completion analogy suggests other partial-information objectives (infilling, masked span prediction) should show the same pattern as block denoising, which can be tested in the same harness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful categorical lens, but the headline predict-detach results are almost certainly future-token leakage; the paper's own abstract states the condition the experiments don't enforce.

read the letter

The thing you should know before reading: the predict-detach results, the paper's central empirical offer, look like leakage. The carrier at position t is computed from the teacher-forced hidden state at t, so any noncausal aggregation that consumes carriers from s > t is consuming values that already encode the gold future. The abstract literally says every carrier consumed at target t must be measurable from the prefix available at t, and nothing in the main text shows that was done. GT-PD hits 1.05 on PTB, matching the paper's own gold-noncausal diagnostic (1.05). That is not a self-conditioning result; that is a calibration check that failed.

Give credit where it's due. The categorical framing—attention as singleton-neighborhood aggregation, geometric mixing as incidence-restricted aggregation, KET as simplicial—is a reasonable organizational language, and the paper is honest that the operator is 'Kan-style' unless specific functoriality and representability conditions hold. The information-regime taxonomy (strict-causal, gold-noncausal, predict-detach) is genuinely useful, and the leakage-diagnostic procedure in Section 17.4 is the right idea. The strict-causal comparisons are modest in scale but show quadratic KET competitive on WikiText-2/103.

Now the soft spots, in proportion. The leak is load-bearing, not cosmetic. Section 16.4 distinguishes a same-index predicted hint (Regime E) from a shifted prefix-valid hint (Regime C), which shows the author understands the issue. But the headline GT-PD/KET-PD runs are not reported in those terms, and the near-perfect perplexities strongly suggest they used the same-index form. The paper should be re-run with the carrier at position t-1 or otherwise enforced prefix-measurability, and the leakage test results reported. Second: the Kan-extension claim is asserted rather than proven—Section 9.6 says a weighted sum 'is precisely a discretized enriched Kan/coend computation' without establishing functoriality or representable hom-objects. That fine as a design lens, but it is a reformulation, not a theorem. Third: single-seed runs, no error bars, and the block-denoising results are unsurprising since the model is literally given corrupted future tokens.

Who gets value: people interested in categorical design patterns for transformers, and anyone working on self-conditioning or auxiliary prediction branches. It is also a good cautionary example of how easy it is to leak future information through noncausal neighborhoods.

Recommendation: send it to peer review, but only with a request for major revision. The flaw is identifiable and fixable, and the question—can noncausal neighborhoods be used validly through detached predictions—is worth settling. A serious referee should see this version, but the empirical claims should not stand as-is.

Referee Report

3 major / 6 minor

Summary. The paper proposes a categorical framing in which Transformer layers are weighted structured extension operators: standard attention is the singleton-neighborhood case, TopoCoend is the learned-geometric case, and the proposed Kan Extension Transformer (KET) is the simplicial case. It introduces predict-detach carriers and claims that this lets models use noncausal neighborhoods without leaking gold future tokens. Experiments compare 12 implementations on PTB, WikiText-2, and WikiText-103 across strict-causal, predict-detach, and gold-noncausal regimes. The headline results are that strict-causal quadratic KET is strongest on WT2/WT103 and that predict-detach produces large gains, with GT-PD reaching test perplexities near 1. The abstract itself qualifies the Kan-extension identity as conditional and states a prefix-measurability requirement for carriers, but the main text does not demonstrate that this requirement is satisfied in the experiments.

Significance. If the causal-validity claim held, the predict-detach regime would be a useful design principle, and the categorical framing could serve as a unifying language for attention, geometric mixing, and diffusion-style completion. The paper is honest about its limitations and ships detailed reproducibility artifacts, including logs and checkpoints. However, the central empirical claim is not supported: the reported predict-detach perplexities are in the same range as the paper's own leakage diagnostics, and no evidence shows that the stated prefix-measurability constraint was enforced. The categorical unification is also asserted rather than proved; Eq. (1) is a generic weighted sum, and Section 9.6 calls it 'precisely' a Kan/coend computation without supplying the required functoriality, representability, or coend conditions. As it stands, the contribution is an interpretive taxonomy plus an invalidated experimental claim, so the significance is low.

major comments (3)

[§3.2 Eq. (8); §17.2 Algorithm 2; §16.4] The load-bearing premise that a predict-detach carrier is prefix-valid at an earlier target is violated whenever the carrier is indexed by a future position. Eq. (8) defines ê_t from h_t, a teacher-forced hidden state, so ê_s is a function of x≤s. Algorithm 2's noncausal branch (lines 9–11) adds φ(e_{t+1}) to target t, and more generally any source s>t consumed at t carries gold tokens x_{t+1..s}. The abstract explicitly requires that 'every carrier consumed at target t must also be measurable from the prefix available at t,' but no main-text experiment or code excerpt shows that the reported GT-PD/KET-PD runs enforce this. Section 16.4 labels Regime E as a same-index predicted hint p_t, which does not match the noncausal future-position transport described in Section 3. This invalidates the predict-detach results in Tables 2, 5, 6, and 7; the d=64 intermediate perplexity is exactly the
[§17.4; Tables 2, 5–8] The paper describes a leakage test—shuffling target tokens and checking whether perplexity collapses—but reports no result from it. The reported GT-PD numbers (1.05 on PTB, 1.59 on WT2, 12.84 on WT103 in Table 2) are essentially indistinguishable from the explicit gold-noncausal (Regime A) diagnostics (1.05 on PTB and 1.05 on WT2 in Tables 5–6). Table 8 makes the problem explicit: transition_gain is ≈1.000 at d=256, meaning the 'self-conditioned' run reaches the augmented-context performance level. Without the shuffling diagnostic, the claim that predict-detach 'preserves causal validity' is unsupported and the near-perfect perplexities are what leakage would look like.
[§9.6 and §2.1 Eq. (1)] The central theoretical claim—that the update in Eq. (1) 'is precisely a discretized enriched Kan/coend computation'—is asserted, not proved. Eq. (1) is a generic weighted sum. To identify it with an enriched left Kan extension one must specify the source category, the functorial assignment of values, representable hom-objects (or a profunctor action), and verify the coend universal property. None of these are given for attention, TopoCoend, or KET. The abstract itself hedges that the operator is 'Kan-style' when these conditions fail, which is appropriate, but the main text's stronger wording overclaims. Since the title and first claim rest on this categorical identity, the paper should either provide the functoriality/representability proof or explicitly reposition the contribution as a design language rather than a theorem.

minor comments (6)

[§16.5 vs. §4.3] Section 16.5 establishes a reporting standard with regime labels (C/E/A), but Tables 2 and 3 in the main text do not label regimes. Readers cannot tell from the tables which runs are strict-causal, predict-detach, or leakage diagnostics.
[§17.5] The reproducibility section states that the LM figures are single-seed runs with no multi-seed error bars. Tables 1–3 should carry an explicit caveat, since the differences between some strict-causal models are small.
[§17.1–17.2] Algorithms 1 and 2 use v_t and h_t without defining their relationship. If v_t is the value base derived from h_t, this should be stated; otherwise the incidence-restricted block's input is underspecified.
[§1 and §4] The paper claims to evaluate 12 Transformer implementations, but the main text does not explicitly enumerate them. A table listing the 12 models with their regime labels would make the comparison easier to verify.
[§3.2 Eq. (8)] The temperature T is used in Eq. (8) but not defined until Section 17.5. Define it when first introduced.
[§16.2] The notation ẋ_{t+1} appears in the right-Kan completion formula but is not defined. This makes the formal regime distinction in that paragraph hard to follow.

Circularity Check

2 steps flagged

The categorical unification is a definitional relabeling of Eq. (1) as a coend, and predict-detach's causal validity assumes the very target-prefix measurability that Algorithm 2's noncausal branch violates.

specific steps

renaming known result [Abstract and §9.6, with Eq. (1) in §2.1 and Eq. (13) in §9.5]
"This operator is an actual enriched left Kan extension only when the source values are functorial, the weights are representable hom-objects (or the specified profunctor action), and aggregation realizes the corresponding coend; otherwise ``Kan-style'' denotes an interpretation rather than an identity theorem. ... Under this interpretation, a familiar update h′_d = ∑_c w_{d,c} V_c is precisely a discretized enriched Kan/coend computation."

The paper's shared update rule, Eq. (1), is an arbitrary weighted sum: h′_t ≈ ∫^{σ∈N} W(t,σ)⊗X(σ) ≈ ∑_σ w(t,σ)V(σ). The enriched left-Kan formula, Eq. (13), is the same weighted-sum form with W replaced by the hom-object D(Kc,d) and X by F(c). Thus identifying attention, TopoCoend, and KET with Kan extensions is true by stipulation: any weighted sum can be written as a discretized coend by choosing an indexing category and setting W to the hom-object. The abstract itself concedes that the operator is an actual Kan extension only under functoriality, representability, and coend conditions that are never established for the architectures. So the headline 'unification' is a renaming of Eq. (1) as Eq. (13), not a derived equivalence.
self definitional [§3.2 Eq. (8), §15.5, §17.2 Algorithm 2 lines 9–11; compare abstract]
"ê_t = detach(softmax(ℓ_t/T)E), ℓ_t = W_o h_t ... if regime is noncausal and t+1 ≤ S−1 then m_t ← m_t + φ(e_{t+1}) ... every carrier consumed at target t must also be measurable from the prefix available at t."

The validity proof in §15.5 argues that each carrier is prefix-valid because 'h_t is causal, so ê_t is also a function only of the prefix x≤t.' That is true at the carrier's own index t, but the noncausal branch of Algorithm 2 consumes e_{t+1} when updating target t. Under teacher forcing, h_{t+1}, and hence ℓ_{t+1} and ê_{t+1}, depends on the gold token x_{t+1}, so it is not measurable from the prefix available at t. The claimed causal boundary is therefore exactly the unproven target-prefix measurability condition stated in the abstract: the conclusion that predict-detach permits noncausal neighborhoods without leakage is assumed, not derived. The near-1 perplexities of GT-PD (e.g., 1.05 on PTB in Table 6) mirror the gold-noncausal leakage diagnostics, which is the expected signature if

full rationale

I did not count the self-citations to Mahadevan 2025a–c and Mahadevan 2026 as circular: although the paper leans on a self-published book and systems, the arguments here do not actually reduce to those citations, and the strict-causal benchmark comparisons in Tables 1 and 3 are ordinary empirical runs that are not circular. The score of 6 comes from the two load-bearing conceptual claims. First, the 'categorical unification' of attention, TopoCoend, and KET is Eq. (1), a generic weighted sum, and §9.6 asserts it is 'precisely' the enriched Kan/coend formula Eq. (13) merely by relabeling the weight and value terms; since any weighted sum can be cast in coend form with a discrete indexing category, the unification is a definitional convention rather than a theorem. The abstract's own hedge confirms that 'Kan-style' is an interpretive label unless strong categorical conditions hold, and those conditions are not verified. Second, the predict-detach validity claim assumes target-prefix measurability rather than establishing it: carriers are causally generated at their own index, but noncausal aggregation at target t consumes carriers from t+1 that depend on gold x_{t+1} under teacher forcing. The reported near-perfect GT-PD perplexities are consistent with that assumption being violated. These are central claims, so the paper is partially circular by construction, but independent benchmark content and the absence of a self-citation-forced uniqueness argument keep the score below 8.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 3 invented entities

The central conceptual claim rests on a standard coend formula, an unverified functoriality/representability assumption, and a prefix-validity assumption on carriers that the experiments appear to violate. The paper introduces no new physical or mathematical entities; the simplex carriers and detach modality are architectural devices without independent falsifiable handles.

free parameters (3)

Carrier temperature T = 1.0
Appears in Eq. (8) and Eq. (18) to control carrier sharpness; chosen by hand, not optimized.
TopoCoend neighborhood size and dimension = topo_k=16, topo_dim=16
Set in Section 17.5 for the TopoCoend geometric neighborhood; chosen by hand.
Denoising corruption schedule = p(s)=0.05+0.45*(s-1)/7 for s=1..8
Defined in Section 5.1; ad hoc schedule from 5% to 50% corruption, not derived.

axioms (4)

standard math Enriched coend formula Eq. (13) and definition of Kan extensions
Standard category theory (Mac Lane, Riehl, Richter) is used to write Eq. (1) as a coend-like sum. The mathematics itself is standard; its applicability to neural attention weights is not established.
ad hoc to paper Attention/KET weights are representable hom-objects and source values are functorial
Required for Eq. (1) to be an actual enriched left Kan extension. The paper only asserts this interpretation and acknowledges it is otherwise 'Kan-style' (Abstract, Section 9.6).
domain assumption Predictive carrier ê_t is prefix-valid
Used to claim noncausal aggregation of carriers is causally valid. Violated if future-position carriers are computed from teacher-forced hidden states at those positions (Eq. (8), Section 3.2).
ad hoc to paper A corrupted future block behaves like a horn in a Kan complex
Section 13-14 use horn-filling as an analogy for denoising; the paper explicitly disclaims that a language model literally forms a Kan complex.

invented entities (3)

Simplex source objects σ (edges, faces, motifs) no independent evidence
purpose: Aggregation sources for the KET update Eq. (3), going beyond tokens
Architectural construction; no falsifiable prediction outside the proposed architecture.
Detached predictive carrier ê_t no independent evidence
purpose: Transport predicted content instead of teacher-forced hidden states through noncausal neighborhoods
Operational trick whose validity is exactly the claim under test; no independent evidence is provided.
Detach modality □ no independent evidence
purpose: Formalize gradient killing as a modal operator on paired forward/backward semantics
A re-description of detach() in autodiff; no independent empirical handle.

pith-pipeline@v1.3.0-alltime-deepseek · 19143 in / 14181 out tokens · 128816 ms · 2026-08-02T12:58:34.838780+00:00 · methodology

0 comments

read the original abstract

We propose Kan Extension Transformers (KETs) as a categorical design language for a diverse group of Transformer implementations. A layer can be viewed generally as a weighted structured extension operator: attention uses token neighborhoods, geometric mixing uses sparse incidences, and KET uses simplicial sources. This operator is an actual enriched left Kan extension only when the source values are functorial, the weights are representable hom-objects (or the specified profunctor action), and aggregation realizes the corresponding coend; otherwise ``Kan-style'' denotes an interpretation rather than an identity theorem. Predict-detach blocks gradients through a predictive carrier and avoids transporting teacher-forced hidden states, but detach alone does not make a noncausal update strictly autoregressive: every carrier consumed at target $t$ must also be measurable from the prefix available at $t$. We evaluate 12 implementations on Penn Treebank, WikiText-2, and WikiText-103 across strict-causal and self-conditioned regimes, using widths $d=64,256$ and depths $L=2,8,16$ across the reported studies. Quadratic KET is strongest among the compared strict-causal architectures on WikiText-2 and WikiText-103; the largest cross-regime gains arise from additional self-conditioning information, not neighborhood design alone.

Figures

Figures reproduced from arXiv: 2605.27259 by Sridhar Mahadevan.

**Figure 2.** Figure 2: Validation perplexity across datasets. The headline tables quantify Transformer, GT, and [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Figure 3: Three information regimes for language modeling. Direct block prediction maps context [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: A simplicial view of denoising language modeling. A corrupted block is a partial simplex, [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

**Figure 5.** Figure 5: Geometric Transformer architectural variants studied in this paper. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_5.png] view at source ↗

**Figure 6.** Figure 6: Information-regime diagrams for GT/Transformer updates. Left: strict causal prefix functor. Middle: admissible auxiliary channel (Definition 3) factors through a prefix-derived object Gt, so the update remains in Regime C. Right: future-informative auxiliary channel (Definition 4) keeps the same type αt : At→Ht, but At now depends on future-indexed objects, placing the model in Regime A. Model family: GT /… view at source ↗

**Figure 7.** Figure 7: Evaluation taxonomy by information budget. Architecture comparisons are fair only within a regime; Regime E is reported separately from both strict-causal controls and explicit augmented-context leakage baselines. Definition 4 (Future-Informative Augmentation) An auxiliary map αt is future-informative iff no such factorization exists through prefix-only objects, equivalently the induced update depends on o… view at source ↗

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Learning in Infinitesimal Non-Compositional Sketches
cs.LG 2026-07 conditional novelty 5.0

The paper defines infinitesimal non-compositionality as the tangent-lift of factorization failures in learning sketches, and proposes learning as converging to a final coalgebra of iterated tangent lifts.

Reference graph

Works this paper leans on

22 extracted references · cited by 1 Pith paper

[1]

M., Veli c kovi \'c , P., and Li \`o , P

Barbero, F., Bodnar, C., S \'a ez de Oc \'a riz Borde, H., Bronstein, M. M., Veli c kovi \'c , P., and Li \`o , P. (2022). Sheaf neural networks with connection laplacians. In Topological, Algebraic and Geometric Learning Workshops\/ , volume 196 of Proceedings of Machine Learning Research\/ , pages 28--36

2022
[2]

Battiloro, C., Testa, L., Giusti, L., Sardellitti, S., Di Lorenzo, P., and Barbarossa, S. (2024). Generalized simplicial attention neural networks. IEEE Transactions on Signal and Information Processing over Networks\/ , 10 , 1--16

2024
[3]

P., Li \`o , P., and Bronstein, M

Bodnar, C., Di Giovanni, F., Chamberlain, B. P., Li \`o , P., and Bronstein, M. M. (2022). Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNN s. In Advances in Neural Information Processing Systems\/ , volume 35, pages 18527--18541

2022
[4]

Goh, C. W. J., Bodnar, C., and Li \`o , P. (2022). Simplicial attention networks

2022
[5]

Gold, E. M. (1967). Language identification in the limit. Information and Control\/ , 10 (5), 447--474

1967
[6]

N., Birdal, T., Dey, T

Hajij, M., Zamzmi, G., Papamarkou, T., Miolane, N., Guzm \'a n-S \'a enz, A., Ramamurthy, K. N., Birdal, T., Dey, T. K., Mukherjee, S., Samaga, S. N., Livesay, N., Walters, R., Rosen, P., and Schaub, M. T. (2022). Topological deep learning: Going beyond graph data

2022
[7]

L., Dawood, S., Cherukuri, B., Kocheemoolayil, J

Hajij, M., Bastian, L., Osentoski, S., Kabaria, H., Davenport, J. L., Dawood, S., Cherukuri, B., Kocheemoolayil, J. G., Shahmansouri, N., Lew, A., Papamarkou, T., and Birdal, T. (2025). Copresheaf topological neural networks: A generalized deep learning framework. arXiv\/

2025
[8]

Ho, J., Jain, A., and Abbeel, P. (2020). Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems\/

2020
[9]

L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T

Li, X. L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T. B. (2022). Diffusion-lm improves controllable text generation. In Advances in Neural Information Processing Systems\/ , volume 35, pages 4328--4343

2022
[10]

Mac Lane, S. (1971). Categories for the Working Mathematician\/ . Springer-Verlag, New York. Graduate Texts in Mathematics, Vol. 5

1971
[11]

Mahadevan, S. (2025a). Categories for AGI . Book manuscript
[12]

Mahadevan, S. (2025b). CLIFF\_CatAgi : Categories for AGI local research interface. GitHub repository
[13]

Mahadevan, S. (2025c). Large causal models from large language models
[14]

Mahadevan, S. (2026). Prometheus: Automating deep causal research integrating text, data and models

2026
[15]

May, J. (1992). Simplicial Objects in Algebraic Topology\/ . University of Chicago Press

1992
[16]

and Xie, S

Peebles, W. and Xie, S. (2023). Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision\/ , pages 4195--4205

2023
[17]

Richter, B. (2020). From Categories to Homotopy Theory\/ . Cambridge Studies in Advanced Mathematics. Cambridge University Press

2020
[18]

Riehl, E. (2017). Category Theory in Context\/ . Aurora: Dover Modern Math Originals. Dover Publications

2017
[19]

N., Kaiser, L., and Polosukhin, I

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Proce...

2017
[20]

Zhang, H., Liu, X., and Zhang, J. (2022). HEGEL : Hypergraph transformer for long document summarization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing\/ , pages 10167--10176

2022
[21]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sente...
[22]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "In " FUNCTION format.date ye...

[1] [1]

M., Veli c kovi \'c , P., and Li \`o , P

Barbero, F., Bodnar, C., S \'a ez de Oc \'a riz Borde, H., Bronstein, M. M., Veli c kovi \'c , P., and Li \`o , P. (2022). Sheaf neural networks with connection laplacians. In Topological, Algebraic and Geometric Learning Workshops\/ , volume 196 of Proceedings of Machine Learning Research\/ , pages 28--36

2022

[2] [2]

Battiloro, C., Testa, L., Giusti, L., Sardellitti, S., Di Lorenzo, P., and Barbarossa, S. (2024). Generalized simplicial attention neural networks. IEEE Transactions on Signal and Information Processing over Networks\/ , 10 , 1--16

2024

[3] [3]

P., Li \`o , P., and Bronstein, M

Bodnar, C., Di Giovanni, F., Chamberlain, B. P., Li \`o , P., and Bronstein, M. M. (2022). Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNN s. In Advances in Neural Information Processing Systems\/ , volume 35, pages 18527--18541

2022

[4] [4]

Goh, C. W. J., Bodnar, C., and Li \`o , P. (2022). Simplicial attention networks

2022

[5] [5]

Gold, E. M. (1967). Language identification in the limit. Information and Control\/ , 10 (5), 447--474

1967

[6] [6]

N., Birdal, T., Dey, T

Hajij, M., Zamzmi, G., Papamarkou, T., Miolane, N., Guzm \'a n-S \'a enz, A., Ramamurthy, K. N., Birdal, T., Dey, T. K., Mukherjee, S., Samaga, S. N., Livesay, N., Walters, R., Rosen, P., and Schaub, M. T. (2022). Topological deep learning: Going beyond graph data

2022

[7] [7]

L., Dawood, S., Cherukuri, B., Kocheemoolayil, J

Hajij, M., Bastian, L., Osentoski, S., Kabaria, H., Davenport, J. L., Dawood, S., Cherukuri, B., Kocheemoolayil, J. G., Shahmansouri, N., Lew, A., Papamarkou, T., and Birdal, T. (2025). Copresheaf topological neural networks: A generalized deep learning framework. arXiv\/

2025

[8] [8]

Ho, J., Jain, A., and Abbeel, P. (2020). Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems\/

2020

[9] [9]

L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T

Li, X. L., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T. B. (2022). Diffusion-lm improves controllable text generation. In Advances in Neural Information Processing Systems\/ , volume 35, pages 4328--4343

2022

[10] [10]

Mac Lane, S. (1971). Categories for the Working Mathematician\/ . Springer-Verlag, New York. Graduate Texts in Mathematics, Vol. 5

1971

[11] [11]

Mahadevan, S. (2025a). Categories for AGI . Book manuscript

[12] [12]

Mahadevan, S. (2025b). CLIFF\_CatAgi : Categories for AGI local research interface. GitHub repository

[13] [13]

Mahadevan, S. (2025c). Large causal models from large language models

[14] [14]

Mahadevan, S. (2026). Prometheus: Automating deep causal research integrating text, data and models

2026

[15] [15]

May, J. (1992). Simplicial Objects in Algebraic Topology\/ . University of Chicago Press

1992

[16] [16]

and Xie, S

Peebles, W. and Xie, S. (2023). Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision\/ , pages 4195--4205

2023

[17] [17]

Richter, B. (2020). From Categories to Homotopy Theory\/ . Cambridge Studies in Advanced Mathematics. Cambridge University Press

2020

[18] [18]

Riehl, E. (2017). Category Theory in Context\/ . Aurora: Dover Modern Math Originals. Dover Publications

2017

[19] [19]

N., Kaiser, L., and Polosukhin, I

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Proce...

2017

[20] [20]

Zhang, H., Liu, X., and Zhang, J. (2022). HEGEL : Hypergraph transformer for long document summarization. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing\/ , pages 10167--10176

2022

[21] [21]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sente...

[22] [22]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in "In " FUNCTION format.date ye...