Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that inserting sparse-attention history tokens during next-token-prediction pretraining gives causal transformers a recurrent-style state vector, fixing their underperformance on future-oriented event-sequence…

desk verdict The paper's core claim that history-token masking drives the gains is undermined by its own p=0 ablation, which shows an appended summary token alone already beats the standard baseline; the missing control is a standard causal transformer with an appended learnable token but no sparse masking. read the letter →

arxiv 2508.01474 v1 pith:SL2DV3FC submitted 2025-08-02 cs.LG

Ivan Karpukhin , Andrey Savchenko This is my paper
classification cs.LG
keywords historytokenseventsequenceclassificationnext-tokenpredictionpretrainingcausaltransformersembeddinginformationbottleneckcontrastivelearningfuture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transformers usually beat recurrent models at modeling event sequences, but on classification tasks that look forward—loan default, customer churn, hospital mortality—the paper says they lag behind recurrent baselines. The claimed cause is that a causal transformer lacks a single compact state vector for the whole sequence, and the proposed fix is a set of learnable history tokens inserted during next-token-prediction pretraining. Each history token is masked to attend to all preceding events, while ordinary events attend only back to the most recent history token, so the history token acts as an information bottleneck and accumulates prefix information like an RNN hidden state. At inference, one history token is appended to the sequence and its embedding is used for classification; across finance, e-commerce, and healthcare benchmarks the method consistently outperforms standard transformers and contrastive pretraining, especially after supervised fine-tuning.

What carries the argument

The central object is the history token together with its custom sparse attention mask. During pretraining, history tokens are inserted at positions sampled with a bias toward the end of the sequence; each history token attends to all earlier event tokens, and each event token attends only to event tokens since the most recent history token plus that history token, so the history token acts as a fixed-capacity information bottleneck. At inference a single history token is appended, and the average of its last-layer activations is treated as the sequence embedding. The next-token-prediction objective with per-field losses—mean absolute error for timestamps and numeric fields, cross-entropy for categorical fields—drives the accumulation, and an application probability $p=0.5$ alternates batches with and without history tokens to keep the model robust to the train/eval placement mismatch.

What would settle it

Train the same decoder-only transformer in three conditions: (a) HT-Transformer with history tokens and masking, (b) the same model with an appended [CLS]-style token but no special attention masking, and (c) the standard next-token-prediction transformer, holding all other hyperparameters fixed. If (b) is statistically indistinguishable from (a) on Churn, MIMIC-III, and Taobao while both beat (c), then the bottleneck masking mechanism is not what carries the reported gains.

Watch

Extended reading notes

Core claim

Identifies the missing recurrent-style state vector as a key limitation of transformers for future-oriented event-sequence classification and introduces history tokens to supply it. A history token is a special input token that attends to all preceding event tokens and is attended to by event tokens only through a narrow local window, so during next-token-prediction pretraining it becomes a bottleneck that accumulates prefix information in the same role as an RNN hidden state. At evaluation, one history token is appended to the end of the sequence and its hidden activation is used as the sequence embedding. The paper reports that this method consistently outperforms standard NTP transformers and contrastive pretraining on real-world benchmarks, and a synthetic Markov experiment shows it is specifically better at local, recent-context classification, whereas global sequence properties remain better captured by contrastive learning or activation averaging.

Load-bearing premise

The load-bearing premise is that the improved classification comes from the history tokens' bottleneck attention masking during pretraining, not merely from appending an extra summary token to the end of the sequence; the paper's own ablation shows that even with history tokens disabled ($p=0$) an appended token still beats the plain transformer, so this premise is not fully isolated.

Editorial extensions

If this is right

  • After supervised fine-tuning, HT-Transformer achieves the top score on four of the five real-world benchmarks, with the only exception being the Alfabattle default-prediction task.
  • The method closes much of the gap between transformer and recurrent baselines in the frozen-embedding setting on three of five datasets, and surpasses all baselines after fine-tuning.
  • History tokens work without auxiliary objectives such as contrastive learning; the next-token-prediction objective alone suffices to train the summary embedding.
  • The design is robust to the insertion frequency $f$—a single history token often matches more frequent insertion—but sensitive to placement and application probability, supporting the paper's recommended defaults of $f=0.1$ and $p=0.5$.
  • On the synthetic Markov dataset, history tokens classify the identity of the final transition matrix more accurately than contrastive pretraining or activation averaging, while those alternatives win on global segment-count classification, indicating a local-versus-global trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The p=0 ablation leaves open that a large part of the gain is simply the use of an appended summary token rather than the history-token mask; an editor would want a control that separates those two factors before attributing the result to bottleneck pretraining.
  • If history tokens are a general substitute for the missing recurrent state, the same scheme could improve other decoder-only embeddings, such as sentence or document embeddings from causal language models, where pooling is currently a weak point.
  • A natural extension the paper does not test is combining the history-token embedding with an average-pooled representation, which might dominate both on tasks that need recent context and on tasks that need global structure.
  • Adaptive placement of history tokens, chosen by the model or by event salience rather than fixed frequency, is a plausible next step given the paper's finding that placement strategy materially affects quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes HT-Transformer, an architecture that inserts special "history tokens" into a causal Transformer trained with a next-token prediction objective on event sequences. During pretraining, each history token attends to all preceding event tokens, while event tokens attend only to recent history tokens and nearby events, creating a sparse information bottleneck. At inference, a single history token is appended to the sequence and its hidden state is used as a sequence-level embedding, either for a frozen-embedding classifier or for end-to-end supervised fine-tuning. The authors report consistent gains over NTP and CoLES baselines across five finance, healthcare, and e-commerce datasets, and use a synthetic Markov task to argue that history tokens are especially suited to future-oriented local classification rather than global sequence classification. The code is publicly released.

Significance. If the central attribution claim were established, the method would provide a simple and practical way to give Transformers a recurrent-like summary state for event sequences, with potential impact on finance, healthcare, and e-commerce applications. The paper has clear strengths: it releases code, reports results across multiple random seeds, includes ablations of the insertion frequency and application probability, and proposes a synthetic task that cleanly separates local from global classification. However, the paper's main mechanistic claim, that sparse history-token masking is responsible for the gains, is not yet isolated from a simpler confound: the use of an appended aggregation token at inference. Because the p=0 ablation already shows that an appended [CLS]-style token without any history-token masking outperforms the standard NTP Transformer, the significance of the method is conditional on an additional control experiment.

major comments (3)
  1. [Ablation Studies, Figure 6] The central claim that sparse history-token masking creates an information bottleneck that drives the downstream gains is not supported because the decisive control is missing. In the p=0 condition, no history tokens are used during pretraining and no sparse mask is applied; the only difference from the standard NTP Transformer baseline is that a randomly initialized [CLS]-style token is appended at the end and its embedding is used. The authors report that p=0 still outperforms the standard NTP Transformer. Without a control consisting of a standard causal NTP Transformer with an appended learnable [CLS] token present during both pretraining and inference, but without any history-token masking, the gains cannot be attributed to the bottleneck mechanism. The gain may instead come from having a dedicated aggregation token. This control is feasible and directly tests the paper's main contribution; it should also be applied to the synthetic task in Table 4.
  2. [Experiments, Tables 2 and 3] The 'state-of-the-art' claim is stronger than the evidence. The comparison set contains only NTP and CoLES baselines on RNN/Transformer backbones; no recent published event-sequence methods or the closely related Recurrent Memory Transformer are included. Moreover, several reported advantages are within one standard deviation or reversed: on AgePred, HT-Transformer + SFT (64.26 ± 0.30) is within noise of NTP Transformer + SFT (64.09 ± 0.31), and on Alfabattle, HT-Transformer + SFT (81.63 ± 0.05) is slightly worse than NTP Transformer + SFT (81.70 ± 0.17). The abstract and conclusion should moderate the 'significantly improves' and 'state-of-the-art' wording unless statistical significance tests and additional baselines are provided.
  3. [Related Work / Experiments] The paper explicitly states in the Related Work that it extends the Recurrent Memory Transformer (RMT) paradigm, but RMT is never used as a baseline in the experiments. Since RMT already uses memory tokens and custom attention masks for sequence compression, it is the closest prior work, and omitting it weakens the novelty and state-of-the-art claims. An RMT baseline should be included in Tables 2 and 3.
minor comments (5)
  1. [Preliminaries on Event Sequences, Pretraining] There is a duplicated phrase in this paragraph: 'the model is trained is trained to predict the next event' should read 'the model is trained to predict the next event.'
  2. [Proposed Method, Figure 3d] The 'Random' attention strategy is underspecified: it is not clear whether the random selection of a preceding history token is performed per attention head, per layer, per token position, or once per forward pass, and whether the random choice is fixed during inference. This should be clarified for reproducibility.
  3. [Ablation Studies, Figure 6] The text says that 'using the embedding of a randomly initialized [CLS] token at the end of the sequence performs better than using the final token's output representation,' but the method section never explains how the [CLS] token is initialized, whether it is updated during supervised fine-tuning, and why a random embedding serves as a better aggregation query. This deserves a brief explanation or a reference to an appendix.
  4. [Ablation Studies, Table 5] The column header 'Uniform pl.' and the row label 'Last sel.' are abbreviations that are not defined in the caption or the text; the 'AVG' column averages ROC AUC and accuracy values across datasets, which are not directly comparable, so this column should be interpreted cautiously or removed.
  5. [Global Classification and Future-Oriented Tasks, Table 4] The column headers 'Local (Last part)' and 'Global (Num. parts)' are not explained in the caption; please define what 'Last part' and 'Num. parts' refer to so the reader can interpret the task setup without referring back to the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: HT-Transformer's claims are empirically supported by a standard pretraining/fine-tuning pipeline, with no load-bearing step that reduces by definition or self-citation to its own inputs.

full rationale

I walked the paper's derivation chain. The history-token mechanism is introduced as an architectural modification to the attention mask: history tokens attend to all preceding events, while event tokens attend only to local context and the most recent history token. The claim that these tokens act as an information bottleneck is a description of the mask, not a conclusion assumed into existence. The pretraining objective is standard next-token prediction with MAE and cross-entropy losses, and downstream classification uses the history-token embedding either as a frozen feature for LightGBM or through supervised fine-tuning; these are ordinary transfer-learning steps, and no equation in the paper reduces the reported gains to a fitted parameter renamed as a prediction. There are no load-bearing self-citations: the Recurrent Memory Transformer and CoLES works are external prior work used as inspiration or baselines, not as authorities that force the present conclusion. No uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation. The one passage that deserves attention is the Figure 6 ablation note that p=0, with no history tokens during pretraining, still outperforms a standard NTP Transformer, attributed partly to using a randomly initialized [CLS]-token embedding at the end of the sequence. That raises a legitimate confounding-variable concern about whether the gains come from the appended token rather than the sparse-attention masking; it is a correctness/experimental-design issue, not circular reasoning, because the paper does not define its success in terms of p=0 or fit the reported metric by construction. I therefore find no circular steps and give a score of 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The central claim relies on the introduced history tokens. The key free parameters (frequency f and application probability p) are tuned and are part of the design. The main axioms are standard modeling assumptions such as the choice of positional encoding and pretraining objective. The history token is an invented architectural element, but it is not a physical entity with independent falsifiable evidence.

free parameters (3)
  • history token insertion frequency f = 0.1 (recommended)
    Controls the number of history tokens as max(1, f*L). Tuned based on experiments; affects performance.
  • application probability p = 0.5 (recommended)
    Proportion of pretraining batches that use history tokens. Tuned based on experiments; p=0 still outperforms baseline.
  • NTP loss weights and model size = Not reported specifically
    Optimized with Bayesian optimization on the NTP RNN baseline and reused for other models. Values are not given.
assumptions (3)
  • domain assumption The time-based positional encoding from Yang et al. (2022) is a valid way to encode irregular event timestamps.
    Used for all event sequences in this paper without comparison to alternative positional encodings.
  • domain assumption The next-token prediction objective with MAE for numerical fields and cross-entropy for categorical fields is a suitable pretraining objective.
    Adopted from prior work; the paper does not test alternative pretraining objectives.
  • domain assumption The benchmark datasets are representative of real-world event sequence classification tasks.
    The paper's conclusions rely on these five datasets covering finance, healthcare, and e-commerce.
invented entities (1)
  • History tokens
    purpose: Learnable tokens that accumulate prefix information during next-token prediction pretraining, acting as a memory bottleneck.
    The method is validated only on the paper's own benchmarks; no external falsifiable prediction is made that could confirm the entity's necessity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens." pith.science (2026). https://pith.science/paper/SL2DV3FC

@misc{pith2026250801474,
  author       = {Pith},
  title        = {Pith review of: HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SL2DV3FC}},
  note         = {Machine review of arXiv:2508.01474}
}
read the original abstract

Deep learning has achieved remarkable success in modeling sequential data, including event sequences, temporal point processes, and irregular time series. Recently, transformers have largely replaced recurrent networks in these tasks. However, transformers often underperform RNNs in classification tasks where the objective is to predict future targets. The reason behind this performance gap remains largely unexplored. In this paper, we identify a key limitation of transformers: the absence of a single state vector that provides a compact and effective representation of the entire sequence. Additionally, we show that contrastive pretraining of embedding vectors fails to capture local context, which is crucial for accurate prediction. To address these challenges, we introduce history tokens, a novel concept that facilitates the accumulation of historical information during next-token prediction pretraining. Our approach significantly improves transformer-based models, achieving impressive results in finance, e-commerce, and healthcare tasks. The code is publicly available on GitHub.

Figures

Figures reproduced from arXiv: 2508.01474 by the authors.

Figure 1
Figure 1. History tokens accumulate prefix information dur [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Event sequences preprocessing. In this work, we propose a novel approach to pretraining Transformer-based embeddings without relying on auxiliary tasks. Our method draws inspiration from recurrent architec￾tures and leverages sparse attention masks to guide the ac￾cumulation of historical information (Bulatov, Kuratov, and Burtsev 2022). Specifically, we introduce History Tokens that gather and summarize contextual … view at source ↗
Figure 3
Figure 3. Comparison of attention masks. Special tokens are orange-colored. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Markovian generative process for the toy dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The dependency of the pretraining quality on the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. When Losses Align: Gradient-Based Composite Loss Weighting for Efficient Pretraining

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    A bilevel method learns composite pretraining loss weights online via gradient alignment with a downstream objective, matching tuned baselines at roughly 30% extra cost over one training run.

  2. A Foundation Model for Multimodal Event Sequences in Financial Applications

    cs.LG 2026-07 conditional novelty 4.5 of 10

    Early-fusion next-event pretraining on multimodal bank event sequences yields reusable user embeddings that, combined with engineered features, improve multi-task financial predictions and production NPV.

Reference graph

Works this paper leans on

34 extracted references · 22 canonical work pages · cited by 2 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Babaev, D.; Ovsov, N.; Kireev, I.; Ivanova, M.; Gusev, G.; Nazarov, I.; and Tuzhilin, A. 2022. Coles: Contrastive learning for event sequences with self-supervision. In Proceedings of the 2022 International Conference on Management of Data, 1190--1199

    work page 2022

  4. [4]

    BehnamGhader, P.; Adlakha, V.; Mosbach, M.; Bahdanau, D.; Chapados, N.; and Reddy, S. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961

    arXiv 2024

  5. [5]

    E.; and Cohan, A

    Beltagy, I.; Peters, M. E.; and Cohan, A. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150

    arXiv 2020

  6. [6]

    Bulatov, A.; Kuratov, Y.; and Burtsev, M. 2022. Recurrent memory transformer. Advances in Neural Information Processing Systems, 35: 11079--11091

    work page 2022

  7. [7]

    Cho, K.; Van Merri \"e nboer, B.; Bahdanau, D.; and Bengio, Y. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

    arXiv 2014

  8. [8]

    Chopra, S.; Hadsell, R.; and LeCun, Y. 2005. Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), volume 1, 539--546. IEEE

    work page 2005

Show all 34 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language tec...

    2019

  2. [10]

    DP, K.; and J, B. 2015. Adam: A method for stochastic optimization. In The Twelfth International Conference on Learning Representations

    2015

  3. [11]

    Gorishniy, Y.; Rubachev, I.; Khrulkov, V.; and Babenko, A. 2021. Revisiting deep learning models for tabular data. Advances in neural information processing systems, 34: 18932--18943

    2021

  4. [12]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  5. [13]

    E.; Pollard, T

    Johnson, A. E.; Pollard, T. J.; Shen, L.; Lehman, L.-w. H.; Feng, M.; Ghassemi, M.; Moody, B.; Szolovits, P.; Anthony Celi, L.; and Mark, R. G. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, 3(1): 1--9

    2016

  6. [14]

    Ke, G.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; and Liu, T.-Y. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30

    2017

  7. [15]

    Lan, Z.; Chen, M.; Goodman, S.; Gimpel, K.; Sharma, P.; and Soricut, R. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations

    2020

  8. [16]

    Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  9. [17]

    McDermott, M.; Nestor, B.; Argaw, P.; and Kohane, I. S. 2023. Event Stream GPT: a data pre-processing and modeling library for generative, pre-trained transformers over continuous-time sequences of complex events. Advances in Neural Information Processing Systems, 36: 24322--24334

    2023

  10. [18]

    Muennighoff, N.; Tazi, N.; Magne, L.; and Reimers, N. 2023. MTEB: Massive Text Embedding Benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 2014--2037

    2023

  11. [19]

    Osin, D.; Udovichenko, I.; Moskvoretskii, V.; Shvetsov, E.; and Burnaev, E. 2024. EBES: Easy Benchmarking for Event Sequences. arXiv preprint arXiv:2410.03399

    2024 arXiv

  12. [20]

    Padhi, I.; Schiff, Y.; Melnyk, I.; Rigotti, M.; Mroueh, Y.; Dognin, P.; Ross, J.; Nair, R.; and Altman, E. 2021. Tabular transformers for modeling multivariate time series. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3...

    2021

  13. [21]

    ???? Improving language understanding by generative pre-training

    Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I.; et al. ???? Improving language understanding by generative pre-training

  14. [22]

    Rizoiu, M.-A.; Lee, Y.; Mishra, S.; and Xie, L. 2017. Hawkes processes for events in social media. In Frontiers of multimedia research, 191--218

    2017

  15. [23]

    Robinson, J.; Sun, L.; Yu, K.; Batmanghelich, K.; Jegelka, S.; and Sra, S. 2021. Can contrastive learning avoid shortcut solutions? Advances in neural information processing systems, 34: 4974--4986

    2021

  16. [24]

    Shchur, O.; Bilo s , M.; and G \"u nnemann, S. 2019. Intensity-free learning of temporal point processes. arXiv preprint arXiv:1909.12127

    2019 arXiv

  17. [25]

    Snoek, J.; Larochelle, H.; and Adams, R. P. 2012. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25

    2012

  18. [26]

    Stankevi c ius, L.; and Luko s evi c ius, M. 2024. Extracting sentence embeddings from pretrained transformer models. Applied Sciences, 14(19): 8887

    2024

  19. [27]

    Synerise . 2025. RecSys Challenge 2025. https://recsys.synerise.com/. Accessed: 2025-07-25

    2025

  20. [28]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  21. [29]

    Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2019. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations

    2019

  22. [30]

    Y.; Wen, Q.; et al

    Xue, S.; Shi, X.; Chu, Z.; Wang, Y.; Hao, H.; Zhou, F.; JIANG, C.; Pan, C.; Zhang, J. Y.; Wen, Q.; et al. 2024. EasyTPP: Towards Open Benchmarking Temporal Point Processes. In The Twelfth International Conference on Learning Representations

    2024

  23. [31]

    Yang, C.; Mei, H.; and Eisner, J. 2022. Transformer Embeddings of Irregularly Spaced Events and Their Participants. In Proceedings of the Tenth International Conference on Learning Representations (ICLR)

    2022

  24. [32]

    C.; Supriya, Y.; Srivastava, G.; Maddikunta, P

    Yenduri, G.; Ramalingam, M.; Selvi, G. C.; Supriya, Y.; Srivastava, G.; Maddikunta, P. K. R.; Raj, G. D.; Jhaveri, R. H.; Prabadevi, B.; Wang, W.; et al. 2024. Gpt (generative pre-trained transformer)—A comprehensive review on enabling technologies, potential applications, eme...

    2024

  25. [33]

    V.; Zhang, B.; Liu, J.; Li, S.; Zhang, L.; and Fan, X

    Zhang, C.; Zhang, Q.; Li, K.; Nuthalapati, S. V.; Zhang, B.; Liu, J.; Li, S.; Zhang, L.; and Fan, X. 2025. GEM: Empowering LLM for both Embedding Generation and Language Understanding. arXiv preprint arXiv:2506.04344

    2025 arXiv

  26. [34]

    Zuo, S.; Jiang, H.; Li, Z.; Zhao, T.; and Zha, H. 2020. Transformer hawkes process. In International conference on machine learning, 11692--11702. PMLR

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.