REVIEW 4 major objections 6 minor 47 references
Looking around you: external information enhances representations for event sequences
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read External context vectors built by aggregating other users' embeddings at the current time improve event-sequence predictions, and a learnable kernel-attention aggregation does so best.
desk verdict The aggregation idea is sensible and the evaluation is systematic, but the external context set appears to include the target user's own embedding, so the headline gains may be self-augmentation rather than cross-user information. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the external context vector $g_t = A(H, h_t)$, built from the matrix $H$ of the most recent encoder embeddings of all users before time $t$ and an aggregation function $A$. The method's novelty is making $A$ a learnable attention head rather than a fixed summary: kernel attention computes similarity weights through a learned nonlinear map $\varphi$ before the dot product, producing a dense, user-specific affinity pattern instead of the near-diagonal self-similarity produced by plain attention under a contrastive objective, and instead of the same vector for everyone produced by mean or max pooling. The learnable parts are trained with the same contrastive self-supervised loss used for the backbone encoder, so no labels are needed for the context module.
What would settle it
Run the same six-dataset pipeline with $H$ built exclusively from training-set users, with the target user's own embedding removed, and with test labels never visible to the aggregator or the downstream head; if the ROC-AUC gains over 'without context' disappear or shrink to noise, the improvement came from leakage or self-similarity rather than from external context.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that sequence embeddings should not be read in isolation: at any time $t$, there is a set $H = \{h_{i,j_i}\}_{i=1}^n$ containing the most recent internal representations of the dataset's users, and the aggregated vector $g_t = A(H, h_t)$ captures shared, time-local information that the individual sequence lacks. Concatenating $g_t$ with $h_t$ to form the representation used for downstream tasks yields consistent predictive gains. The paper compares eight aggregators, from mean and max pooling to attention variants to Hawkes-inspired temporal weightings, and finds that kernel attention—$g_t = H\,\mathrm{softmax}(\langle \varphi(H), \varphi(h_t)\rangle)$ with $\varphi$ a learnable two-layer network—performs best or second best on all but one dataset and task combination. The gains appear with both RNN and transformer encoders, for global classification and local next-event-type prediction, and when the backbone is frozen or fine-tuned, although the size of the effect varies across datasets.
Load-bearing premise
The reported gains rest on the assumption that the context pool $H$ contains only other users' embeddings and that no label information from the downstream task reaches the context through the unsupervised embeddings; the paper never states how train and test splits interact with context construction.
Editorial extensions
If this is right
- Global target prediction (e.g., churn, default, fraud, gender) and local next-event-type prediction both improve when the external context vector is concatenated to the user embedding, across financial, e-commerce, education, and movie-rating datasets.
- Kernel attention is the strongest aggregator in the comparison; when it is not the best it is second best, and it leaves the largest margin on the class-balanced churn task.
- The benefit is not tied to one encoder: the same qualitative pattern appears for RNN and transformer backbones, and for frozen and fine-tuned encoders.
- Mean pooling provides a smaller but still consistent gain with no trainable parameters, so an external context can be added cheaply to an existing pipeline.
- Larger external context sets help up to a saturation point—roughly 500 sequences on the churn dataset—so the method scales to realistic user populations with bounded context.
Reading between the lines
- A testable extension the paper does not run: restrict $H$ to users from the training split only and exclude the target user's own embedding; if the gains survive this restriction, they are attributable to genuine cross-user context rather than to label or self-information leakage.
- The mean-pooling baseline is a natural implicit macro covariate: if its improvement traces to common time effects (e.g., market-wide shifts), then comparing it against explicit macroeconomic indicators would show whether learned aggregation is actually needed or whether a simple time-indexed summary suffices.
- Kernel attention's dense user-affinity structure could be most valuable for sparse users with few recent events, whose own embeddings are weakest; the paper does not stratify results by sequence length, so testing that subgroup would sharpen the practical case.
- The production recipe of updating a vector database at fixed intervals means the context is slightly stale between updates; measuring sensitivity to update frequency would tell whether real-time aggregation matters.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes augmenting a user's event-sequence representation with an 'external context' vector obtained by aggregating representations of other users at nearby points in time. It compares mean/max pooling, several attention variants, and a learnable kernel attention method, built on CoLES-pretrained RNN and Transformer encoders, over six datasets with global and local downstream tasks. The central empirical claim is that adding this external context improves ROC-AUC scores and that the proposed Kernel attention is the best aggregator, with mean pooling also giving a significant but smaller improvement.
Significance. If validated, the idea is practically appealing: it requires no additional data, operates on top of an existing encoder, supports fine-tuning, and the authors document linear inference overhead. The paper also releases code and includes more baselines than is typical, plus SHAP and computational-cost analyses. However, the current evaluation has a load-bearing confound: the context set H appears to include the target user's own embedding, so the reported gains may reflect self-augmentation rather than external information. Because the central claim depends on removing this confound, the significance is currently conditional on a corrected evaluation.
major comments (4)
- [Section III-C, step 2; Section IV-E] The set H is defined as H={h_{i,j_i}}_{i=1}^n over all n users, and although the text says 'other users,' no exclusion of the target user is made. At time t the target user's last embedding h_t is exactly one of the columns of H. Section IV-E confirms this: the interpretation analysis computes softmax(H^T H) and states that 'each user is most similar to themselves.' Consequently, g_t = A(H, h_t) contains a copy of h_t for every aggregation method, and for attention-based methods h_t receives the highest weight. The improvements in Table I (e.g., Churn global from 0.7432 to 0.7728 for Mean and 0.7745 for Kernel attention) may therefore be self-augmentation rather than cross-user information. The definition and implementation must exclude the target user before the central claim can be assessed.
- [Section III-F and Section IV-A] The paper does not describe the train/test split protocol used when constructing the external context. For global validation, h_i summarizes the whole sequence and is used to train the LightGBM classifier; if the same h_i appears in H for a test user, the context vector contains information from the test user's own sequence. It is also unclear whether H for a test user includes only training/validation users or also test users, and how the context vectors for training and test are separated. A precise statement of which users are included in H for train and test contexts is required before the reported improvements can be interpreted as clean gains from external information.
- [Table I, Assessment global row] All aggregation methods jump from 0.8711 (Without context) to 0.9958-0.9962 on the Assessment global task. This near-perfect saturation across all methods, including simple Mean and Max pooling, is not explained. Appendix D defines the Assessment global target as one of four grades while each gameplay session is treated as a separate sequence; if a user has multiple sessions, H may contain other sessions of the same user and therefore directly correlated labels. This row should be analyzed, and either fixed or removed; as presented, it undermines the claim of consistent cross-domain improvements.
- [Tables I and II; Appendix F] There is no random-context control in the main results. Appendix F reports SHAP values for a 'Rand' external vector, but no ROC-AUC is given for this control. A control in which H is replaced by random embeddings, or by embeddings from a different time window, is needed to show that the improvement over 'Without context' is due to genuine external information and not merely to the concatenation of an extra feature vector. Adding such a control is necessary to support the paper's central claim.
minor comments (6)
- [Abstract] The abstract at the top of the manuscript says 'nine diverse event sequence datasets', while the paper's internal abstract and Section IV-A list six datasets (Churn, Default, HSBC, Assessment, Taobao, MovieLens-1M). Please reconcile this discrepancy.
- [Section III-C] The notation for j_i uses 'arg max_i' but the maximization should be over the event index j. Also, H is said to have rows of size m although it is introduced as an m x n matrix; please clarify the orientation of H and the indices.
- [Section III-A] There is a typo: 'MMC code' should be 'MCC code'.
- [Section III-D, Eq. (3)] The notation <phi(H), phi(h_t)> is undefined for a matrix H. Please specify whether the inner product is applied row-wise or via a different contraction.
- [Appendix C vs. Section III-C] Section III-C defines H over all n users from the dataset, but Appendix C states that only a random subset of 1000 clients (Churn) or 300 clients (other datasets) is used in training. Please clarify how n is chosen and whether the reported results use the full set or the subset.
- [Tables I and II] The paper reports mean ± std over three runs but does not provide significance tests. Several comparisons in Table I are within one standard deviation (e.g., Churn local Kernel attention 0.7102±0.003 vs. Mean 0.7102±0.003), so the claim that Kernel attention 'performs best or second best for all but one task' is stronger than the evidence supports.
Circularity Check
The 'external context' H is defined over all users without excluding the target, so the claimed external-information gains partly reduce to self-augmentation by construction.
-
self definitional
[Section III-C, step 2 (definition of H) and Section IV-E (self-similarity analysis)]
"Select internal representations from other users that precede the current time point but are close to it in terms of temporal proximity: H={h_{i,j_i}}_{i=1}^n, j_i = arg max_i t_{ij} s.t. t_{ij} ≤ t, where H∈R^{m×n} is a matrix, which rows are embeddings of size m for all n users from dataset at a given time point. ... our goal is to construct an external context vector for this user. ... In the Attention case, the outcome is expected: each user is most similar to themselves, as dictated by the CoLES learning objective."
The formula indexes all n users with no exclusion of the target user, despite the surrounding text promising 'other users'. For the target user, h_t is the last available internal representation, so h_t is itself a row of H under the stated definition. Every aggregation is A(H,h_t)—e.g., g_t = H softmax(H^T h_t) for Attention—so the 'external context' vector is a function of the very representation it is supposed to augment, and the self-row receives high weight because CoLES embeddings are maximally self-similar. ROC-AUC improvements over 'Without context' therefore do not isolate information from other users; part of the gain can be produced by duplicating the input embedding through the aggregation.
full rationale
The paper is an empirical comparison of aggregation heads on top of CoLES embeddings, and most of the evaluation is externally anchored on six benchmark datasets with LightGBM/linear-head downstream tasks. The learnable aggregation methods are trained with a contrastive loss rather than on the downstream labels, so there is no fitted-parameter-renamed-as-prediction circularity. Self-citations to CoLES [5] and the authors' earlier external-context study [6] are used as motivation and baseline, but the central comparison is implemented independently in this paper, so the citations are not load-bearing in a circular sense. The significant definitional flaw is in Section III-C step 2: H is written as all n users at time t with no i≠target exclusion, so the 'external' context includes the target user's own embedding. Because the downstream gains are attributed to external information, and because a random-context ROC-AUC control is absent from Tables I and II, the external-information claim partially reduces to self-augmentation by construction. The relative ranking of aggregation methods (e.g., Kernel attention vs. Mean) remains internally meaningful, but the headline claim that information from other users drives the improvement is not cleanly established.
Assumptions & free parameters
free parameters (4)
- External context cohort size =
1000 (Churn), 300 (other datasets)
- Learnable attention matrix A =
trained m x m matrix
- Kernel attention MLP =
two-layer FC, hidden size 100
- Local validation window size =
32
assumptions (4)
- domain assumption All users' last internal representations before time t are available when computing context at inference.
- domain assumption The last embedding before time t is a sufficient summary of each user's state for aggregation.
- domain assumption Dot-product similarity between CoLES embeddings is a meaningful measure of user relatedness.
- ad hoc to paper The external context set H excludes the target user and respects train/test separation.
Cite this review
Pith. "Pith review of Looking around you: external information enhances representations for event sequences." pith.science (2026). https://pith.science/paper/AOZXBFDI
@misc{pith2026250210205,
author = {Pith},
title = {Pith review of: Looking around you: external information enhances representations for event sequences},
year = {2026},
howpublished = {\url{https://pith.science/paper/AOZXBFDI}},
note = {Machine review of arXiv:2502.10205}
}
read the original abstract
Representation learning produces models in different domains, such as store purchases, client transactions, and general people's behavior. However, such models for event sequences usually process each sequence in isolation, ignoring context from those that co-occur in time. This limitation is particularly problematic in domains with fast-evolving conditions, like finance and e-commerce, or when certain sequences lack recent events. We develop a method that aggregates information from multiple user representations, augmenting a specific user's representation in a setting with multiple co-occurring event sequences, achieving better quality than processing each sequence independently. Our study considers diverse aggregation approaches, ranging from simple pooling techniques to Learnable attention aggregation, that can highlight more complex information flow among other users. The proposed methods operate on top of an existing encoder and support its efficient fine-tuning. Across nine diverse event sequence datasets (finance, e-commerce, entertainment, etc.) and downstream tasks, Learnable attention improves metric scores, both with and without fine-tuning, while mean pooling yields a smaller but still significant gain.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Recent advance in temporal point process: from machine learning perspective,
J. Yan, “Recent advance in temporal point process: from machine learning perspective,”SJTU Technical Report, 2019
work page 2019
-
[2]
ET-RNN: Applying deep learning to credit loan applications,
D. Babaevet al., “ET-RNN: Applying deep learning to credit loan applications,” inACM SIGKDD, pp. 2183–2190, 2019
work page 2019
-
[3]
Neural temporal point processes: A review,
O. Shchuret al., “Neural temporal point processes: A review,” inIJCAI, Survey track, pp. 4585–4593, 8 2021
work page 2021
-
[4]
Review of machine learning approach on credit card fraud detection,
B. Sulaimanet al., “Review of machine learning approach on credit card fraud detection,”Human-Centric Intelligent Systems, vol. 2, no. 1- 2, pp. 55–68, 2022
work page 2022
-
[5]
Coles: Contrastive learning for event sequences with self-supervision,
D. Babaevet al., “Coles: Contrastive learning for event sequences with self-supervision,” inICDM, 2022
work page 2022
-
[6]
A. Bazarovaet al., “Learning transactions representations for infor- mation management in banks: Mastering local, global, and external knowledge,”International Journal of Information Management Data Insights, vol. 5, no. 1, p. 100323, 2025
work page 2025
-
[7]
A. Zaytsevet al., “Designing an attack-defense game: how to increase robustness of financial transaction models via a competition,”ICDM, 2024
work page 2024
-
[8]
Shaping social activity by incentivizing users,
M. Farajtabaret al., “Shaping social activity by incentivizing users,” NeurIPS, vol. 27, 2014
work page 2014
Show all 47 references
-
[9]
Coevolve: A joint point process model for information diffusion and network evolution,
M. Farajtabaret al., “Coevolve: A joint point process model for information diffusion and network evolution,”JMLR, vol. 18, no. 41, pp. 1–49, 2017
2017
-
[10]
Learning and forecasting opinion dynamics in social networks,
A. De, S. others, and M. Gomez-Rodriguez, “Learning and forecasting opinion dynamics in social networks,” inNeurIPS, p. 397–405, Curran Associates Inc., 2016
2016
-
[11]
Mutually exciting point process graphs for modeling dynamic networks,
F. Passinoet al., “Mutually exciting point process graphs for modeling dynamic networks,”Journal of Computational and Graphical Statistics, vol. 32, no. 1, pp. 116–130, 2023
2023
-
[12]
Continuous-time convolutions model of event se- quences,
V . Zhuzhelet al., “Continuous-time convolutions model of event se- quences,”arXiv preprint arXiv:2302.06247, 2023
2023 arXiv
-
[13]
A deep learning model for behavioural credit scoring in banks,
M. Ala’rajet al., “A deep learning model for behavioural credit scoring in banks,”Neural Computing and Applications, pp. 1–28, 2022
2022
-
[14]
Time series change point detection with self- supervised contrastive predictive coding,
S. Deldariet al., “Time series change point detection with self- supervised contrastive predictive coding,” inthe Web Conference, 2021
2021
-
[15]
A survey on contrastive self-supervised learning,
A. Jaiswalet al., “A survey on contrastive self-supervised learning,” Technologies, vol. 9, no. 1, p. 2, 2020
2020
-
[16]
Emerging properties in self-supervised vision trans- formers,
M. Caronet al., “Emerging properties in self-supervised vision trans- formers,” inCVPR, pp. 9650–9660, 2021
2021
-
[17]
Self-supervised learning: Generative or contrastive,
X. Liuet al., “Self-supervised learning: Generative or contrastive,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 857–876, 2023
2023
-
[18]
Self-attentive hawkes process,
Q. Zhanget al., “Self-attentive hawkes process,” inICML, pp. 11183– 11193, PMLR, 2020
2020
-
[19]
Deep metric learning using triplet network,
E. Hoffer and N. Ailon, “Deep metric learning using triplet network,” inSimilarity-Based Pattern Recognition: Third International Workshop, SIMBAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceed- ings 3, pp. 84–92, Springer, 2015
2015
-
[20]
A simple framework for contrastive learning of visual representations,
T. Chenet al., “A simple framework for contrastive learning of visual representations,” inICML, pp. 1597–1607, PMLR, 2020
2020
-
[21]
Barlow twins: Self-supervised learning via redundancy reduction,
J. Zbontaret al., “Barlow twins: Self-supervised learning via redundancy reduction,” inICML, PMLR, 2021
2021
-
[22]
Self-supervised learning for time series analysis: Taxonomy, progress, and prospects,
K. Zhanget al., “Self-supervised learning for time series analysis: Taxonomy, progress, and prospects,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
2024
-
[23]
Long short-term memory,
S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997
1997
-
[24]
Attention is all you need,
A. Vaswaniet al., “Attention is all you need,”NeurIPS, vol. 30, 2017
2017
-
[25]
Ts2vec: Towards universal representation of time series,
Z. Yueet al., “Ts2vec: Towards universal representation of time series,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8980–8987, 2022
2022
-
[26]
Learning and forecasting opinion dynamics in social networks,
A. Deet al., “Learning and forecasting opinion dynamics in social networks,”NeurIPS, vol. 29, 2016
2016
-
[27]
Spectra of some self-exciting and mutually exciting point processes,
A. G. Hawkes, “Spectra of some self-exciting and mutually exciting point processes,”Biometrika, vol. 58, no. 1, pp. 83–90, 1971
1971
-
[28]
Hawkes processes and their applications to finance: a review,
A. Hawkes, “Hawkes processes and their applications to finance: a review,”Quantitative Finance, vol. 18, no. 2, pp. 193–198, 2018
2018
-
[29]
Semi-supervised classification with graph convolutional networks,
T. Kipfet al., “Semi-supervised classification with graph convolutional networks,” inICLR, 2022
2022
-
[30]
A comparative study of some point process models for dynamic networks,
S. Dizajiet al., “A comparative study of some point process models for dynamic networks,”Complexity, vol. 2022, no. 1, p. 1616116, 2022
2022
-
[31]
Linking bank clients using graph neural net- works powered by rich transactional data,
V . Shumovskaiaet al., “Linking bank clients using graph neural net- works powered by rich transactional data,”International Journal of Data Science and Analytics, vol. 12, no. 2, pp. 135–145, 2021
2021
-
[32]
A survey on session-based recommender systems,
S. Wanget al., “A survey on session-based recommender systems,”ACM Comput. Surv., vol. 54, July 2021
2021
-
[33]
Adaptive collaborative filtering with personalized time decay functions for financial product recommendation,
A. Ghiyeet al., “Adaptive collaborative filtering with personalized time decay functions for financial product recommendation,” inACM RecSys, RecSys ’23, (New York, NY , USA), p. 798–804, Association for Computing Machinery, 2023
2023
-
[34]
Performance evaluation of time-based recommendation system in collaborative filtering technique,
G. Jainet al., “Performance evaluation of time-based recommendation system in collaborative filtering technique,”Procedia Computer Science, vol. 218, pp. 1834–1844, 2023. International Conference on Machine Learning and Data Engineering
2023
-
[35]
Dynamic item-based recommendation algorithm with time decay,
C. Xiaet al., “Dynamic item-based recommendation algorithm with time decay,” inICNC, 2010
2010
-
[36]
Performance evaluation of aggregation-based group recommender systems for ephemeral groups,
E. Ceh-Varelaet al., “Performance evaluation of aggregation-based group recommender systems for ephemeral groups,”ACM TIST, vol. 13, no. 6, pp. 1–26, 2022
2022
-
[37]
Group recommendation with latent voting mechanism,
L. Guoet al., “Group recommendation with latent voting mechanism,” inICDE 2020, pp. 121–132, IEEE, 2020
2020
-
[38]
Glove: Global vectors for word representation,
J. Penningtonet al., “Glove: Global vectors for word representation,” in EMNLP, pp. 1532–1543, 2014
2014
-
[39]
A theoretical analysis of feature pooling in visual recognition,
Y . Boureauet al., “A theoretical analysis of feature pooling in visual recognition,” inICML, pp. 111–118, 2010
2010
-
[40]
Comparative analysis of pooling mechanisms in llms: A sentiment analysis perspective,
J. Xinget al., “Comparative analysis of pooling mechanisms in llms: A sentiment analysis perspective,”arXiv preprint arXiv:2411.14654, 2024
2024 arXiv
-
[41]
Temporal graph benchmark for machine learning on temporal graphs,
S. Huanget al., “Temporal graph benchmark for machine learning on temporal graphs,”NeurIPS, vol. 36, 2024
2024
-
[42]
Graph attention networks,
P. Veli ˇckovi´cet al., “Graph attention networks,” inICLR, 2018
2018
-
[43]
Bank transactions embeddings help to uncover current macroeconomics,
M. Begicheva and A. Zaytsev, “Bank transactions embeddings help to uncover current macroeconomics,” inICMLA, 2021
2021
-
[44]
EasyTPP: Towards open benchmarking the temporal point processes,
S. Xueet al., “EasyTPP: Towards open benchmarking the temporal point processes,”ICLR, 2023
2023
-
[45]
LightGBM: A highly efficient gradient boosting decision tree,
G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.- Y . Liu, “LightGBM: A highly efficient gradient boosting decision tree,” NeurIPS, vol. 30, pp. 3146–3154, 2017
2017
-
[46]
Seqnas: Neural architecture search for event sequence classification,
I. Udovichenkoet al., “Seqnas: Neural architecture search for event sequence classification,”IEEE Access, vol. 12, pp. 3898–3909, 2024
2024
-
[47]
A unified approach to interpreting model predic- tions,
S. Lundberget al., “A unified approach to interpreting model predic- tions,” inNeurIPS, pp. 4765–4774, Curran Associates, Inc., 2017. APPENDIX A. Hawkes-process inspired methods The original Hawkes process is described by the following formula for the conditional intensity fun...
2017
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.