REVIEW 3 major objections 5 minor 28 references
Classical sequence models, trained from scratch, remain the best predictors of the next activity in a running process, while tabular foundation models are competitive on timing predictions.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 01:11 UTC pith:GNCU6AC2
load-bearing objection Useful, honest PPM benchmark with a real confound: tabular FMs get task-specific predictors while sequence/LLM baselines share a multi-task head, so the temporal-competitiveness claim needs qualification — and the novelty claim should be checked against ref [1]. the 3 major comments →
Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes, through a controlled benchmark with leakage-avoiding temporal splits, that no single paradigm dominates across all PPM tasks. Sequence models trained from scratch (LSTM and Transformer) achieve the best next-activity accuracy on all five logs, with margins up to 14 percentage points over tabular foundation models. Tabular foundation models are competitive on remaining-time prediction, taking the best result most frequently, and split the best next-event-time results with sequence models. LoRA-fine-tuned small LLMs, despite sharing the sequential prefix representation, usually lag behind on temporal tasks and exhibit dataset-dependent adaptation failures: on one log the
What carries the argument
The paper's conceptual framework is a three-way comparison along prefix representation, backbone parameterization, and adaptation regime. Sequence models and LLMs both encode prefixes as ordered event sequences, differing in pretraining and in whether only a small low-rank adapter is trained; tabular foundation models replace the sequence with a fixed-length feature vector (e.g., current activity, elapsed time, positional indicators), so ordering and temporal dependencies must be encoded explicitly. The benchmark uses this distinction to attribute performance differences: the branching analysis isolates when order information matters most, and the runtime measurements quantify the cost of ad
Load-bearing premise
The tabular foundation models receive engineered fixed-length feature vectors rather than the raw event order, so the finding that they are competitive on temporal tasks holds only to the extent that those features preserve the temporal signal—otherwise the comparison measures feature engineering, not paradigm capability.
What would settle it
Run a tabular foundation model on the same event logs with a feature set that includes the full ordered event history (e.g., frequency-encoded recent activities) and compare next-activity accuracy to the sequence models; if the tabular model still falls short by the same margin, the order-encoding hypothesis is wrong, and if it matches, the original gap was an artifact of the summary features. Alternatively, remove all engineered time-related features from the tabular inputs and check whether remaining-time and next-event-time performance collapses to majority-level.
If this is right
- For next-activity prediction, practitioners should prefer trained-from-scratch sequence models; the advantage holds across logs and grows with branching complexity.
- For remaining-time and next-event-time predictions, tabular foundation models are a viable, cheap alternative to sequence models, often matching or beating them.
- LoRA-fine-tuned small LLMs are generally not worth their runtime cost for these tasks, except possibly on logs where their sequential representation yields competitive next-activity accuracy.
- The branching analysis offers a rule of thumb: if a process has low branching at the prediction point, tabular models may suffice; high branching favors sequence models.
- Future PPM benchmarks should include tabular foundation models as a standard baseline, not only LLMs.
Where Pith is reading between the lines
- If the next-activity gap truly stems from order information, then giving a tabular foundation model richer sequential features (e.g., a padded vector of recent event embeddings) should close part of the gap; the paper's own remarks on feature engineering point to this test.
- The premature end-of-sequence behavior seen in one LLM suggests that decoding with a process-specific stop criterion—rather than the standard end-of-sequence token—could lift LLM accuracy more than scaling up the backbone.
- The dataset-dependent LLM deficit hints that adaptation quality, not paradigm capacity, is the limiting factor; larger or better-adapted LLMs remain an open possibility, as the authors acknowledge.
- Because the paper finds paradigm-dependent winners, a practical portfolio approach is to run a sequence model and a tabular model per log, then choose per task based on measured branching levels.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a comparative benchmark of three PPM modeling paradigms—classical sequence models trained from scratch (LSTM, Transformer), LoRA-adapted LLMs (Llama-3.2-1B, Gemma-2-2b), and tabular foundation models with in-context learning (TabPFN-3, ConTextTab)—across five public event logs and three tasks (next activity, remaining time, next event time). The central empirical claims are that sequence models achieve the best next-activity accuracy on all logs, tabular FMs are competitive on the temporal tasks, and LLMs generally lag despite much higher runtime. The paper also provides a conceptual comparison of the paradigms and a branching-based analysis of when the sequence–tabular gap appears.
Significance. If the findings hold, the paper gives practitioners a useful, controlled benchmark: trained-from-scratch sequence models remain the default for NA prediction, tabular FMs are a viable option for RT/NT, and small LoRA-tuned LLMs are not cost-effective for these tasks. Strengths include the use of five public logs with a leakage-avoiding temporal split, multiple seeds, a shared code repository, and a first systematic evaluation of tabular FMs in PPM. The main claims are conditional on resolving a predictor-granularity confound and on adding statistical support, as detailed below.
major comments (3)
- [§5.1, Table 3] The temporal-task comparison is confounded by predictor granularity. Section 5.1 states that 'sequence models and LLM-based approaches jointly predict the three targets in a multi-task setting, whereas the tabular FMs use separate task-specific predictors.' Thus the RT/NT results—e.g., ConTextTab's 0.64 on BPI20RfP RT vs. Transformer 0.66 and LSTM 0.68, or TabPFN-3's 1.02 on BPI20PTC RT vs. LSTM 1.07—compare task-specific regressors against shared multi-task heads. A dedicated head can outperform a shared head even with the same backbone, so the conclusion that 'tabular foundation models are competitive on temporal tasks' is not a clean paradigm comparison. Please rerun the sequence and LLM baselines with task-specific heads (or, alternatively, a multi-task tabular predictor), or provide an ablation that isolates the effect of predictor granularity.
- [§5.2, Table 3] No statistical significance testing is reported across the five seeds. Many differences are within one standard deviation (e.g., BPI20RfP RT: ConTextTab 0.64±0.02 vs. TabPFN-3 0.66±0.02; BPI20PTC NA: Llama-3.2-1B 0.68±0.18). The claim that sequence models 'consistently' beat tabular models on NA, and the more nuanced 'competitive on temporal tasks,' would be much stronger with paired tests (e.g., Wilcoxon signed-rank or bootstrap over seeds) or effect sizes. At minimum, the authors should state which comparisons remain significant under multiple-testing correction.
- [§4.3, §5.1, §6] The temporal-task claim also depends on the engineered tabular feature representation. Tabular FMs receive a fixed-length feature vector (ϕ_tab) and no sequence order; the paper acknowledges in Section 6 that 'this work does not investigate the impact of alternative feature representations.' Since the RT/NT conclusion is central, the benchmark should include a classical tabular learner (e.g., XGBoost or a tuned gradient-boosting model) on the same features. Without such a baseline, it is unclear whether the tabular FMs' temporal competitiveness comes from in-context learning or from the feature-engineering itself. A feature ablation or a simple feature-sensitivity check would strengthen the paradigm-level interpretation.
minor comments (5)
- [Tables 1–3] Typos: 'T able' appears in the table captions. Also Table 1's formatting of 'BBθ BBθ0,ψ TFM' is garbled; please render clearly.
- [§5.2, Fig. 1] The branching analysis is based on 'one representative seed' and only BPI12, but the Discussion generalizes to 'the sequence–tabular gap generally increases with branching.' Please either present results across seeds and datasets, or clearly label this as a single-log illustrative observation.
- [§5.1, Features] The event definition includes a resource attribute r, but the feature list in §5.1 uses only activity and timestamp-derived numerical features. Please state whether resource was considered and, if not, why it was excluded from all paradigms.
- [§5.2, Runtime] The runtime values are described as 'indicative' and exclude preprocessing and setup. Please clarify whether they include inference/evaluation time for the LLMs, since the paragraph on runtime is otherwise hard to interpret for cost comparisons.
- [References] Some references are incomplete, e.g., [28] 'TabPFN-3: Technical report (2026)' lists no author, and [5] lacks publication venue/page details. Also, the GitHub URL in the footnote should be in the references or a data-availability statement.
Circularity Check
No circularity: the claims are empirical benchmark results against external datasets and open baselines, with acknowledged validity caveats rather than circular derivation.
full rationale
The paper's central claims are empirical outcomes of a controlled benchmark on five public event logs, comparing sequence models, LLM-based models, and tabular foundation models. There is no derivation chain in which an output is defined in terms of an input, no fitted parameter is renamed as a prediction, and no load-bearing argument relies on a self-citation. The benchmark follows Oyamada et al. [5] for datasets, preprocessing, and split, but the comparison is run independently against external public data and external model implementations (LSTM [15], Transformer, Llama-3.2-1B, Gemma-2-2b, TabPFN-3 [28], ConTextTab [7]); the cited prior work does not itself supply the paper's conclusions. The asymmetry noted in Section 5.1 — sequence and LLM models jointly predict all three targets in a multi-task setting, while tabular FMs use separate task-specific predictors — is a genuine methodological confound affecting interpretation of the temporal-task comparisons, but it is not circularity: it does not make the reported RT/NT numbers true by construction or reduce a prediction to a fitted input. Similarly, Section 6's caveat that the work does not investigate alternative feature representations is an acknowledged limitation, not a circular step. The branching analysis is a post hoc explanatory analysis of observed results, not a parameter fitted to those results. I find no self-definitional step, no fitted-input-called-prediction step, and no uniqueness or ansatz smuggled in via self-citation. The paper is self-contained as an empirical benchmark, so the appropriate circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (1)
- Per-family hyperparameters (LSTM/Transformer/LoRA rank/epochs) =
Grid-selected values, e.g., LoRA rank r∈{16,32,64,128,256} with α=2r; Transformer epochs from {10,25}; LSTM layers/hidde
axioms (5)
- domain assumption The five public BPI event logs are valid, representative process data for the empirical claims
- domain assumption The leakage-avoiding temporal split of Weytjens and De Weerdt [26] prevents train/test contamination
- domain assumption Five seeds and accuracy/MSE are sufficient statistical evidence for the conclusions
- domain assumption The engineered tabular feature vector (current activity, elapsed time, calendar features) preserves the information needed for RT/NT
- domain assumption Activity labels treated as process-native tokens are a suitable input encoding for LLMs
read the original abstract
Predictive process monitoring (PPM) leverages event logs to forecast the future of running process instances, for instance, predicting the next activity, the remaining time until case completion, or the time to the next event. While PPM research in recent years has been dominated by deep sequence models trained from scratch, such as Long Short-Term Memory (LSTM) models, foundation-model approaches---particularly large language models (LLMs)---are increasingly explored for PPM. At the same time, tabular foundation models with in-context learning capabilities offer a promising alternative but have not yet been systematically benchmarked for PPM. Thus, it remains unclear whether classical sequence-based models remain competitive in this evolving landscape. This paper compares the three modeling paradigms both conceptually and empirically through a controlled benchmark across multiple datasets and prediction tasks. The results show that sequence models consistently perform best for next activity prediction, whereas tabular foundation models are competitive on temporal tasks, with LLMs usually lagging behind despite higher cost.
Figures
Reference graph
Works this paper leans on
-
[1]
IEEEAccess14(2026)16959–16983
Berti, A., Aalst, W.M.P.V.D.: An In-Context Foundation Model for Predic- tiveProcessMonitoringonEventLogs. IEEEAccess14(2026)16959–16983
2026
-
[2]
IEEE Access11(2023) 76722– 76738
Wang, J., et al.: MTLFormer: Multi-Task Learning Guided Transformer Network for Business Process Prediction. IEEE Access11(2023) 76722– 76738
2023
-
[3]
Journal of Science & Technology (5) (Sep
Tanikonda, A., Pandey, B.K., Peddinti, S.R., Katragadda, S.R.: Application of Transformer Models for Advanced Process Optimization and Process Mining. Journal of Science & Technology (5) (Sep. 2022) 128–150
2022
-
[4]
IEEE TSC16(1) (2023) 739–756
Rama-Maneiro, E., Vidal, J.C., Lama, M.: Deep Learning for Predictive Business Process Monitoring: Review and Benchmark. IEEE TSC16(1) (2023) 739–756
2023
-
[5]
Oyamada, R.S., Peeperkorn, J., Weerdt, J.D., Smedt, J.D.: Domain Adap- tation of LLMs for Process Data (September 2025)
2025
-
[6]
Nature637(8045) (January 2025) 319–326
Hollmann, N., et al.: Accurate predictions on small data with a tabular foundation model. Nature637(8045) (January 2025) 319–326
2025
-
[7]
In: NeurIPS
Spinaci, M., Polewczyk, M., Schambach, M., Thelin, S.: Contexttab: A semantics-aware tabular in-context learner. In: NeurIPS. (2026)
2026
-
[8]
In: Process Mining Handbook
Di Francescomarino, C., Ghidini, C.: Predictive Process Monitoring. In: Process Mining Handbook. Springer (2022) 320–346
2022
-
[9]
Volume 448 of Lecture Notes in Business Information Processing
Van Der Aalst, W.M.P., Carmona, J., eds.: Process Mining Handbook. Volume 448 of Lecture Notes in Business Information Processing. Springer (2022)
2022
-
[10]
Information Systems36(2) (2011) 450–475
van der Aalst, W.M.P., Schonenberg, M.H., Song, M.: Time prediction based on process mining. Information Systems36(2) (2011) 450–475
2011
-
[11]
Computing100(9) (2018) 1005–1031
Polato, M., Sperduti, A., Burattin, A., de Leoni, M.: Time and activity sequence prediction of business process instances. Computing100(9) (2018) 1005–1031
2018
-
[12]
In: IEEE SCC
Appice, A., Di Mauro, N., Malerba, D.: Leveraging Shallow Machine Learn- ing to Predict Business Process Behavior. In: IEEE SCC. (2019)
2019
-
[13]
Nature521(7553) (2015) 436–444
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature521(7553) (2015) 436–444
2015
-
[14]
BISE63(3) (2021) 261–276
Kratsch, W., Manderscheid, J., Röglinger, M., Seyfried, J.: Machine Learn- ing in Business Process Monitoring: A Comparison of Deep Learning and Classical Approaches Used for Outcome Prediction. BISE63(3) (2021) 261–276
2021
-
[15]
In: CAISE
Tax, N., Verenich, I., La Rosa, M., Dumas, M.: Predictive Business Process Monitoring with LSTM Neural Networks. In: CAISE. (2017)
2017
-
[16]
Camargo, M., Dumas, M., González-Rojas, O.: Learning Accurate LSTM Models of Business Processes. In: BPM. (2019)
2019
-
[17]
In: AI*IA
Di Mauro, N., et al.: Activity Prediction of Business Process Instances with Inception CNN Models. In: AI*IA. (2019) 12 L. Fertig et al
2019
-
[18]
Knowledge-Based Systems248(2022) 108773
Wickramanayake, B., et al.: Building interpretable models for business process prediction using shared and specialised attention mechanisms. Knowledge-Based Systems248(2022) 108773
2022
-
[19]
Bukhsh, Z.A., Saeed, A., Dijkman, R.M.: ProcessTransformer: Predictive Business Process Monitoring with Transformer Network (April 2021)
2021
-
[20]
https://arxiv.org/abs/2407.13244v1 (July 2024)
Berti, A., Kourani, H., van der Aalst, W.M.P.: PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks. https://arxiv.org/abs/2407.13244v1 (July 2024)
Pith/arXiv arXiv 2024
-
[21]
In: NeurIPS
Brown, T., et al.: Language Models are Few-Shot Learners. In: NeurIPS. (2020)
2020
-
[22]
Kubrak, K., et al.: Explanatory Capabilities of Large Language Models in Prescriptive Process Monitoring. In: BPM. (2024)
2024
-
[23]
In: ICPM
Pasquadibisceglie, V., Appice, A., Malerba, D.: LUPIN: A LLM Approach for Activity Suffix Prediction in Business Process Event Logs. In: ICPM. (2024)
2024
-
[24]
In: ICPM
Rebmann, A., Schmidt, F.D., Glavas, G., van der Aa, H.: Evaluating the ability of llms to solve semantics-aware process mining tasks. In: ICPM. (2024)
2024
-
[25]
In: ICLR
Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: ICLR. (2022)
2022
-
[26]
In: BPM Workshops
Weytjens,H.,DeWeerdt,J.: CreatingUnbiasedPublicBenchmarkDatasets with Data Leakage Prevention for Predictive Process Monitoring. In: BPM Workshops. (2022)
2022
-
[27]
Roider, J., Zanca, D., Eskofier, B.M.: Efficient Training of Recurrent Neural Networks for Remaining Time Prediction in Predictive Process Monitoring. In: BPM. (2024)
2024
-
[28]
Grinsztajn, L., et al.: TabPFN-3: Technical report (2026)
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.