Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Classical sequence models, trained from scratch, remain the best predictors of the next activity in a running process, while tabular foundation models are competitive on timing predictions.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-01 01:11 UTC pith:GNCU6AC2

load-bearing objection Useful, honest PPM benchmark with a real confound: tabular FMs get task-specific predictors while sequence/LLM baselines share a multi-task head, so the temporal-competitiveness claim needs qualification — and the novelty claim should be checked against ref [1]. the 3 major comments →

arxiv 2607.27797 v1 pith:GNCU6AC2 submitted 2026-07-30 cs.LG

Revisiting Predictive Process Monitoring in the Age of Foundation Models: A Comparative Study of Sequence, Tabular, and LLM Approaches

classification cs.LG
keywords predictive process monitoringlarge language modelstabular foundation modelssequence modelsnext activity predictionremaining time predictionnext event time predictionLoRA fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper compares three ways to build predictive process monitoring models: classical sequence models (LSTM, Transformer) trained from scratch, LoRA-adapted large language models, and tabular foundation models using in-context learning. Across five real-world event logs and three prediction tasks—next activity, remaining time, and time to next event—the authors find that sequence models consistently achieve the highest next-activity accuracy, while tabular foundation models match or beat them on the temporal tasks. LLMs generally trail on temporal tasks and cost far more in runtime. The paper's central message for practitioners is that no single paradigm dominates: sequence models for activity forecasting, tabular foundation models for timing estimates, and LLMs only when their sequential strengths outweigh high fine-tuning cost.

Core claim

The paper establishes, through a controlled benchmark with leakage-avoiding temporal splits, that no single paradigm dominates across all PPM tasks. Sequence models trained from scratch (LSTM and Transformer) achieve the best next-activity accuracy on all five logs, with margins up to 14 percentage points over tabular foundation models. Tabular foundation models are competitive on remaining-time prediction, taking the best result most frequently, and split the best next-event-time results with sequence models. LoRA-fine-tuned small LLMs, despite sharing the sequential prefix representation, usually lag behind on temporal tasks and exhibit dataset-dependent adaptation failures: on one log the

What carries the argument

The paper's conceptual framework is a three-way comparison along prefix representation, backbone parameterization, and adaptation regime. Sequence models and LLMs both encode prefixes as ordered event sequences, differing in pretraining and in whether only a small low-rank adapter is trained; tabular foundation models replace the sequence with a fixed-length feature vector (e.g., current activity, elapsed time, positional indicators), so ordering and temporal dependencies must be encoded explicitly. The benchmark uses this distinction to attribute performance differences: the branching analysis isolates when order information matters most, and the runtime measurements quantify the cost of ad

Load-bearing premise

The tabular foundation models receive engineered fixed-length feature vectors rather than the raw event order, so the finding that they are competitive on temporal tasks holds only to the extent that those features preserve the temporal signal—otherwise the comparison measures feature engineering, not paradigm capability.

What would settle it

Run a tabular foundation model on the same event logs with a feature set that includes the full ordered event history (e.g., frequency-encoded recent activities) and compare next-activity accuracy to the sequence models; if the tabular model still falls short by the same margin, the order-encoding hypothesis is wrong, and if it matches, the original gap was an artifact of the summary features. Alternatively, remove all engineered time-related features from the tabular inputs and check whether remaining-time and next-event-time performance collapses to majority-level.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For next-activity prediction, practitioners should prefer trained-from-scratch sequence models; the advantage holds across logs and grows with branching complexity.
  • For remaining-time and next-event-time predictions, tabular foundation models are a viable, cheap alternative to sequence models, often matching or beating them.
  • LoRA-fine-tuned small LLMs are generally not worth their runtime cost for these tasks, except possibly on logs where their sequential representation yields competitive next-activity accuracy.
  • The branching analysis offers a rule of thumb: if a process has low branching at the prediction point, tabular models may suffice; high branching favors sequence models.
  • Future PPM benchmarks should include tabular foundation models as a standard baseline, not only LLMs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the next-activity gap truly stems from order information, then giving a tabular foundation model richer sequential features (e.g., a padded vector of recent event embeddings) should close part of the gap; the paper's own remarks on feature engineering point to this test.
  • The premature end-of-sequence behavior seen in one LLM suggests that decoding with a process-specific stop criterion—rather than the standard end-of-sequence token—could lift LLM accuracy more than scaling up the backbone.
  • The dataset-dependent LLM deficit hints that adaptation quality, not paradigm capacity, is the limiting factor; larger or better-adapted LLMs remain an open possibility, as the authors acknowledge.
  • Because the paper finds paradigm-dependent winners, a practical portfolio approach is to run a sequence model and a tabular model per log, then choose per task based on measured branching levels.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a comparative benchmark of three PPM modeling paradigms—classical sequence models trained from scratch (LSTM, Transformer), LoRA-adapted LLMs (Llama-3.2-1B, Gemma-2-2b), and tabular foundation models with in-context learning (TabPFN-3, ConTextTab)—across five public event logs and three tasks (next activity, remaining time, next event time). The central empirical claims are that sequence models achieve the best next-activity accuracy on all logs, tabular FMs are competitive on the temporal tasks, and LLMs generally lag despite much higher runtime. The paper also provides a conceptual comparison of the paradigms and a branching-based analysis of when the sequence–tabular gap appears.

Significance. If the findings hold, the paper gives practitioners a useful, controlled benchmark: trained-from-scratch sequence models remain the default for NA prediction, tabular FMs are a viable option for RT/NT, and small LoRA-tuned LLMs are not cost-effective for these tasks. Strengths include the use of five public logs with a leakage-avoiding temporal split, multiple seeds, a shared code repository, and a first systematic evaluation of tabular FMs in PPM. The main claims are conditional on resolving a predictor-granularity confound and on adding statistical support, as detailed below.

major comments (3)
  1. [§5.1, Table 3] The temporal-task comparison is confounded by predictor granularity. Section 5.1 states that 'sequence models and LLM-based approaches jointly predict the three targets in a multi-task setting, whereas the tabular FMs use separate task-specific predictors.' Thus the RT/NT results—e.g., ConTextTab's 0.64 on BPI20RfP RT vs. Transformer 0.66 and LSTM 0.68, or TabPFN-3's 1.02 on BPI20PTC RT vs. LSTM 1.07—compare task-specific regressors against shared multi-task heads. A dedicated head can outperform a shared head even with the same backbone, so the conclusion that 'tabular foundation models are competitive on temporal tasks' is not a clean paradigm comparison. Please rerun the sequence and LLM baselines with task-specific heads (or, alternatively, a multi-task tabular predictor), or provide an ablation that isolates the effect of predictor granularity.
  2. [§5.2, Table 3] No statistical significance testing is reported across the five seeds. Many differences are within one standard deviation (e.g., BPI20RfP RT: ConTextTab 0.64±0.02 vs. TabPFN-3 0.66±0.02; BPI20PTC NA: Llama-3.2-1B 0.68±0.18). The claim that sequence models 'consistently' beat tabular models on NA, and the more nuanced 'competitive on temporal tasks,' would be much stronger with paired tests (e.g., Wilcoxon signed-rank or bootstrap over seeds) or effect sizes. At minimum, the authors should state which comparisons remain significant under multiple-testing correction.
  3. [§4.3, §5.1, §6] The temporal-task claim also depends on the engineered tabular feature representation. Tabular FMs receive a fixed-length feature vector (ϕ_tab) and no sequence order; the paper acknowledges in Section 6 that 'this work does not investigate the impact of alternative feature representations.' Since the RT/NT conclusion is central, the benchmark should include a classical tabular learner (e.g., XGBoost or a tuned gradient-boosting model) on the same features. Without such a baseline, it is unclear whether the tabular FMs' temporal competitiveness comes from in-context learning or from the feature-engineering itself. A feature ablation or a simple feature-sensitivity check would strengthen the paradigm-level interpretation.
minor comments (5)
  1. [Tables 1–3] Typos: 'T able' appears in the table captions. Also Table 1's formatting of 'BBθ BBθ0,ψ TFM' is garbled; please render clearly.
  2. [§5.2, Fig. 1] The branching analysis is based on 'one representative seed' and only BPI12, but the Discussion generalizes to 'the sequence–tabular gap generally increases with branching.' Please either present results across seeds and datasets, or clearly label this as a single-log illustrative observation.
  3. [§5.1, Features] The event definition includes a resource attribute r, but the feature list in §5.1 uses only activity and timestamp-derived numerical features. Please state whether resource was considered and, if not, why it was excluded from all paradigms.
  4. [§5.2, Runtime] The runtime values are described as 'indicative' and exclude preprocessing and setup. Please clarify whether they include inference/evaluation time for the LLMs, since the paragraph on runtime is otherwise hard to interpret for cost comparisons.
  5. [References] Some references are incomplete, e.g., [28] 'TabPFN-3: Technical report (2026)' lists no author, and [5] lacks publication venue/page details. Also, the GitHub URL in the footnote should be in the references or a data-availability statement.

Circularity Check

0 steps flagged

No circularity: the claims are empirical benchmark results against external datasets and open baselines, with acknowledged validity caveats rather than circular derivation.

full rationale

The paper's central claims are empirical outcomes of a controlled benchmark on five public event logs, comparing sequence models, LLM-based models, and tabular foundation models. There is no derivation chain in which an output is defined in terms of an input, no fitted parameter is renamed as a prediction, and no load-bearing argument relies on a self-citation. The benchmark follows Oyamada et al. [5] for datasets, preprocessing, and split, but the comparison is run independently against external public data and external model implementations (LSTM [15], Transformer, Llama-3.2-1B, Gemma-2-2b, TabPFN-3 [28], ConTextTab [7]); the cited prior work does not itself supply the paper's conclusions. The asymmetry noted in Section 5.1 — sequence and LLM models jointly predict all three targets in a multi-task setting, while tabular FMs use separate task-specific predictors — is a genuine methodological confound affecting interpretation of the temporal-task comparisons, but it is not circularity: it does not make the reported RT/NT numbers true by construction or reduce a prediction to a fitted input. Similarly, Section 6's caveat that the work does not investigate alternative feature representations is an acknowledged limitation, not a circular step. The branching analysis is a post hoc explanatory analysis of observed results, not a parameter fitted to those results. I find no self-definitional step, no fitted-input-called-prediction step, and no uniqueness or ansatz smuggled in via self-citation. The paper is self-contained as an empirical benchmark, so the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

All empirical conclusions rest on public datasets, a specific temporal split, metric choices, and explicit feature/adaptation design choices. No new entities, forces, or latent constructs are introduced. The free parameter is the per-family hyperparameter tuning, which is a legitimate but potentially fairness-relevant choice.

free parameters (1)
  • Per-family hyperparameters (LSTM/Transformer/LoRA rank/epochs) = Grid-selected values, e.g., LoRA rank r∈{16,32,64,128,256} with α=2r; Transformer epochs from {10,25}; LSTM layers/hidde
    Hyperparameters are selected separately per model family, so the comparison is not matched in compute or tuning budget. These choices can affect relative performance and are effectively free parameters of the benchmark.
axioms (5)
  • domain assumption The five public BPI event logs are valid, representative process data for the empirical claims
    All conclusions depend on the correctness and representativeness of BPI12, BPI17, BPI20PTC, BPI20RfP, and BPI20TPD. The paper does not validate the logs beyond their standard usage.
  • domain assumption The leakage-avoiding temporal split of Weytjens and De Weerdt [26] prevents train/test contamination
    Fairness of the benchmark relies on the split. If this split introduces shortcuts or mismatched distributions, the relative rankings could be artifacts.
  • domain assumption Five seeds and accuracy/MSE are sufficient statistical evidence for the conclusions
    The paper reports means and standard deviations but performs no paired significance tests. Several error bars overlap with differences the paper highlights.
  • domain assumption The engineered tabular feature vector (current activity, elapsed time, calendar features) preserves the information needed for RT/NT
    The tabular-FM representation summarizes prefixes as fixed-length vectors. The conclusion that tabular FMs are competitive on temporal tasks depends on this feature set being adequate.
  • domain assumption Activity labels treated as process-native tokens are a suitable input encoding for LLMs
    The LLM pipeline does not use natural-language descriptions; it feeds process tokens to the decoder. Whether this fully exploits or unfairly limits LLM capabilities is an open representation choice.

pith-pipeline@v1.3.0-daily-deepseek · 8461 in / 10447 out tokens · 102370 ms · 2026-08-01T01:11:13.123638+00:00 · methodology

0 comments
read the original abstract

Predictive process monitoring (PPM) leverages event logs to forecast the future of running process instances, for instance, predicting the next activity, the remaining time until case completion, or the time to the next event. While PPM research in recent years has been dominated by deep sequence models trained from scratch, such as Long Short-Term Memory (LSTM) models, foundation-model approaches---particularly large language models (LLMs)---are increasingly explored for PPM. At the same time, tabular foundation models with in-context learning capabilities offer a promising alternative but have not yet been systematically benchmarked for PPM. Thus, it remains unclear whether classical sequence-based models remain competitive in this evolving landscape. This paper compares the three modeling paradigms both conceptually and empirically through a controlled benchmark across multiple datasets and prediction tasks. The results show that sequence models consistently perform best for next activity prediction, whereas tabular foundation models are competitive on temporal tasks, with LLMs usually lagging behind despite higher cost.

Figures

Figures reproduced from arXiv: 2607.27797 by Lennart Fertig, Lukas Kirchdorfer, Tobias Sesterhenn.

Figure 1
Figure 1. Figure 1: Illustrative branching analysis for BPI12 based on one representative seed. Sequence and tabular models separate more strongly for next activity prediction as branching increases, whereas no comparable separation is visible for next event time prediction. Although LLM-based approaches preserve the sequential prefix representa￾tion, they remain affected by branching-related difficulty. Unlike the broadly lo… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 1 linked inside Pith

  1. [1]

    IEEEAccess14(2026)16959–16983

    Berti, A., Aalst, W.M.P.V.D.: An In-Context Foundation Model for Predic- tiveProcessMonitoringonEventLogs. IEEEAccess14(2026)16959–16983

  2. [2]

    IEEE Access11(2023) 76722– 76738

    Wang, J., et al.: MTLFormer: Multi-Task Learning Guided Transformer Network for Business Process Prediction. IEEE Access11(2023) 76722– 76738

  3. [3]

    Journal of Science & Technology (5) (Sep

    Tanikonda, A., Pandey, B.K., Peddinti, S.R., Katragadda, S.R.: Application of Transformer Models for Advanced Process Optimization and Process Mining. Journal of Science & Technology (5) (Sep. 2022) 128–150

  4. [4]

    IEEE TSC16(1) (2023) 739–756

    Rama-Maneiro, E., Vidal, J.C., Lama, M.: Deep Learning for Predictive Business Process Monitoring: Review and Benchmark. IEEE TSC16(1) (2023) 739–756

  5. [5]

    Oyamada, R.S., Peeperkorn, J., Weerdt, J.D., Smedt, J.D.: Domain Adap- tation of LLMs for Process Data (September 2025)

  6. [6]

    Nature637(8045) (January 2025) 319–326

    Hollmann, N., et al.: Accurate predictions on small data with a tabular foundation model. Nature637(8045) (January 2025) 319–326

  7. [7]

    In: NeurIPS

    Spinaci, M., Polewczyk, M., Schambach, M., Thelin, S.: Contexttab: A semantics-aware tabular in-context learner. In: NeurIPS. (2026)

  8. [8]

    In: Process Mining Handbook

    Di Francescomarino, C., Ghidini, C.: Predictive Process Monitoring. In: Process Mining Handbook. Springer (2022) 320–346

  9. [9]

    Volume 448 of Lecture Notes in Business Information Processing

    Van Der Aalst, W.M.P., Carmona, J., eds.: Process Mining Handbook. Volume 448 of Lecture Notes in Business Information Processing. Springer (2022)

  10. [10]

    Information Systems36(2) (2011) 450–475

    van der Aalst, W.M.P., Schonenberg, M.H., Song, M.: Time prediction based on process mining. Information Systems36(2) (2011) 450–475

  11. [11]

    Computing100(9) (2018) 1005–1031

    Polato, M., Sperduti, A., Burattin, A., de Leoni, M.: Time and activity sequence prediction of business process instances. Computing100(9) (2018) 1005–1031

  12. [12]

    In: IEEE SCC

    Appice, A., Di Mauro, N., Malerba, D.: Leveraging Shallow Machine Learn- ing to Predict Business Process Behavior. In: IEEE SCC. (2019)

  13. [13]

    Nature521(7553) (2015) 436–444

    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature521(7553) (2015) 436–444

  14. [14]

    BISE63(3) (2021) 261–276

    Kratsch, W., Manderscheid, J., Röglinger, M., Seyfried, J.: Machine Learn- ing in Business Process Monitoring: A Comparison of Deep Learning and Classical Approaches Used for Outcome Prediction. BISE63(3) (2021) 261–276

  15. [15]

    In: CAISE

    Tax, N., Verenich, I., La Rosa, M., Dumas, M.: Predictive Business Process Monitoring with LSTM Neural Networks. In: CAISE. (2017)

  16. [16]

    Camargo, M., Dumas, M., González-Rojas, O.: Learning Accurate LSTM Models of Business Processes. In: BPM. (2019)

  17. [17]

    In: AI*IA

    Di Mauro, N., et al.: Activity Prediction of Business Process Instances with Inception CNN Models. In: AI*IA. (2019) 12 L. Fertig et al

  18. [18]

    Knowledge-Based Systems248(2022) 108773

    Wickramanayake, B., et al.: Building interpretable models for business process prediction using shared and specialised attention mechanisms. Knowledge-Based Systems248(2022) 108773

  19. [19]

    Bukhsh, Z.A., Saeed, A., Dijkman, R.M.: ProcessTransformer: Predictive Business Process Monitoring with Transformer Network (April 2021)

  20. [20]

    https://arxiv.org/abs/2407.13244v1 (July 2024)

    Berti, A., Kourani, H., van der Aalst, W.M.P.: PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Tasks. https://arxiv.org/abs/2407.13244v1 (July 2024)

  21. [21]

    In: NeurIPS

    Brown, T., et al.: Language Models are Few-Shot Learners. In: NeurIPS. (2020)

  22. [22]

    Kubrak, K., et al.: Explanatory Capabilities of Large Language Models in Prescriptive Process Monitoring. In: BPM. (2024)

  23. [23]

    In: ICPM

    Pasquadibisceglie, V., Appice, A., Malerba, D.: LUPIN: A LLM Approach for Activity Suffix Prediction in Business Process Event Logs. In: ICPM. (2024)

  24. [24]

    In: ICPM

    Rebmann, A., Schmidt, F.D., Glavas, G., van der Aa, H.: Evaluating the ability of llms to solve semantics-aware process mining tasks. In: ICPM. (2024)

  25. [25]

    In: ICLR

    Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: ICLR. (2022)

  26. [26]

    In: BPM Workshops

    Weytjens,H.,DeWeerdt,J.: CreatingUnbiasedPublicBenchmarkDatasets with Data Leakage Prevention for Predictive Process Monitoring. In: BPM Workshops. (2022)

  27. [27]

    Roider, J., Zanca, D., Eskofier, B.M.: Efficient Training of Recurrent Neural Networks for Remaining Time Prediction in Predictive Process Monitoring. In: BPM. (2024)

  28. [28]

    Grinsztajn, L., et al.: TabPFN-3: Technical report (2026)