{"id":"03f0e217-1c51-423e-bd83-ac065a6df2b6","arxiv_id":"2411.14263","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Latent-space perturbations of VAE-encoded business process traces generate realistic-looking adversarial examples that flip outcome predictions more often than input-space permutations.","lead":"This paper proposes two latent-space attacks that create adversarial examples for predictive process monitoring models by perturbing the encoded representation of process traces and decoding back to sequences. A benchmark on eleven event logs and four classifiers reports the latent sampling attack as the most successful at flipping predictions, while keeping examples close to the original data distribution.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Latent Sampling attack realism and label invariance are asserted, not verified; success-rate comparison may conflate 'realistic' with merely classifier-flipping.","rationale":"The reader's weakest_assumption precisely identifies the same load-bearing concern: the lack of independent validation that latent-space decoding yields realistic traces with preserved true labels. This is the core of the paper's contribution ('realistic adversarial examples'). The paper's internal text supports this concern: Section III-C defines success without label verification, and the realism metrics (EMD, DL Edit, Euclidean distance) are proxies, not process-model checks. Additionally, the difference between Latent Sampling (50.17%) and the best projected attack (50.08%) is tiny, so the claim of 'most effective' is fragile without statistical testing. But the paper is otherwise a solid benchmark with a legitimate empirical setup: 11 logs, 4 models, 8 attacks, and the code is referenced. The correct response is CONDITIONAL, not REJECT, because the identified issue is addressable via a concrete validation step (process-fit and label-invariance checks, significance testing).","tokens_in":20351,"tokens_out":1650,"duration_ms":14240,"concrete_test":"Run the released code from Section IV-C on one event log (e.g., BPIC2012) to compare the latent-sampling generated adversarial traces against the original event log using an independent process-mining checker. Compute the fraction of generated traces that fit a directly-follows graph or a discovered process model (e.g., inductive miner) trained on the training set. Also verify label invariance by checking whether each generated trace's true label (e.g., via the LTL rule used in Section IV-A) matches the original true label. If less than, say, 95% of generated traces are process-fit or if label invariance fails for a non-negligible fraction, the claim is unsupported. Additionally, run a paired significance test (e.g., McNemar) on the success rates of Latent Sampling vs 3 Event (Projected).","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the Latent Sampling attack generates realistic adversarial examples with the highest success rate (50.17%) while remaining within the class-specific data distribution, without domain knowledge (Section V-A, Figure 7). The load-bearing assumption is that decoding a perturbed latent code sampled via the reparameterization trick produces traces that (a) are realistic/executable and (b) preserve the true label. The paper explicitly notes that success is judged solely by classifier prediction flip (Section III-C, Eq. 1), and realism is operationalized only through VAE reconstruction/Latent distances and EMD/DL Edit (Section V-B, Tables III-IV). Critically, Section III-C equates 'successful' with 'prediction changes' and never verifies label invariance for latent-space outputs: the true label of generated prefixes is never checked against process-model constraints or domain validation. Moreover, the projected attacks achieve 50.08% and the difference of 0.09 percentage points is presented as 'most effective' without significance testing. Since the latent sampling method samples z = mu + eps*sigma with eps ~ N(0,1), large-scale perturbations can cross the class boundary in latent space, producing decoded sequences that may not be valid traces. The paper's own cluster analysis shows Latent Sampling traces fall into the 'Others' or 'Subtle' categories, but this does not confirm that the decoded traces are executable. The realism claim is therefore not independently supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies adversarial attacks on outcome-oriented predictive process monitoring (OOPPM). It introduces two latent-space attacks, Latent Sampling and Gradient Steps, built on class-specific LSTM variational autoencoders, and benchmarks them against six input-space attacks (three regular permutation attacks and three projected variants) across eleven real-life event logs and four classifiers. The headline result is that Latent Sampling achieves the highest average attack success rate (50.17%), slightly ahead of projected 3-event attacks (50.08%), and the authors claim it produces realistic, domain-agnostic adversarial examples without process-specific knowledge. The paper also proposes a cluster-profile analysis of attack types and releases code based on CARLA.","tokens_in":20620,"tokens_out":5013,"duration_ms":48704,"significance":"The paper addresses a relevant and under-studied problem, and the benchmark is broad: eleven logs, four classifiers, eight attack methods, with a public code repository. If the realism and label-invariance claims were validated, domain-agnostic latent-space attacks would be a useful contribution to robustness evaluation in PPM, especially as they avoid hand-crafted process constraints. However, the current manuscript does not provide independent evidence that decoded latent-space examples are feasible process traces or preserve true labels, and the success-rate advantage over projected attacks is not statistically established. These gaps directly affect the central claim, so the contribution is interesting but not yet fully supported.","major_comments":[{"comment":"The label invariance assumption is defined in Eq. (1) but never verified for any generated adversarial example. Section III-C defines attack success solely as a change in the classifier's prediction, and Section V-A reports success rates using that definition. For the latent space attacks, the true label of the decoded example is never compared with the original label, and decoded traces are not checked against the LTL rules used to define the labels in Section IV-A or against any process-model constraint. This matters because Section III-A states that projected and latent attacks can change the prefix length after decoding; a length or control-flow change can alter the true outcome. The claim in Section V-A that latent sampling produces realistic attacks is therefore not supported by the presented evidence. The authors should report, at minimum, the fraction of generated examples that appear in the event log or that satisfy a directly-follows or process-model conformance check, and restrict success-rate comparisons to examples whose true label is verified unchanged.","section":"Section III-C, Eq. (1)"},{"comment":"The average success rate of Latent Sampling is 50.17%, while 3 Event (Projected) achieves 50.08%, a difference of 0.09 percentage points. The paper states that Latent Sampling 'remains the most effective across all the four OOPPM methods' without reporting any significance test, confidence interval, or per-log variance. Since the averages are aggregated over eleven event logs and four classifiers, a paired test across logs or a per-classifier Wilcoxon signed-rank test is needed before claiming superiority; on the reported aggregate numbers, the difference is not statistically distinguishable from noise.","section":"Section V-A"},{"comment":"The Gradient Steps attack is described as applicable only to the LSTM classifier, yet its reported 33.17% success rate is included in the global ranking and in Figure 7's box plots along with the other classifiers. This mixes different support sets: Gradient Steps is evaluated on a single classifier, while the other methods are averaged over four classifiers. The ranking of methods is therefore not directly comparable. The authors should report results separately for the common support, or present per-classifier tables, so that the ranking is not confounded by the classifier subset.","section":"Section V-A, Figure 7"},{"comment":"The realism evaluation operationalizes 'realistic' through Euclidean distance in latent space, EMD, and DL Edit distance to the original prefix. These are similarity measures, not validity measures. The fact that Latent Sampling falls into the 'Others' cluster, with low-to-moderate EMD and DL Edit values, does not establish that the decoded sequences are feasible process executions. For sparse logs such as BPIC2015, the class-specific VAE may decode sequences that never occur in the log; no generative validation is reported, such as decoding random class-specific latent codes and checking them against a discovered process model or the original event log. The conclusion in Section V-B that latent attacks 'conform to the original data distribution' is therefore an assumption inherited from the VAE training objective, not a demonstrated property of the generated adversarial examples.","section":"Sections V-B and V-C"}],"minor_comments":[{"comment":"The text says the paper evaluates 'these two latent space methods with six other adversarial attacking methods', but then describes only 'the first three attacking methods' and 'the fourth method'. The counting should be clarified: the projected strategy yields three attacks, so there are six non-latent methods in total.","section":"Abstract and Section I"},{"comment":"The assignment 'mu, ←encodeonlatentspace(prefixes)' is syntactically incomplete; it should be 'mu, _ ← encodeonlatentspace(prefixes)' or 'mu ← encodeonlatentspace(prefixes)'.","section":"Algorithm 1, line 24"},{"comment":"The column 'LCP' is not defined in the table caption or the surrounding text; it should be defined explicitly, for example as the longest common prefix length between original and adversarial trace.","section":"Table III"},{"comment":"The cluster-profile thresholds are described verbally as 'third quartile', 'first quartile', and 'below the median' after normalization, but the exact normalization and threshold computation are not stated; please provide the precise formulas and quartile values for reproducibility.","section":"Section V-C"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a solid empirical benchmark: eight attack methods, four classifiers, eleven real event logs. That is genuinely useful for the PPM community. Second, the headline claim—that the latent sampling attack generates 'realistic' adversarial examples—is asserted, not demonstrated. The paper's definition of success is purely a classifier flip (Eq. 1) and it never checks whether the generated prefix's true label is preserved.\n\nWhat's new here: the k-event attack (a contextual permutation that respects activity-position tuples) is a reasonable addition, and adapting latent-space perturbation to discrete event sequences is a sensible extension of prior work. The benchmark itself is the main contribution. The conceptual ingredients—VAEs, latent-space noise, reparameterization tricks—are standard elsewhere, so novelty is moderate, as the reader says.\n\nThe soft spots are real. Realism is operationalized through reconstruction loss, latent distance, EMD, and DL edit distance—none of which verify that a decoded sequence is an executable process trace. The stress-test note is right: sampling z = mu + eps * sigma can cross class boundaries or produce invalid sequences. The paper's cluster analysis places latent sampling in 'Others', which does not demonstrate realism. Also, the success-rate margin is trivial: 50.17% vs. 50.08% for the 3-event projected attack, with no significance testing. Calling latent sampling 'most effective' overstates a difference that is likely noise. Gradient steps is only run against the LSTM, so its ranking isn't comparable across classifiers. Finally, the code is referenced but not accessible, which weakens reproducibility.\n\nWho is this for? People working on adversarial robustness or attack generation in predictive process monitoring. They'll get a useful baseline and a benchmark they can extend. The paper deserves serious peer review, but the realism claim needs to be supported with direct evidence: process-model conformance, label-invariance checks on generated traces, or domain-expert evaluation. If that evidence is missing after revision, the realistic framing should be dropped or heavily qualified. I'd also ask for a significance test or confidence interval on the success-rate comparisons.\n\nI'd bring this to a reading group only if someone is actively working on PPM attacks. Otherwise, it's worth reading but not urgent.\n\nRecommendation: engage with it, but require the missing validation before accepting the central claims.","headline":"Useful PPM benchmark, but the 'realistic' claim for latent-space attacks is unverified and the success margin over projected attacks is negligible.","tokens_in":21132,"tokens_out":2753,"would_cite":false,"duration_ms":23844,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By sampling nearby points in a class-specific variational autoencoder's latent space and decoding them back to activity sequences, this paper generates realistic adversarial examples that flip outcome predictions of business-process…","keywords":["adversarial machine learning","predictive process monitoring","variational autoencoder","latent space attacks","adversarial examples","process mining","outcome-oriented predictive process monitoring","manifold learning"],"falsifier":"For a fixed event log, take the latent sampling attack's generated adversarial traces and check each one against a process model discovered from the training log (e.g., a directly-follows graph or a declarative model); if a substantial fraction of successful adversarial traces violate the model's allowed behavior, or if domain experts judge them impossible, the realism claim is refuted. A second check is to re-run the attack while measuring whether the true outcome label stays invariant under the perturbation; if it does not, the reported success rates overstate vulnerability.","tokens_in":20152,"feed_emoji":"🎯","tokens_out":7489,"duration_ms":61281,"temperature":0.7,"pith_summary":"This paper claims that realistic adversarial examples for business-process outcome predictions can be generated by perturbing the latent space of a class-specific variational autoencoder, rather than by directly editing the activity sequence. The authors introduce two latent-space attacks—a black-box latent sampling attack and a white-box gradient steps attack—and benchmark them against six input-space methods on eleven real-life event logs and four classifiers. The latent sampling attack achieves the highest average success rate (50.17%), narrowly edging out the best projected input-space attack (50.08%) and far surpassing regular edits (12–38%). If this holds, OOPPM models are vulnerable to realistic, domain-agnostic attacks that produce plausible-looking traces, which matters for high-stakes decisions such as loan approvals or hospital discharge.","feed_headline":"VAE latent sampling flips process-outcome predictions at 50 percent","feed_subtitle":"Latent sampling tops six rival attack methods on 11 event logs without any domain-specific tuning.","key_machinery":"The central object is a class-wise Long Short-Term Memory variational autoencoder that learns a low-dimensional Gaussian latent manifold for each outcome class from trace prefixes. The reparameterization trick $z = \\mu + \\epsilon \\cdot \\sigma$ generates stochastic latent samples from the same mean and variance as the original trace, and the decoder maps them back to activity sequences. The pipeline then encodes the original and all candidate decoded traces, and selects the one with smallest Euclidean distance in latent space as the adversarial example, an additional heuristic intended to keep the perturbation close to the original while staying on the learned manifold.","core_discovery":"The paper establishes that adversarial examples for discrete business-process sequences can be generated in a continuous latent space: encode a trace prefix into the class-specific LSTM VAE, sample a nearby latent code via the reparameterization trick, decode, and select the adversarial candidate closest to the original in latent space. This latent sampling attack is reported to be the most effective across all four OOPPM classifiers, with a 50.17% average success rate, and produces examples with low-to-moderate distributional and sequence distance, falling into the 'Others' profile that is subtle enough to avoid easy detection yet highly successful. The white-box gradient steps attack, adapted from a counterfactual-generation method, is included mainly as a benchmark and reaches only 33.17% average success, underscoring that strong attacks on these models do not require gradient access.","pith_inferences":["Beyond the paper: the gap between latent sampling (50.17%) and the best projected attack (50.08%) is small enough that the 'most effective' claim should be read cautiously unless paired significance tests over event logs confirm it.","Beyond the paper: because the paper never validates that decoded traces obey real process constraints, some successful adversarial examples may be impossible sequences; a defender who can discover the underlying process model could filter them, lowering the true vulnerability.","Beyond the paper: the latent sampling recipe should transfer to other discrete sequence classifiers (clinical pathways, software logs, user journeys), and a testable extension is to measure attack success plus human-expert realism in those domains.","Beyond the paper: since latent sampling only uses the VAE's mean and variance and never queries the classifier's gradients, combining its candidates with a query-efficient black-box search over the latent code could push success rates higher while remaining domain-agnostic."],"forward_implications":["OOPPM classifiers (logistic regression, XGBoost, random forest, LSTM) can be fooled by realistic-looking, domain-agnostic attacks at roughly 50% success, so deployments in loan approval or healthcare should include adversarial robustness testing.","The latent sampling attack is black-box and model-agnostic, so the generated examples can be stored as a robustness test set and reused to compare any classifier.","Attack success increases with prefix length for latent-space and projected attacks, meaning longer-running cases are more vulnerable than early predictions.","The white-box gradient steps attack (33% average success) underperforms the black-box latent sampling attack, suggesting that gradient information is not necessary for strong attacks on these models.","The 'Others' cluster profile, occupied by latent sampling and projected attacks, shows that modest hard-to-detect changes can be highly effective, pointing to a need for defenses that target distributional shifts."],"supporting_citations":[{"why":"Supplies the VAE and the reparameterization trick that the latent sampling attack uses to draw nearby latent codes.","marker":"[16]"},{"why":"Supplies the gradient-based counterfactual generation approach adapted into the white-box gradient steps attack benchmark.","marker":"[14]"},{"why":"Defines the regular last-event and all-event attacks and the vulnerability assessment setup that this paper extends and compares against.","marker":"[30]"},{"why":"Introduces the projected on-manifold attack strategy and class-specific manifolds that are the main predecessor and strongest baseline (50.08 percent).","marker":"[32]"},{"why":"Provides the eleven event-log prediction tasks, LTL labelling rules, preprocessing, and the four benchmark classifiers used in the evaluation.","marker":"[38]"},{"why":"Provides the LSTM VAE implementation that the class-specific manifolds in this paper are built on.","marker":"[31]"},{"why":"Supplies the counterfactual-generation benchmark harness that was transformed into the adversarial-example evaluation framework.","marker":"[21]"}],"fun_headline_variants":["VAE latent attacks fool process models 50% of the time","Latent-space VAE attacks beat six rivals on 11 event logs","No domain knowledge needed: VAE attacks flip outcomes 50%","Latent sampling tops process-prediction attacks at 50%","Realistic adversarial examples via latent VAE perturbations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's realism and label invariance rest on the assumption that decoding a perturbed latent code from the class-specific VAE always yields a valid, executable process trace whose true outcome is unchanged, but the paper never verifies this against process models or domain knowledge.","fun_headline_variants_meta":{"raw":{"variants":["VAE latent attacks fool process models 50% of the time","Latent-space VAE attacks beat six rivals on 11 event logs","No domain knowledge needed: VAE attacks flip outcomes 50%","Latent sampling tops process-prediction attacks at 50%","Realistic adversarial examples via latent VAE perturbations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1305,"prompt_tokens":957,"completion_tokens":348,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":260}},"tokens_in":573,"tokens_out":348,"duration_ms":4125,"temperature":1.0,"reasoning_tokens":260,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:21:33.271467+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a fixed event log, take the latent sampling attack's generated adversarial traces and check each one against a process model discovered from the training log (e.g., a directly-follows graph or a declarative model); if a substantial fraction of successful adversarial traces violate the model's allowed behavior, or if domain experts judge them impossible, the realism claim is refuted. A second check is to re-run the attack while measuring whether the true outcome label stays invariant under the perturbation; if it does not, the reported success rates overstate vulnerability.","supporting_citations":[{"cited_title":"In: 2022 4th International Conference on Process Mining (ICPM)","cited_arxiv_id":null,"evidence_quote":"Defines the regular last-event and all-event attacks and the vulnerability assessment setup that this paper extends and compares against."},{"cited_title":"In: 2023 5th International Conference on Process Mining (ICPM)","cited_arxiv_id":null,"evidence_quote":"Introduces the projected on-manifold attack strategy and class-specific manifolds that are the main predecessor and strongest baseline (50.08 percent)."},{"cited_title":"Generating Feasible and Plausible Counterfactual Explanations for Outcome Prediction of Business Processes","cited_arxiv_id":"2403.09232","evidence_quote":"Provides the LSTM VAE implementation that the class-specific manifolds in this paper are built on."},{"cited_title":"In: Van- schoren, J., Yeung, S","cited_arxiv_id":null,"evidence_quote":"Supplies the counterfactual-generation benchmark harness that was transformed into the adversarial-example evaluation framework."}],"review_version":1}