{"id":"1acda4eb-c19d-495d-a88a-db346dbfae73","arxiv_id":"2507.23037","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Daily counts of actor handovers and interruptions Granger-cause changes in process throughput time in three public event logs.","lead":"This paper studies whether how people work in business processes, like handing tasks to each other or interrupting work, is linked to how long those processes take. It finds that handover behaviors are statistically associated with changes in process speed across three real-world datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Granger p-values are computed on the same data used to select lags, so the reported handover-to-TT significance may be selection artifacts, and the near-chance directionality evidence undercuts the causal claim.","rationale":"The reader's stated weakest assumption is the actor behavior classification (single-task assumption and timestamp precision), but the reader's rationale already flags the more severe statistical issues: lag selection and testing on the same data, no multiple-testing correction, and near-chance directionality. I agree with those concerns and view them as load-bearing because the paper's central causal claim is operationalized entirely through Granger F-tests. Even if behavior classification were perfect, the reported p-values would still be invalid due to selection on the same data; the near-chance support for Hypothesis 2 strengthens this concern by suggesting that many significant edges are noise. The paper has real independent support: public code and data, a clear methodology, and consistent qualitative patterns across logs. But the quantitative evidence for the headline claim is not robust as presented. Since the reader already issued a conditional verdict, my concern does not change the verdict; it sharpens the conditions under which acceptance should be granted, namely out-of-sample validation and corrected inference. A single chronological holdout test would settle whether the handover-to-TT effects survive; if they do not, the central claim should be downgraded. The actor-behavior classification concern remains secondary: it affects interpretation of which behavior drives the effect, but it is not the first thing that would invalidate the current results.","tokens_in":9418,"tokens_out":3837,"duration_ms":50988,"concrete_test":"Perform a chronological holdout: fit the Group Lasso lag-selection on the first 70% of each daily series, then run the Granger F-tests at the selected lags only on the held-out last 30%. Apply a multiple-testing correction across all behavior-KPI pairs (e.g., Benjamini-Hochberg). If HB→TT and HI→TT are not significant after correction in the held-out portion for at least BPIC2017 and BPIC2011, the paper's central claim of consistent causal handover influence would not survive. A permutation null that shuffles the behavioral series can serve as a robustness check on the held-out F-statistics.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 6: 'The most consistent finding is the strong causal influence of handovers, both HB and HI, on TT') rests on F-test p-values from Section 4.3. Those p-values are not trustworthy as reported because the lags are selected on the same data: Section 5.1 chooses the five most frequently selected lags via Group Lasso over all (lambda_g, lambda_1) combinations, and Section 5.2 then tests Granger causality at exactly those lags on the same time series. This is post-selection inference without a holdout or multiple-testing correction, so the nominal 0.05 threshold overstates the evidence. The problem is visible in the results: in BPIC2019, only two handover-to-TT p-values are below 0.05 (Table 2, l=21 p=.039; l=2 p=.031), which is what one would expect from multiple comparisons, while BPIC2011 reports many tiny p-values but Figure 2c shows all lags selected at near-equal frequency, so the selected lag set is not informative. Hypothesis 2 is claimed to be satisfied in all datasets, but the reported numbers are 55.55%, 50%, and 53.33%, i.e., near chance; this is internal evidence that many edges labeled significant are noise rather than directional causal effects. Because the strongest cross-log consistency claim is built on these p-values, the causal conclusion is not yet secured.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an actor-centric time series framework that links actor behaviors (continuation, interruption, handover idle, handover busy) to process performance (throughput time and outcome fractions). For three BPIC event logs, the authors build daily behavioral and KPI time series, select influential lags with Group Lasso, and test Granger causality between behavior and outcome series. They report that handovers, especially HB and HI, Granger-cause throughput time across datasets, and that the causal direction is predominantly one-way. The claimed contribution is a new integration of actor behavior decomposition with temporal causality analysis.","tokens_in":9667,"tokens_out":9162,"duration_ms":114328,"significance":"If the findings were statistically valid, this would be a useful and novel contribution: it extends prior actor-behavior decomposition work by adding a temporal/causal dimension, uses real-world logs from three different domains, and makes the implementation publicly available. The behavioral taxonomy is interpretable, and the cross-log application is a clear strength. However, the current inferential pipeline does not support the central causal claims: p-values are obtained after model selection on the same data, no multiple-testing correction is applied, and the reported directionality evidence is at chance level. The open code and concrete KPI construction are valuable, but the statistical validation needs to be reworked before the conclusions can be accepted.","major_comments":[{"comment":"The Granger p-values in Tables 1-3 are post-selection statistics. The five lags are chosen by Group Lasso over all (lambda_g, lambda_1) combinations on the full series, and the F-test is then run on those same series at those same lags. This invalidates the nominal 0.05 threshold. The near-chance pattern in BPIC2019 (only two handover-to-TT p-values below 0.05 among the ten HB/HI-to-TT tests, at l=21 p=0.039 and l=2 p=0.031) is exactly what multiple testing would produce. Moreover, Figure 2c shows roughly equal selection frequencies across all lags for BPIC2011, so the decision to use the first five lags {1,2,3,4,5} is arbitrary and does not identify an influential set. The causal conclusion in Section 6 is not secured until a holdout period or nested data-splitting is used and the selection step is accounted for.","section":"Section 4.3 and Section 5.1"},{"comment":"Hypothesis 2 is reported as satisfied at 55.55% (BPIC2017), 50% (BPIC2019), and 53.33% (BPIC2011), and Section 6 states that 'one-directionality was satisfied in all datasets.' These percentages are indistinguishable from chance and do not provide evidence for one-directional Granger causality. The denominator of 'significant cases' is never defined (per pair, per lag, or per graph edge), and no confidence interval or null model is given. The reverse-direction test should be reported for all tested pairs, not only those where the forward test was significant, and the asymmetry rate should be compared with the null expectation.","section":"Section 5.2 and Section 6"},{"comment":"The behavioral time series are event-date counts, while TT(d) is the average duration of cases started on day d. Consequently, X_{t-l} describes handovers occurring before the cases whose outcome is Y_t begin, so the causal statement that past handovers Granger-cause later changes in TT mixes event-date and case-start-date alignment. The authors should clarify whether the intended mechanism is a system-level workload effect; if the claim is per-case causality, the KPI series should be aligned to the completion date or to the relevant case cohort.","section":"Section 2.3 and Section 4.1"},{"comment":"The selected lag set for BPIC2019 is reported as {2,10,11,19,2}, which contains a duplicate and omits 21, yet Table 2 reports results at l=21. In addition, the text says 'I → %OP shows significance' for BPIC2019, but Table 2 uses %GR as the outcome. These inconsistencies make the results non-reproducible as printed and must be corrected.","section":"Section 5.2 and Table 2"},{"comment":"The handover classification assumes that each actor performs one task at a time and that tasks are atomic events; the discussion acknowledges this but provides no sensitivity analysis. Since the headline finding is that handovers, especially HB, affect throughput time, a systematic misclassification due to timestamp granularity or multi-tasking could create or mask the effect. A robustness check using coarsened timestamps or excluding uncertain transitions would significantly strengthen the claim.","section":"Section 2.2 and Section 6"}],"minor_comments":[{"comment":"The F-statistic uses L both as the number of lags in the numerator and in the degrees of freedom; please state explicitly whether each test uses a single lag or the full selected lag set, since the tables report p-values per lag.","section":"Section 4.3"},{"comment":"The roles of lambda_g and lambda_1 should be defined more explicitly in terms of the Group Lasso objective; the current text relies on a standard formulation without giving the objective.","section":"Section 5.1"},{"comment":"The header contains a stray period in 'HB → %OP .'; it should read 'HB → %OP'.","section":"Table 3"},{"comment":"The phrase 'one-directionality was satisfied in all datasets' conflicts with the percentages reported in Section 5.2; after reanalysis, a weaker statement such as 'reverse causality was detected in no more than chance proportion' would be appropriate.","section":"Section 6"},{"comment":"The caption says 'causality graph' but does not define the threshold or ranking used for the 'top 10 most significant influences'; a legend with edge weights or p-value labels would improve interpretability.","section":"Figure 4"},{"comment":"The phrase 'direct and measurable impacts' is stronger than what Granger causality can establish; 'temporal predictive relationships' would be more accurate.","section":"Abstract and Section 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution with openly available code and real-world logs, but the central claim rests on post-selection p-values and a near-chance directionality test. I would encourage the editor to treat this as major revision rather than rejection: the framework is useful, and a reanalysis with data splitting, multiple-testing correction, and clearer temporal alignment could make the findings credible. The BPIC2019 lag-set inconsistency and the %OP/%GR typo should be fixed as part of that revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2507.23037. First, the genuinely new bit is the integration: taking the actor behavior decomposition from Klijn et al. (continuations, interruptions, handover-idle, handover-busy), building daily time series of those counts, and running Granger causality against throughput time and outcome KPIs on three public BPIC logs. That combination is new, and the authors ship code and data, so the work is reproducible. Second, the headline causal claim — that handovers Granger-cause throughput time — is not yet supported by the statistics as reported. The lag selection and the significance tests are done on the same data, and the directionality results are close to coin-flip.\n\nWhat the paper does well: it is clearly written, the time series construction is sensible, the three logs cover different domains, and the authors are upfront about limitations (Granger causality cannot handle latent confounders, and the single-task actor assumption may misclassify handovers). The results tables are honest enough to show near-threshold p-values, e.g., BPIC2019 has only two handover-to-TT values below 0.05. The citation pattern is appropriate, crediting [11] for the behavior decomposition and [8] for earlier Granger use in process mining.\n\nThe soft spots are real. Group Lasso selects the five lags from the full data; then the F-test p-values are computed at exactly those lags on the same series. That is post-selection inference, so nominal 0.05 p-values overstate the evidence. On top of that, roughly forty tests per dataset run without any multiple-testing correction, and several of the \"significant\" effects are what you would expect by chance. The bigger problem: Hypothesis 2, the one-directionality check, is satisfied in 55.55%, 50%, and 53.33% of significant cases respectively. The Discussion says \"one-directionality was satisfied in all datasets.\" That is not what the numbers show. The central claim of a robust directional handover-to-TT effect rests on p-values that may be selection artifacts, plus a directionality rate at chance.\n\nNone of this makes the paper worthless. The underlying observation — handover frequency and throughput time move together at daily scale — is plausible and consistent across logs. But the causal framing needs to be dialed back, and the inference needs a holdout or a correction. Who gets value? Process mining researchers working on actor-centric and time-series analyses, and anyone wanting a concrete example of post-selection pitfalls. It deserves a serious referee, but with a clear request: redo lag selection with cross-validation or a split sample, correct for multiple tests, and rewrite the directionality claims. I would not cite the causal results until that is done.","headline":"A useful, reproducible integration of actor behavior decomposition with Granger causality, but the headline causal claim outruns the statistics because the lags are selected on the same data and the directionality evidence is near chance.","tokens_in":10244,"tokens_out":2539,"would_cite":false,"duration_ms":28409,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that actor handovers, idle or busy, Granger-cause later throughput time across three real-world event logs.","keywords":["Process performance","actor behavior","Granger causality","time series","process mining","handover","event logs","throughput time"],"falsifier":"A concrete test: take one of the logs, re-run the behavior classification under an alternative multi-tasking model (for example, treating overlapping event intervals for the same actor as parallel work), and repeat the Group Lasso and Granger pipeline; if the handover-to-throughput-time links do not survive the reclassification, the reported causal links are artifacts of the busy/idle labeling rather than a property of the process.","tokens_in":9200,"feed_emoji":"🔀","tokens_out":5027,"duration_ms":57049,"temperature":0.7,"pith_summary":"This paper tries to establish that the way actors hand work to each other has a direct, time-lagged effect on how long process cases take. By converting event-log behavior into daily counts of continuations, interruptions, idle handovers, and busy handovers, and then applying Granger causality with Group Lasso lag selection, the authors find that handover counts in the past predict later changes in average throughput time. The central finding is consistent across three real-world logs from healthcare, finance, and procurement: both idle and busy handovers causally influence throughput time, while other outcomes such as loan acceptance or goods receipt show weaker and more sporadic links. The paper's value is in showing that actor-centric, time-series methods can expose temporal dependencies that aggregate static process analyses miss.","feed_headline":"Handovers drive later process delays in three real logs","feed_subtitle":"Idle and busy handovers show a lagged causal link to throughput time in hospital, finance, and procurement processes.","key_machinery":"The load-bearing machinery is the four-way classification of each consecutive event pair inside a case: continuation (same actor, no other case in between), interruption (same actor but with intervening work on another case), handover idle (different actor, idle during the handover), and handover busy (different actor, already busy with another case). Daily counts of these four behaviors become behavioral time series aligned with daily average throughput time and outcome fractions. Granger causality then asks whether the past values of a behavioral series improve a linear autoregressive prediction of the KPI series, with the F-test comparing nested models; Group Lasso selects which lags enter as a group, keeping the model sparse and interpretable.","core_discovery":"The paper's central claim is that actor behavior, decomposed into continuations, interruptions, handover-idle, and handover-busy transitions, has direct and time-lagged effects on process performance, and that the most consistent and strongest of these effects is the influence of both types of handover on average throughput time. The authors assert this follows from Granger causality tests on daily time series built from three real event logs: past handover counts significantly improve prediction of future throughput time beyond the history of throughput time itself, at lags selected by Group Lasso and usually in one direction. They report in the discussion that 'the most consistent finding is the strong causal influence of handovers, both HB and HI, on TT,' and take this as evidence that actor-centric, time-series-based analysis can uncover temporal dependencies that static aggregate process mining misses.","pith_inferences":["My inference: because Granger causality only tests predictive improvement in linear models, the paper's phrase 'direct and measurable impacts' would need a randomized intervention, such as reassigning handovers between actors, to establish manipulation-level causality.","My inference: the daily aggregation window is a modeling choice; weekly or shift-level buckets might reveal that handover effects are much shorter-lived than the selected lags imply, which would change where to place monitoring dashboards.","My inference: the per-activity and per-user behavioral time series could feed directly into predictive monitoring models for remaining time, but multivariate Granger or vector autoregression would be needed to separate spurious cross-correlations among behavior types."],"forward_implications":["If the claim holds, daily handover counts become leading indicators: an operations team can forecast rising throughput time a few days before it appears in completed-case averages.","Reducing idle and busy handovers, or smoothing their daily peaks, would be expected to lower future average throughput time.","The same pipeline transfers to other event logs with actor and timestamp information, so hospital, finance, and procurement settings can be monitored with one common procedure.","Since most significant links are one-directional, monitoring systems can treat behavior-to-KPI directions as the primary warning direction rather than worrying equally about feedback loops.","A small set of lags (typically two to seven and occasionally fifteen to twenty-one days) captures most of the causal influence, so parsimonious models with a handful of lagged features are enough for prediction."],"supporting_citations":[{"why":"Defines Granger causality, the core statistical test used to decide whether past actor behavior improves prediction of process performance.","marker":"[7]"},{"why":"Supplies the actor-behavior decomposition into continuation, interruption, handover idle, and handover busy that the paper's time series are built from.","marker":"[11]"},{"why":"Provides the stationarity and lag-selection foundations, including the autoregressive modeling and testing assumptions behind the Granger procedure.","marker":"[15]"},{"why":"Demonstrates a prior application of Granger causality to business-process performance variation that the paper extends toward actor-level dynamics.","marker":"[8]"}],"fun_headline_variants":["Handovers causally predict later throughput time","Lagged handovers signal future process delays","Granger links handovers to delayed throughput","Handover counts precede process slowdowns"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire argument rests on the behavioral labels being correct: if actors perform multiple tasks at once, or timestamps are too coarse to say whether someone was busy between events, then the handover-busy and interruption counts are misclassified and the Granger links may be artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Handovers causally predict later throughput time","Lagged handovers signal future process delays","Granger links handovers to delayed throughput","Handover counts precede process slowdowns"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0004,"raw_usage":{"total_tokens":2059,"prompt_tokens":882,"completion_tokens":1177,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":1123}},"tokens_in":498,"tokens_out":1177,"duration_ms":10165,"temperature":1.0,"reasoning_tokens":1123,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:06:17.376086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: take one of the logs, re-run the behavior classification under an alternative multi-tasking model (for example, treating overlapping event intervals for the same actor as parallel work), and repeat the Group Lasso and Granger pipeline; if the handover-to-throughput-time links do not survive the reclassification, the reported causal links are artifacts of the busy/idle labeling rather than a property of the process.","supporting_citations":[{"cited_title":"Econometrica37(3), 424–438 (1969)","cited_arxiv_id":null,"evidence_quote":"Defines Granger causality, the core statistical test used to decide whether past actor behavior improves prediction of process performance."},{"cited_title":"In: 2024 6th International Conference on Process Mining (ICPM)","cited_arxiv_id":null,"evidence_quote":"Supplies the actor-behavior decomposition into continuation, interruption, handover idle, and handover busy that the paper's time series are built from."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates a prior application of Granger causality to business-process performance variation that the paper extends toward actor-level dynamics."}],"review_version":1}