Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single diffusion model segments observed actions and anticipates future actions in one pass.

desk verdict A solid, useful unification of TAS and LTA with a genuinely valuable evaluation-protocol correction; the main weak spot is that the anticipation-helps-segmentation claim rests on single-run ablations without variance estimates. read the letter →

arxiv 2412.04353 v1 pith:PUQAIJAN submitted 2024-12-05 cs.CV cs.LG

Dayoung Gong , Suha Kwak , Minsu Cho This is my paper · ORCID
classification cs.CVcs.LG
keywords temporalactionsegmentationlong-termanticipationdiffusionmodelunifiedanticipativemaskingjointlearningvideounderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Temporal action segmentation and long-term action anticipation are usually treated as separate problems with separate models. ActFusion claims that one diffusion-based model can do both at once: it classifies the visible frames of a video while predicting the invisible future frames, trained in a single process. The paper argues the two tasks help each other, so joint training improves both, and reports state-of-the-art results on standard benchmarks. If correct, this removes the need for task-specific architectures and makes a shared temporal understanding of actions feasible.

What carries the argument

Anticipative masking is the load-bearing mechanism: a binary mask hides a late part of the input frame sequence, and learnable mask tokens occupy the hidden positions so the encoder produces embeddings for both visible and invisible tokens, and the denoising decoder learns to reconstruct action labels for the whole sequence. Random masking, relation masking, and boundary masking are auxiliary strategies that make predictions robust to ambiguous or missing video parts. The architecture is an encoder-decoder transformer with dilated attention; the decoder follows the DDPM/DDIM diffusion framework, predicting clean action labels from noisy ones conditioned on the encoder embeddings.

What would settle it

Run the same ActFusion training on 50 Salads with several random seeds, with and without anticipative masking, and compare the spread of average TAS accuracy; if the difference between the two conditions is within the seed noise, the paper's core mutual-benefit claim fails. The same test applies to the LTA ablations, where removing anticipative masking drops the beta=0.5, alpha=0.2 score from 22.07 to 9.38.

Watch

Extended reading notes

Core claim

ActFusion's central claim is that action segmentation and long-term anticipation are two views of one generative task: reconstructing a complete action sequence from a partially observed one. The model is a diffusion decoder that iteratively denoises action-label sequences from Gaussian noise, conditioned on embeddings produced by an encoder fed with masked video features. During training, one of five masking strategies is randomly applied; the key one is anticipative masking, where a late portion of the sequence is replaced by learnable mask tokens so the model must infer the future. At inference, no masking is used for segmentation and anticipative masking for anticipation, so the identical weights handle both tasks. With this unified training, ActFusion reports state-of-the-art performance in both tasks on 50 Salads, Breakfast, and GTEA, and its ablations show segmentation loss improves anticipation and anticipative masking improves segmentation, i.e., bi-directional benefits. The paper also flags that some prior LTA evaluations use ground-truth video length at test time and demonstrates the method still leads under a rectified protocol.

Load-bearing premise

The paper's claim that anticipation helps segmentation rests on ablation gaps like 88.5 versus 87.2 in average TAS accuracy, reported without standard deviations; if run-to-run variance across random seeds is larger than that gap, the bi-directional benefit is not established.

Editorial extensions

If this is right

  • A single trained model can be deployed for both tasks, eliminating the need for per-task architectures or per-task training.
  • Joint training gives bi-directional gains: segmentation supervision improves anticipation, and anticipative masking improves segmentation accuracy and segment metrics.
  • Masked training with learnable tokens lets the model keep performing when input parts are missing or ambiguous, not just for the two benchmark tasks.
  • LTA can be evaluated without ground-truth prediction length, and ActFusion retains its lead in that more realistic setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If joint training works by learning temporal action structure, anticipative masking may serve as a general pretraining objective for other sequential action understanding tasks, such as early recognition or anomaly prediction, before fine-tuning.
  • The reported gains in anticipation from segmentation supervision suggest that stronger segmenters or boundary-supervised encoders could further improve long-horizon prediction; this is a testable hypothesis the paper does not run.
  • Since the paper's ablation numbers appear to lack variance estimates, a multi-seed replication is needed before the bi-directional benefit claim can be treated as quantitative rather than qualitative.
  • The rectified LTA evaluation suggests future benchmarks should ban ground-truth length and report prediction length as a function of observed frames only, which would change leaderboard comparisons.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ActFusion, a unified diffusion model that jointly addresses temporal action segmentation (TAS) and long-term action anticipation (LTA). The key idea is to train an encoder--decoder with two masking strategies: anticipative masking, which hides a late part of the video and replaces it with learnable tokens to force future prediction, and random masking, which improves robustness against ambiguous visible frames. A single model is trained once with all five masking types and, at inference, uses no masking for TAS and anticipative masking for LTA. The authors claim state-of-the-art performance on 50 Salads, Breakfast, and GTEA for TAS, competitive or superior LTA results on 50 Salads and Breakfast, and bi-directional benefits between the two tasks. They also identify a protocol flaw in prior LTA evaluations that use ground-truth video length to set the prediction length, and they re-evaluate several methods under a rectified protocol without ground-truth length information.

Significance. If the claims are substantiated, this is a meaningful contribution: it demonstrates that a single unified model can outperform task-specific models on both TAS and LTA, and it highlights a previously underexamined source of information leakage in long-term action anticipation benchmarks. The paper provides a thorough set of ablations (mask types, losses, conditioning features, position embeddings, inference steps) and releases algorithmic details in the appendix, which strengthen the empirical study. However, the central bi-directional-benefit claim currently rests on a single-run ablation whose effect size is comparable to typical seed noise, and the rectified-protocol comparison is confounded by a training/inference asymmetry. These issues need to be resolved before the reported conclusions can be regarded as fully supported.

major comments (3)
  1. [Section 5.2, Table 4a] The conclusion that anticipation helps segmentation is based on the comparison of rows (1) and (2) in Table 4a: removing anticipative masking lowers the 50 Salads TAS average from 88.5 to 87.2, a 1.3-point gap. No standard deviations, number of seeds, or significance tests are reported anywhere in the paper, for the main results or for any ablation. On 50 Salads (50 videos, 5 splits), run-to-run variance in F1 scores from random initialization and training noise is commonly on the order of 1--2 points, so the observed gap may be indistinguishable from noise. Since the abstract and conclusion claim bi-directional benefits, this measurement is load-bearing. The authors should report mean and standard deviation over multiple random seeds for the main results and the key ablations, or provide a rigorous justification (e.g., paired significance testing) that the 1.3-point gap is not attributable to seed noise.
  2. [Section 5.4, Table 6] The rectified-protocol comparison is asymmetric. ActFusion† is retrained with the modified anticipation masking strategy that uses prediction length rN_O, as stated in Section 5.4, while Cycle Cons.† and FUTR† are modified only at inference time to use the new prediction length while retaining their original training. The substantial performance drops for the baselines (e.g., FUTR† on 50 Salads, α=0.3, β=0.5, dropping from 15.26 to 7.48) may largely reflect that the models were not trained for the tested condition rather than an inherent inability to handle unknown prediction length. To support the claim that ActFusion outperforms baselines under the rectified protocol, the authors should retrain the baselines with a matching protocol (or, at minimum, report both inference-only and retrained baseline results) and explicitly disclose this asymmetry in the main text.
  3. [Section B (Algorithm 1) and Table S5] The training algorithm applies a signal scaling step, 'a_gt = (a_gt * 2 - 1) * scale', but no value for 'scale' is reported in the implementation details or the hyperparameter table. This parameter determines the range of the diffusion target and directly affects the loss scale and sampling behavior; without it, the reported numbers cannot be reproduced. Please specify the value of 'scale' for each dataset and, if it has a material effect, include an ablation over plausible values.
minor comments (4)
  1. [Section 4.3 and Section 5 (Implementation details)] The random-mask notation is inconsistent: the method defines a clip size Q and number of masked clips N_R, but Section 5 says 'we fix the size of patch w to 10' and Figure 3 mentions 'window size Q of each masked clip N_P to 10'; please harmonize the variable names and caption wording.
  2. [Table 4b] The caption states that results are reported for observation ratio α=0.3, but the table header shows β(α=0.2); the caption and table should be aligned.
  3. [Appendix E, 'Loss ablations'] The text refers to 'Table R4' when presenting loss ablation results, but the corresponding table is labeled Table S3; please correct the cross-reference.
  4. [Table 2] On 50 Salads with α=0.2 and β=0.1, ActFusion achieves 39.55, which ties with FUTR's 39.55; the abstract's wording 'outperforming task-specific models' is slightly too strong for this cell, although the overall average remains superior.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the unified-model claims and bidirectional-benefit ablations are empirical and do not reduce to their inputs.

full rationale

The paper's derivation chain is empirical and self-contained. The central claims—unified TAS/LTA via anticipative masking and bidirectional benefit—are supported by external benchmark comparisons (50 Salads, Breakfast, GTEA) and by internal ablations that remove one component at a time (Table 3 for segmentation-to-anticipation, Table 4a for anticipation-to-segmentation). The anticipative mask M_A is defined by M_A^i = 1(i <= N_O), and LTA evaluation predicts labels on exactly the masked future interval; this is a genuine prediction target from ground-truth labels, not a quantity fitted from the reported result. No parameter is fitted to the target metrics; the diffusion loss L = 1/2 ||f(x_s,s) - x_0||^2 plus CE/smoothing/boundary losses is a standard training objective. Self-citations to FUTR and KARI appear only as baselines or as prior empirical observations; the paper re-derives the segmentation-helps-anticipation effect through its own loss ablations. Masking strategies M_S and M_B are adopted from DiffAct (an external work), and the architecture follows ASFormer/DiffAct; neither smuggles the paper's conclusion. The statistical fragility of the 1.3-point TAS gain (88.5 vs 87.2 in Table 4a) and the admitted slight drop in frame-wise accuracy (Appendix I) are correctness and evidence concerns about variance, not circularity: they do not show that any prediction is equivalent to its inputs by construction. The Appendix text/table label mismatch (Table R4 vs S4) is an editorial inconsistency, not a circular reduction. Therefore no circular step is identifiable with the required quoted reduction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no physical or conceptual entities beyond model components. The main inventions are the anticipative masking strategy and the rectified evaluation protocol, both of which are algorithmic rather than ontological. All other ingredients are standard diffusion models, ASFormer, and pre-trained I3D features.

free parameters (5)
  • scale = not specified
    Signal scaling factor in Algorithm 1 is used but its value is not reported anywhere in the paper, leaving a reproducibility gap.
  • r = 4
    Relative prediction length in the rectified LTA protocol (Section 5.4); chosen by hand, with no sweep or justification.
  • N_R = 25 for 50 Salads, 10 for Breakfast, 20 for GTEA
    Number of randomly masked clips in random masking; dataset-specific hand-picked values reported in Implementation details.
  • Q = 10
    Clip size for random masking; fixed without an ablation in the main text (only N_R is swept in Figure 3).
  • alpha (training) = {0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8}
    Observation ratios for anticipative masking during training; the paper does not study sensitivity to this choice.
assumptions (4)
  • domain assumption Action label sequences can be modeled as continuous vectors under Gaussian noise
    The diffusion process in Eq. 1 adds Gaussian noise to one-hot action labels; this continuous relaxation is assumed to be a valid training target, following DiffAct.
  • domain assumption ASFormer with relative position bias is a sufficiently strong backbone for both tasks
    The architecture is adopted from DiffAct/ASFormer with no comparison to alternative backbones, so the method's success is conditional on this backbone choice.
  • domain assumption Pre-trained I3D features contain enough information for both segmentation and anticipation
    All models use I3D features from prior work; no feature-level analysis or alternative features are considered.
  • standard math Standard diffusion mathematics (forward process, DDIM update) is correct
    Equations 13-20 and the DDIM update rule are taken from established diffusion literature and are not re-derived here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation." pith.science (2026). https://pith.science/paper/PUQAIJAN

@misc{pith2026241204353,
  author       = {Pith},
  title        = {Pith review of: ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PUQAIJAN}},
  note         = {Machine review of arXiv:2412.04353}
}
read the original abstract

Temporal action segmentation and long-term action anticipation are two popular vision tasks for the temporal analysis of actions in videos. Despite apparent relevance and potential complementarity, these two problems have been investigated as separate and distinct tasks. In this work, we tackle these two problems, action segmentation and action anticipation, jointly using a unified diffusion model dubbed ActFusion. The key idea to unification is to train the model to effectively handle both visible and invisible parts of the sequence in an integrated manner; the visible part is for temporal segmentation, and the invisible part is for future anticipation. To this end, we introduce a new anticipative masking strategy during training in which a late part of the video frames is masked as invisible, and learnable tokens replace these frames to learn to predict the invisible future. Experimental results demonstrate the bi-directional benefits between action segmentation and anticipation. ActFusion achieves the state-of-the-art performance across the standard benchmarks of 50 Salads, Breakfast, and GTEA, outperforming task-specific models in both of the two tasks with a single unified model through joint learning.

Figures

Figures reproduced from arXiv: 2412.04353 by the authors.

Figure 1
Figure 1. Task-specific models vs. ActFusion (ours). (a) Conventional task-specific models for TAS and LTA. (b) Our unified model ActFusion to address both tasks. (c) Performance comparison across tasks. Tasks-specific models such as DiffAct [43] for TAS and FUTR [25] for LTA exhibits poor performance on cross-task evaluations. ActFusion outperforms task-specific models on both TAS and LTA, including TempAgg [52], which train… view at source ↗
Figure 2
Figure 2. Overall pipeline of ActFusion. During training, we randomly select one of five masking strategies and apply it to input video frames F, replacing masked regions with learnable tokens to obtain masked features F ′ . These features are processed by the encoder g to produce visual embeddings E, which condition the decoder h to denoise action labels from As to Aˆ0 at time-step s. For inference, we use different masking … view at source ↗
Figure 4
Figure 4. Inference steps of diffusion process The number of masked clips N R in random masking. To determine the number of masked clips N R in random masking, we conduct experiments by adjusting N R from 10 to 100 while fixing the window size Q of each masked clip NP to 10 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Qualitative results [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-level and Multi-modal Action Anticipation

    cs.CV 2025-06 reject novelty 4.0 of 10

    A multi-modal action anticipation model with a clustering-based fine-grained text generator and temporal consistency loss reports state-of-the-art gains, though key claims are not fully supported by the provided evidence.

Reference graph

Works this paper leans on

67 extracted references · 60 canonical work pages · cited by 1 Pith paper

  1. [1]

    Abu Farha and J

    Y . Abu Farha and J. Gall. Uncertainty-aware anticipation of activities. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019. 2, 7

    work page 2019

  2. [2]

    Abu Farha, A

    Y . Abu Farha, A. Richard, and J. Gall. When will you do what?-anticipating temporal occur- rences of activities. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5343–5352, 2018. 1, 2, 3, 7, 9

    work page 2018

  3. [3]

    Ahn and D

    H. Ahn and D. Lee. Refining action segmentation with hierarchical video representations. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 16302–16310, 2021. 1, 2, 6

    work page 2021

  4. [4]

    Aziere and S

    N. Aziere and S. Todorovic. Multistage temporal convolution transformer for action segmenta- tion. Image and Vision Computing, 128:104567, 2022. 6

    work page 2022

  5. [5]

    Bahrami, G

    E. Bahrami, G. Francesca, and J. Gall. How much temporal long-term context is needed for action segmentation? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10351–10361, 2023. 2, 6

    work page 2023

  6. [6]

    H. Bao, L. Dong, S. Piao, and F. Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.08254, 2021. 2

    arXiv 2021

  7. [7]

    Baranchuk, A

    D. Baranchuk, A. V oynov, I. Rubachev, V . Khrulkov, and A. Babenko. Label-efficient semantic segmentation with diffusion models. In International Conference on Learning Representations,

    work page

  8. [8]

    Behrmann, S

    N. Behrmann, S. A. Golestaneh, Z. Kolter, J. Gall, and M. Noroozi. Unified fully and times- tamp supervised temporal action segmentation via sequence to sequence translation. In Proc. European Conference on Computer Vision (ECCV), pages 52–68. Springer, 2022. 1, 2, 6, 21

    work page 2022

Show all 67 references
  1. [9]

    Carreira and A

    J. Carreira and A. Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6299–6308, 2017. 7

    2017

  2. [10]

    L. Chen, M. Li, Y . Duan, J. Zhou, and J. Lu. Uncertainty-aware representation learning for action segmentation. In IJCAI, volume 2, page 6, 2022. 6

    2022

  3. [11]

    M.-H. Chen, B. Li, Y . Bao, and G. AlRegib. Action segmentation with mixed temporal domain adaptation. In Proc. IEEE Winter Conference on Applications of Computer Vision (WACV), pages 605–614, 2020. 6

    2020

  4. [12]

    M.-H. Chen, B. Li, Y . Bao, G. AlRegib, and Z. Kira. Action segmentation with joint self- supervised temporal domain adaptation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9454–9463, 2020. 1, 2, 6

    2020

  5. [13]

    S. Chen, P. Sun, Y . Song, and P. Luo. Diffusiondet: Diffusion model for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19830– 19843, 2023. 3

    2023

  6. [14]

    T. Chen, R. ZHANG, and G. Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. In The Eleventh International Conference on Learning Repre- sentations, 2022. 3

    2022

  7. [15]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1), 2019. 2, 18, 19

    2019

  8. [16]

    G. Ding, F. Sener, and A. Yao. Temporal action segmentation: An analysis of modern techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 2

    2023

  9. [17]

    Du and Q

    Z. Du and Q. Wang. Dilated transformer with feature aggregation module for action segmenta- tion. Neural Processing Letters, pages 1–17, 2022. 6

    2022

  10. [18]

    Fan, Y .-C

    W.-C. Fan, Y .-C. Chen, D. Chen, Y . Cheng, L. Yuan, and Y .-C. F. Wang. Frido: Feature pyramid diffusion for complex scene image synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 579–587, 2023. 3

    2023

  11. [19]

    Y . A. Farha and J. Gall. Ms-tcn: Multi-stage temporal convolutional network for action segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3575–3584, 2019. 1, 2, 6, 7, 17, 19 11

    2019

  12. [20]

    Y . A. Farha, Q. Ke, B. Schiele, and J. Gall. Long-Term Anticipation of Activities with Cycle Consistency. In Proc. German Conference on Pattern Recognition (GCPR). Springer, 2020. 2, 3, 7, 9, 10, 17, 20

    2020

  13. [21]

    Fathi, X

    A. Fathi, X. Ren, and J. M. Rehg. Learning to recognize objects in egocentric activities. In CVPR 2011, pages 3281–3288. IEEE, 2011. 6, 7, 20

    2011

  14. [22]

    Feichtenhofer, Y

    C. Feichtenhofer, Y . Li, K. He, et al. Masked autoencoders as spatiotemporal learners.Advances in neural information processing systems, 35:35946–35958, 2022. 19, 21

    2022

  15. [23]

    S.-H. Gao, Q. Han, Z.-Y . Li, P. Peng, L. Wang, and M.-M. Cheng. Global2local: Efficient structure search for video action segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 16805–16814, 2021. 2, 6

    2021

  16. [24]

    D. Gong, J. Lee, D. Jung, S. Kwak, and M. Cho. Activity grammars for temporal action segmentation. Advances in Neural Information Processing Systems, 36, 2024. 2, 6

    2024

  17. [25]

    D. Gong, J. Lee, M. Kim, S. J. Ha, and M. Cho. Future transformer for long-term action anticipation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3052–3061, 2022. 1, 2, 3, 7, 9, 10, 17, 20, 21

    2022

  18. [26]

    Gupta, J

    A. Gupta, J. Liu, L. Bo, A. K. Roy-Chowdhury, and T. Mei. A-act: Action anticipation through cycle transformations. arXiv preprint arXiv:2204.00942, 2022. 2, 7

    2022 arXiv

  19. [27]

    Harvey, S

    W. Harvey, S. Naderiparizi, V . Masrani, C. Weilbach, and F. Wood. Flexible diffusion modeling of long videos. Advances in Neural Information Processing Systems, 35:27953–27965, 2022. 3

    2022

  20. [28]

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022. 2, 8, 19, 21

    2022

  21. [29]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pages 6840–6851,

  22. [30]

    Huang, Y

    Y . Huang, Y . Sugano, and Y . Sato. Improving action segmentation via graph-based temporal reasoning. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14024–14034, 2020. 2, 6

    2020

  23. [31]

    Ishihara, G

    K. Ishihara, G. Nakano, and T. Inoshita. Mcfm: Mutual cross fusion module for intermediate fusion-based action segmentation. In 2022 IEEE International Conference on Image Processing (ICIP), pages 1701–1705. IEEE, 2022. 6

    2022

  24. [32]

    Ishikawa, S

    Y . Ishikawa, S. Kasai, Y . Aoki, and H. Kataoka. Alleviating over-segmentation errors by detecting action boundaries. In Proc. IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2322–2331, 2021. 1, 2, 6

    2021

  25. [33]

    Karaman, L

    S. Karaman, L. Seidenari, and A. Del Bimbo. Fast saliency based pooling of fisher encoded dense trajectories. In ECCV THUMOS Workshop, volume 1, page 5, 2014. 2

    2014

  26. [34]

    Q. Ke, M. Fritz, and B. Schiele. Time-conditioned action anticipation in one shot. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9925–9934,

  27. [35]

    Kim and E

    G.-h. Kim and E. Kim. Stacked encoder–decoder transformer with boundary smoothing for action segmentation. Electronics Letters, 58(25):972–974, 2022. 6

    2022

  28. [36]

    Kuehne, A

    H. Kuehne, A. Arslan, and T. Serre. The language of actions: Recovering the syntax and semantics of goal-directed human activities. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 780–787, 2014. 2, 6, 7, 20

    2014

  29. [37]

    Kuehne, A

    H. Kuehne, A. Richard, and J. Gall. Weakly supervised learning of actions from transcripts. Computer Vision and Image Understanding, 163:78–89, 2017. 2, 21

    2017

  30. [38]

    C. Lea, M. D. Flynn, R. Vidal, A. Reiter, and G. D. Hager. Temporal convolutional networks for action segmentation and detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 156–165, 2017. 1, 2

    2017

  31. [39]

    J. Li, P. Lei, and S. Todorovic. Weakly supervised energy-based learning for action segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October

  32. [40]

    M. Li, L. Chen, Y . Duan, Z. Hu, J. Feng, J. Zhou, and J. Lu. Bridge-prompt: Towards ordinal action understanding in instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19880–19889, 2022. 6

    2022

  33. [41]

    S.-J. Li, Y . AbuFarha, Y . Liu, M.-M. Cheng, and J. Gall. Ms-tcn++: Multi-stage temporal convolutional network for action segmentation. IEEE transactions on pattern analysis and machine intelligence, 2020. 6

    2020

  34. [42]

    X. Li, W. Chu, Y . Wu, W. Yuan, F. Liu, Q. Zhang, F. Li, H. Feng, E. Ding, and J. Wang. Videogen: A reference-guided latent diffusion approach for high definition text-to-video generation. arXiv preprint arXiv:2309.00398, 2023. 3

    2023 arXiv

  35. [43]

    D. Liu, Q. Li, A.-D. Dinh, T. Jiang, M. Shah, and C. Xu. Diffusion action segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10139– 10149, 2023. 1, 2, 3, 5, 6, 7, 17, 19, 20, 21

    2023

  36. [44]

    Nawhal, A

    M. Nawhal, A. A. Jyothi, and G. Mori. Rethinking learning approaches for long-term action anticipation. In European Conference on Computer Vision, pages 558–576. Springer, 2022. 2, 3, 7

    2022

  37. [45]

    Nawhal and G

    M. Nawhal and G. Mori. Activity graph transformer for temporal action localization. arXiv preprint arXiv:2101.08540, 2021. 1

    2021 arXiv

  38. [46]

    J. Park, D. Kim, S. Huh, and S. Jo. Maximization and restoration: Action segmentation through dilation passing and temporal reconstruction. Pattern Recognition, 129:108764, 2022. 6

    2022

  39. [47]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 21

    2019

  40. [48]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 17, 18

    2020

  41. [49]

    Richard, H

    A. Richard, H. Kuehne, and J. Gall. Weakly supervised action learning with rnn based fine- to-coarse modeling. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 754–763, 2017. 7, 21

    2017

  42. [50]

    Rohrbach, S

    M. Rohrbach, S. Amin, M. Andriluka, and B. Schiele. A database for fine grained activity detection of cooking activities. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1194–1201. IEEE, 2012. 2

    2012

  43. [51]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  44. [52]

    Sener, D

    F. Sener, D. Singhania, and A. Yao. Temporal aggregate representations for long-range video understanding. In Proc. European Conference on Computer Vision (ECCV), pages 154–171. Springer, 2020. 1, 2, 3, 6, 7, 17, 20

    2020

  45. [53]

    P. Shaw, J. Uszkoreit, and A. Vaswani. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155, 2018. 18

    2018 arXiv

  46. [54]

    J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020. 3, 4, 7, 15

    2020

  47. [55]

    Song and S

    Y . Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019. 3

    2019

  48. [56]

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020. 3

    2020

  49. [57]

    Souri, Y

    Y . Souri, Y . A. Farha, F. Despinoy, G. Francesca, and J. Gall. Fifa: Fast inference approximation for action segmentation. In Pattern Recognition: 43rd DAGM German Conference, DAGM GCPR 2021, Bonn, Germany, September 28–October 1, 2021, Proceedings, pages 282–296. Springer, 2022. 2

    2021

  50. [58]

    Stein and S

    S. Stein and S. J. McKenna. Combining embedded accelerometers with computer vision for recognizing food preparation activities. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, pages 729–738, 2013. 6, 7, 20 13

    2013

  51. [59]

    Z. Tong, Y . Song, J. Wang, and L. Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022. 2, 5

    2022

  52. [60]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Proc. Neural Information Processing Systems (NeurIPS), 30, 2017. 5

    2017

  53. [61]

    Z. Wang, Z. Gao, L. Wang, Z. Li, and G. Wu. Boundary-aware cascade networks for temporal action segmentation. In Proc. European Conference on Computer Vision (ECCV), pages 34–51. Springer, 2020. 1, 6

    2020

  54. [62]

    C. Wei, K. Mangalam, P.-Y . Huang, Y . Li, H. Fan, H. Xu, H. Wang, C. Xie, A. Yuille, and C. Feichtenhofer. Diffusion models as masked autoencoders. arXiv preprint arXiv:2304.03283,

    arXiv

  55. [63]

    Z. Xu, Y . Rawat, Y . Wong, M. S. Kankanhalli, and M. Shah. Don’t pour cereal into coffee: Differentiable temporal logic for temporal action segmentation.Advances in Neural Information Processing Systems, 35:14890–14903, 2022. 1, 2, 6

    2022

  56. [64]

    R. Yang, P. Srivastava, and S. Mandt. Diffusion probabilistic modeling for video generation. Entropy, 25(10):1469, 2023. 3

    2023

  57. [65]

    F. Yi, H. Wen, and T. Jiang. Asformer: Transformer for action segmentation. In Proc. British Machine Vision Conference (BMVC), 2021. 1, 2, 5, 6, 7, 17, 19

    2021

  58. [66]

    Zhang, C

    C. Zhang, C. Fu, S. Wang, N. Agarwal, K. Lee, C. Choi, and C. Sun. Object-centric video representation for long-term action anticipation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6751–6761, 2024. 3, 7

    2024

  59. [67]

    "" f: video features [B, T, C] a_gt: ground-truth action labels [B, T, K] # B: batch # T: number of frames # C: video feature dimension # K: number of action classes

    Z. Zhong, M. Martin, M. V oit, J. Gall, and J. Beyerer. A survey on deep learning techniques for action anticipation. arXiv preprint arXiv:2309.17257, 2023. 7 14 Appendix In this appendix, we offer detailed descriptions and additional results, which are omitted in the main pap...

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.