REVIEW 4 major objections 5 minor 38 references
Generalized Uncertainty-Based Evidential Fusion with Hybrid Multi-Head Attention for Weak-Supervised Temporal Action Localization
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that hybrid multi-head attention plus generalized uncertainty-based evidential fusion outperforms prior weakly supervised methods on THUMOS14, reaching 74.5 mAP at IoU 0.1 and 48.9 average mAP.
desk verdict SOTA WS-TAL numbers undermined by an evidential fusion step that fuses evidence with its own attention-weighted copy, violating Dempster's independence assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the generalized evidential fusion rule in Eq. 10: for each snippet, the original evidence $e_s^1$ is combined with the attention-reweighted evidence $e_s^2 = e_s^1 \cdot A$ through Dempster's combination rule, producing a fused belief assignment and a multiplet uncertainty $m_s(\{\Theta\})$. The word 'generalized' refers to replacing the scalar uncertainty mass of traditional evidential deep learning by a multiplet $\Theta$ that contains all class singletons, so uncertainty and belief are fused as one evidence structure. This fused uncertainty is then sorted and inserted into the loss $L_{\mu ef}$, making the attention score and the classifier's background probability complementary, while the evidential loss $L_{hge}$ reweights training snippets by their uncertainty. The machinery therefore turns attention-weighted feature enhancement into a full evidence-fusion loop whose outputs directly shape the training objective.
What would settle it
Retrain with Eq. 10 replaced by the arithmetic average of $e_s^1$ and $e_s^2$: if mAP at IoU 0.5 stays close to the reported 42.1, the Dempster combination itself is not responsible for the gain, whereas a large drop would confirm that the specific fusion formula matters.
Extended reading notes
Core claim
The paper's central claim is that action-background ambiguity can be reduced by treating snippet-level classifier outputs as evidential masses instead of plain probabilities, and by fusing two pieces of evidence per snippet: the original evidence $e_s^1$ and the attention-reweighted evidence $e_s^2 = e_s^1 \cdot A$. The GUEF module combines these with the standard Dempster-Shafer rule for combining independent evidence, after replacing the single uncertainty mass with a multiplet $\Theta = \{p_1,\dots,p_N\}$, so that belief values and uncertainty are carried by one unified piece of evidence. The resulting uncertainty measures are used in a loss that forces the attention score and the background probability to be complementary, and in an evidential loss that down-weights high-uncertainty snippets. On THUMOS14, with HMHA supplying the attention weights and feature alignment, the paper reports state-of-the-art weakly supervised results, including 74.5 mAP at IoU 0.1 and 48.9 average mAP across IoU 0.1-0.7.
Load-bearing premise
The load-bearing premise is that the snippet evidence and its attention-weighted copy can be treated as two independent sources of information under the standard evidence-combination rule, even though the second copy is just the first multiplied by a learned attention weight; if that independence fails, the fusion formula in Eq. 10 lacks its theoretical justification.
Editorial extensions
If this is right
- The GUEF module is portable: it can be attached to other WS-TAL pipelines that already produce a class activation sequence and an attention score, without changing their supervision.
- The uncertainty measure from the fusion gives the model a principled way to mask out background snippets during training, potentially making top-k aggregation more robust.
- HMHA's feature alignment suggests that frozen I3D RGB and optical-flow features can be cheaply adapted to a video task before classification, a step that could transfer to weakly supervised video tasks beyond localization.
- At the strictest IoU 0.7, the fully supervised BSN baseline still reports 20.0 versus the proposed method's 15.5, so the claimed advance is strongest at coarse localization rather than exact boundary estimation.
Reading between the lines
- Because Eq. 10 combines $e_s^2 = e_s^1 \cdot A$, which is a deterministic scaling of $e_s^1$, the two 'evidence sources' are not independent in the sense Dempster's rule requires; testing the fusion against a simple average of the two evidences would isolate whether the gain comes from the combination rule or simply from the extra attention-style reweighting.
- The paper evaluates only THUMOS14; a natural probe is ActivityNet 1.3, where action instances are longer and background segments differ, which would reveal whether the uncertainty-based background suppression generalizes beyond this dataset's statistics.
- A residual possibility is that HMHA's feature alignment and the two new loss terms, rather than the evidential combination itself, drive the reported improvement; an ablation that keeps the losses but replaces the Dempster fusion with a convex combination would separate these contributions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript addresses weakly supervised temporal action localization (WS-TAL) by proposing two modules: a hybrid multi-head attention (HMHA) module that reweights RGB and optical-flow features through shared attention and a filtering module, and a generalized uncertainty-based evidential fusion (GUEF) module that combines the original snippet evidence with an attention-weighted copy under Dempster's combination rule. The authors add auxiliary losses intended to align attention scores with background probabilities and to down-weight uncertain snippets. Experiments on THUMOS14 report state-of-the-art performance (74.5 mAP@0.1, 48.9 average mAP@0.1-0.7), and a 2x2 ablation indicates that both modules contribute to the final result.
Significance. If the proposed modules work as described, the main contribution is a novel application of evidential uncertainty to suppress background snippets in weakly supervised temporal localization, which is a meaningful direction. The paper releases code and provides a comparison against ten weakly supervised baselines on THUMOS14, which is a useful reference point for the community. The theoretical development of the evidential fusion is, however, under-specified, and the empirical support is limited to a single dataset with no variance estimates or significance tests. The reported gain over the strongest compared weakly supervised method is about 1.9 points in average mAP (0.1-0.7), which is positive but modest; if the theoretical issues are resolved and a more thorough experimental protocol is added, the contribution could be solid.
major comments (4)
- [§2.2, Eq. (10)] The fusion operation in Eq. (10) is written as a Dempster combination of e1_s and e2_s = e1_s · A. Dempster's rule in Eq. (8) is derived under the assumption that the two evidence sources are independent, but e2_s is a deterministic scalar reweighting of e1_s produced from the same features; when A = 1, the two sources are identical. The conflict term and normalization in Eq. (8) therefore do not have a probabilistic interpretation for this pair. The authors should either supply a combination rule that is valid for dependent evidence or explicitly recast GUEF as a heuristic attention-based reweighting; as presented, the claim that GUEF 'fuses snippet-level evidences' is not supported by the stated theory.
- [§2.3, Eq. (16)] The final training objective is defined as L = L_cla + λ1 L_µgl + λ2 L_hge, but the only uncertainty-related loss defined in §2.3 is L_µef in Eq. (13). The relationship between L_µgl and L_µef is never stated. If they are the same quantity, the notation should be unified; otherwise the actual training objective is underspecified, making the reported results impossible to reproduce independently.
- [§2.2, Eq. (7)] The definition of the multiplet Θ is internally inconsistent: Eq. (7) writes Θ = {p1, ..., pN} = p_{N+1}. If Θ is the full frame of discernment {p1, ..., pN}, then Eq. (8) is the standard Dempster combination for Bayesian mass functions plus full-frame mass, and the claimed generalization over traditional evidential deep learning is not substantiated. If instead Θ is treated as a new singleton p_{N+1}, then {pk} ∩ Θ = ∅, and the product terms in Eq. (8) do not follow from Dempster's rule. The frame of discernment and focal elements need to be stated explicitly.
- [§2.2, Eq. (10), surrounding text] The paper never defines how e1_s is obtained from the classifier output or from the features, nor does it define the normalization S used in e1_s/S. Since e1_s is a W × T matrix and S in Eq. (6) is a scalar total evidence per snippet, the expression Concat[e1_s/S, m1({Θ})] is dimensionally unclear. Without a precise definition of e1_s and S, the central GUEF operation cannot be reproduced from the manuscript.
minor comments (5)
- [Table 1] The table header does not clearly separate the three average columns (AVG 0.1-0.5, AVG 0.3-0.7, AVG 0.1-0.7), and several entries are marked with '-' without a footnote explaining whether those values were not reported or not applicable.
- [Eq. (11)] The notation for the top-L aggregation is garbled: the line 'p = 1/L ξ=arg max_ξ P_{t in ξ} A_t' does not parse as a standard top-L selection. Please rewrite this with a clear argmax over subsets of size L or a per-snippet attention threshold.
- [§3.4] The statement that removing GUEF results in 'significant performance degradation' is not supported by any significance test; the average-mAP difference in Table 2 is about 1.6 points. If possible, report mean and standard deviation over multiple runs.
- [§3.2] The sentence 'the maximum number of sample snippets on THUMOS14 is set to 320' is ambiguous: it is unclear whether this is the number of snippets per video, a training truncation length, or a hyperparameter of the top-L selection.
- [Eq. (13)] There are mismatched parentheses and stray braces in the definition of L_µef, e.g., 'tanh(σ(h)φ(ms({Θ)})+1)' and 'φ(ms({Θ)}'. These should be corrected for readability.
Circularity Check
No circularity: the benchmark is held-out and the fusion equations are explicit model definitions, not predictions equivalent to their inputs.
full rationale
The paper's central empirical claim is assessed on held-out THUMOS14 test videos, and the ablation study removes modules to measure their contribution, so the reported improvement is not an artifact of fitting the test set or renaming a fitted parameter as a prediction. The GUEF equations (Eqs. 6-10) are explicit construction rules rather than derivations claiming to produce a result from first principles, so there is no step where a claimed output is equal by construction to its input. The evidential machinery is attributed to standard, non-self sources (Jøsang [21], Yager [23], Dempster [25], Tong et al. [26], Sensoy et al. [32]), and the paper's self-citations appear only in the reference list without being load-bearing in the main derivation. The concern that Eq. 10 fuses e1_s with e2_s = e1_s * A under an implicit independence assumption is a validity or correctness issue for Dempster combination, but it is not circularity: the reported performance is not defined as the input of the fusion, and the method's behavior is still externally evaluated against held-out ground-truth annotations. Therefore, no circular step meeting the evidence bar is present.
Assumptions & free parameters
free parameters (3)
- loss weights lambda1, lambda2 =
0.8, 1.0
- amplitude Delta =
0.7
- top-L snippet count =
not reported (maximum 320 snippets)
assumptions (3)
- domain assumption Dempster combination in Eq. 10 requires independence of the two evidence sources e1_s and e2_s.
- domain assumption Attention score A and background class probability z_{t,T+1} are complementary, so |1 - A_t - z_{t,T+1}| is a meaningful regularizer.
- ad hoc to paper The relationship alpha_j = e_j + 1 and the evidential loss from TEDL remain valid after replacing uncertainty mass with the multiplet Theta.
invented entities (1)
-
Multiplet Theta = {p1,...,pN} as a combined evidence mass replacing the EDL uncertainty U
Cite this review
Pith. "Pith review of Generalized Uncertainty-Based Evidential Fusion with Hybrid Multi-Head Attention for Weak-Supervised Temporal Action Localization." pith.science (2026). https://pith.science/paper/GHAJ4LGM
@misc{pith2026241219418,
author = {Pith},
title = {Pith review of: Generalized Uncertainty-Based Evidential Fusion with Hybrid Multi-Head Attention for Weak-Supervised Temporal Action Localization},
year = {2026},
howpublished = {\url{https://pith.science/paper/GHAJ4LGM}},
note = {Machine review of arXiv:2412.19418}
}
read the original abstract
Weakly supervised temporal action localization (WS-TAL) is a task of targeting at localizing complete action instances and categorizing them with video-level labels. Action-background ambiguity, primarily caused by background noise resulting from aggregation and intra-action variation, is a significant challenge for existing WS-TAL methods. In this paper, we introduce a hybrid multi-head attention (HMHA) module and generalized uncertainty-based evidential fusion (GUEF) module to address the problem. The proposed HMHA effectively enhances RGB and optical flow features by filtering redundant information and adjusting their feature distribution to better align with the WS-TAL task. Additionally, the proposed GUEF adaptively eliminates the interference of background noise by fusing snippet-level evidences to refine uncertainty measurement and select superior foreground feature information, which enables the model to concentrate on integral action instances to achieve better action localization and classification performance. Experimental results conducted on the THUMOS14 dataset demonstrate that our method outperforms state-of-the-art methods. Our code is available in \url{https://github.com/heyuanpengpku/GUEF/tree/main}.
Reference graph
Works this paper leans on
-
[1]
INTRODUCTION As one of the most essential tasks of video understanding, Temporal Action localization (TAL) targets at accurately positioning action boundaries, including start and end times- tamps of action instances in untrimmed video and sorting out them. Numerous early studies [1, 2, 3] focus on utiliz- ing fully-supervised methods to solve the task an...
work page Pith review arXiv 2024
-
[2]
METHODOLOGY 2.1. Hybrid Multi-Head Attention A Hybrid Multi-Head Attention (HMHA), which consists of two sharing multi-head attention modules and a filtering mod- ule with an attention-like information interaction mechanism, is applied to enable the two modalities’ weight distribution to approach each other. Following the existing methods, the untrimmed v...
-
[3]
EXPERIMENTS AND RESULTS 3.1. Datasets and Metrics We conduct a large amount of experiments to evaluate the proposed method on THUMOS14 [34] dataset. THUMOS14 is composed of 200 validation videos and 213 testing videos with 20 action classes. Besides, The mean Average Precision (mAP) with different Intersection-over-Union (IoU) thresh- olds, which is regar...
-
[4]
CONCLUSION In this paper, we propose a generalized uncertainty-based evidential fusion and hybrid multi-head attention module, which effectively eliminates action-background ambiguity and filters redundant information from pre-trained features to enable the model to focus on foreground snippets, conse- quently improving performance. Experimental results o...
-
[5]
Temporal action detec- tion with structured segment networks,
Yue Zhao, Yuanjun Xiong, Limin Wang, Zhirong Wu, Xiaoou Tang, and Dahua Lin, “Temporal action detec- tion with structured segment networks,” Int. J. Comput. Vis., vol. 128, no. 1, pp. 74–95, 2020
work page 2020
-
[6]
BSN: boundary sensitive network for temporal action proposal generation,
Tianwei Lin, Xu Zhao, Haisheng Su, Chongjing Wang, and Ming Yang, “BSN: boundary sensitive network for temporal action proposal generation,” in ECCV 2018 , Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, Eds. vol. 11208 of Lecture Notes in Computer Science, pp. 3–21, Springer
work page 2018
-
[7]
Gaussian temporal aware- ness networks for action localization,
Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo, and Tao Mei, “Gaussian temporal aware- ness networks for action localization,” in CVPR 2019, pp. 344–353
work page 2019
-
[8]
Residual feature-reutilization incep- tion network,
Yuanpeng He, Wenjie Song, Lijian Li, Tianxiang Zhan, and Wenpin Jiao, “Residual feature-reutilization incep- tion network,” Pattern Recognition, p. 110439, 2024
work page 2024
Show all 38 references
-
[9]
Weakly supervised temporal action localization via representative snippet knowledge propagation,
Linjiang Huang, Liang Wang, and Hongsheng Li, “Weakly supervised temporal action localization via representative snippet knowledge propagation,” in CVPR 2022, pp. 3262–3271
2022
-
[10]
Dual-evidential learning for weakly- supervised temporal action localization,
Mengyuan Chen, Junyu Gao, Shicai Yang, and Chang- sheng Xu, “Dual-evidential learning for weakly- supervised temporal action localization,” in ECCV 2022, vol. 13664 of Lecture Notes in Computer Science, pp. 192–208
2022
-
[11]
Temporal feature enhancement dilated convolution network for weakly- supervised temporal action localization,
Jianxiong Zhou and Ying Wu, “Temporal feature enhancement dilated convolution network for weakly- supervised temporal action localization,” in WACV 2023, pp. 6017–6026
2023
-
[12]
Boosting weakly- supervised temporal action localization with text infor- mation,
Guozhang Li, De Cheng, Xinpeng Ding, Nannan Wang, Xiaoyu Wang, and Xinbo Gao, “Boosting weakly- supervised temporal action localization with text infor- mation,” in CVPR 2023, pp. 10648–10657
2023
-
[13]
Combining belief functions based on distance of evi- dence,
Deng Yong, Shi WenKang, Zhu ZhenFu, and Liu Qi, “Combining belief functions based on distance of evi- dence,” Decision support systems , vol. 38, no. 3, pp. 489–493, 2004
2004
-
[14]
Deng entropy,
Yong Deng, “Deng entropy,” Chaos, Solitons & Frac- tals, vol. 91, pp. 549–553, 2016
2016
-
[15]
Conflicting manage- ment of evidence combination from the point of im- provement of basic probability assignment,
Yuanpeng He and Fuyuan Xiao, “Conflicting manage- ment of evidence combination from the point of im- provement of basic probability assignment,” Interna- tional Journal of Intelligent Systems , vol. 36, no. 5, pp. 1914–1942, 2021
1914
-
[16]
Time evidence fusion network: Multi-source view in long-term time series forecasting,
Tianxiang Zhan, Yuanpeng He, Yong Deng, Zhen Li, Wenjie Du, and Qingsong Wen, “Time evidence fusion network: Multi-source view in long-term time series forecasting,” arXiv preprint arXiv:2405.06419, 2024
2024 arXiv
-
[17]
Mmget: a markov model for generalized evidence theory,
Yuanpeng He and Yong Deng, “Mmget: a markov model for generalized evidence theory,” Computational and Applied Mathematics, vol. 41, pp. 1–41, 2022
2022
-
[18]
Witold Pedrycz, Fuzzy control and fuzzy systems , Re- search Studies Press Ltd., 1993
1993
-
[19]
A matrix-based distance of pythagorean fuzzy set and its application in medical diagnosis,
Yuanpeng He, Lijian Li, and Tianxiang Zhan, “A matrix-based distance of pythagorean fuzzy set and its application in medical diagnosis,” 2024
2024
-
[20]
Witold Pedrycz and Fernando Gomide, Fuzzy systems engineering: toward human-centric computing , John Wiley & Sons, 2007
2007
-
[21]
A new base func- tion in basic probability assignment for conflict manage- ment,
Yuanpeng He and Fuyuan Xiao, “A new base func- tion in basic probability assignment for conflict manage- ment,” Applied Intelligence , vol. 52, no. 4, pp. 4473– 4487, 2022
2022
-
[22]
Nndf: A new neural detection network for aspect-category sentiment analy- sis,
Lijian Li, Yuanpeng He, and Li Li, “Nndf: A new neural detection network for aspect-category sentiment analy- sis,” in International Conference on Knowledge Science, Engineering and Management. Springer, 2022, pp. 339– 355
2022
-
[23]
Quo vadis, ac- tion recognition? A new model and the kinetics dataset,
Jo ˜ao Carreira and Andrew Zisserman, “Quo vadis, ac- tion recognition? A new model and the kinetics dataset,” in CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 . 2017, pp. 4724–4733, IEEE Computer Society
2017
-
[24]
Cross-modal consensus network for weakly supervised temporal action localization,
Fa-Ting Hong, Jia-Chang Feng, Dan Xu, Ying Shan, and Wei-Shi Zheng, “Cross-modal consensus network for weakly supervised temporal action localization,” inMM 2021, pp. 1591–1599
2021
-
[25]
Springer, 2016
Audun Jøsang, Subjective Logic - A F ormalism for Reasoning Under Uncertainty , Artificial Intelligence: Foundations, Theory, and Algorithms. Springer, 2016
2016
-
[26]
Ordinal belief entropy,
Yuanpeng He and Yong Deng, “Ordinal belief entropy,” Soft Computing, vol. 27, no. 11, pp. 6973–6981, 2023
2023
-
[27]
Yager and Liping Liu, Eds., Classic Works of the Dempster-Shafer Theory of Belief Functions , vol
Ronald R. Yager and Liping Liu, Eds., Classic Works of the Dempster-Shafer Theory of Belief Functions , vol. 219 of Studies in Fuzziness and Soft Computing , Springer, 2008
2008
-
[28]
Tdqmf: Two- dimensional quantum mass function,
Yuanpeng He and Yong Deng, “Tdqmf: Two- dimensional quantum mass function,” Information Sci- ences, vol. 621, pp. 749–765, 2023
2023
-
[29]
Upper and lower probabilities in- duced by a multivalued mapping,
Arthur P. Dempster, “Upper and lower probabilities in- duced by a multivalued mapping,” in Classic Works of the Dempster-Shafer Theory of Belief Functions , Ronald R. Yager and Liping Liu, Eds., vol. 219 of Studies in Fuzziness and Soft Computing , pp. 57–72. Springer, 2008
2008
-
[30]
An evidential classifier based on dempster-shafer theory and deep learning,
Zheng Tong, Philippe Xu, and Thierry Denoeux, “An evidential classifier based on dempster-shafer theory and deep learning,” Neurocomputing, vol. 450, pp. 275– 293, 2021
2021
-
[31]
Fine-grained temporal contrastive learning for weakly- supervised temporal action localization,
Junyu Gao, Mengyuan Chen, and Changsheng Xu, “Fine-grained temporal contrastive learning for weakly- supervised temporal action localization,” inCVPR 2022, pp. 19967–19977
2022
-
[32]
Asm-loc: Action-aware seg- ment modeling for weakly-supervised temporal action localization,
Bo He, Xitong Yang, Le Kang, Zhiyu Cheng, Xin Zhou, and Abhinav Shrivastava, “Asm-loc: Action-aware seg- ment modeling for weakly-supervised temporal action localization,” in CVPR 2022, pp. 13915–13925
2022
-
[33]
Forcing the whole video as background: An adversarial learning strategy for weakly temporal action localization,
Ziqiang Li, Yongxin Ge, Jiaruo Yu, and Zhongming Chen, “Forcing the whole video as background: An adversarial learning strategy for weakly temporal action localization,” in MM 2022, pp. 5371–5379
2022
-
[34]
Two-stream networks for weakly-supervised temporal action local- ization with semantic-aware mechanisms,
Yu Wang, Yadong Li, and Hongbin Wang, “Two-stream networks for weakly-supervised temporal action local- ization with semantic-aware mechanisms,” in CVPR 2023, pp. 18878–18887
2023
-
[35]
Proposal-based multiple instance learning for weakly-supervised temporal action localization,
Huan Ren, Wenfei Yang, Tianzhu Zhang, and Yong- dong Zhang, “Proposal-based multiple instance learning for weakly-supervised temporal action localization,” in CVPR 2023, pp. 2394–2404
2023
-
[36]
Evidential deep learning to quantify classification un- certainty,
Murat Sensoy, Lance M. Kaplan, and Melih Kandemir, “Evidential deep learning to quantify classification un- certainty,” in NeurIPS 2018, pp. 3183–3193
2018
-
[37]
Ordinal fuzzy entropy,
Y He and Y Deng, “Ordinal fuzzy entropy,” Iranian Journal of Fuzzy Systems , vol. 19, no. 3, pp. 171–186, 2022
2022
-
[38]
THUMOS challenge: Action recognition with a large number of classes,
Y .-G. Jiang, J. Liu, A. Roshan Zamir, G. Toderici, I. Laptev, M. Shah, and R. Sukthankar, “THUMOS challenge: Action recognition with a large number of classes,”http://crcv.ucf.edu/THUMOS14/, 2014
2014
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.