REVIEW 2 major objections 2 minor 45 references
Attention Mechanism in Randomized Time Warping
T0 review · 2 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper claims that Randomized Time Warping's per-element contribution weights are functionally self-attention weights, reporting a 0.80 average weight-pattern correlation and a 5% accuracy gain over Transformers on Something-Something V2
desk verdict The submission is an abstract for one paper (RTW/self-attention) attached to a different paper (RotaTouille); the claimed result is absent from the full text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two objects carry the argument. The first is the per-element contribution weight that RTW learns for each position of a sequence, RTW being a general extension of Dynamic Time Warping that matches and compares sequential patterns. The second is the comparison device: canonical correlation analysis, a statistical way of measuring how well two weight patterns align across directions, reported here as an average correlation of 0.80 over the ten smallest canonical angles between the RTW weight pattern and a Transformer self-attention weight pattern. The claimed advantage is carried by receptive-field scope: RTW weights cover the whole input, while self-attention is limited to a local window for
What would settle it
Repeat the weight-pattern comparison over all canonical angles instead of only the ten smallest: if the average correlation on the omitted directions is near zero, the claimed identity loses its support. The supplied full text, being a different paper, already blocks this check, since it contains no RTW experiments; retrieving the actual RTW manuscript and re-running the Something-Something V2 comparison with matched attention settings would settle whether the 5% gain is real.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the optimal contribution weights RTW assigns to each element of a sequence are interpretable as self-attention weights: they score the importance of sequence elements in producing discriminative features, exactly as Transformer self-attention scores its inputs. The two weight patterns are reported to agree at average correlation 0.80 over the ten smallest canonical angles. The functional difference is scope — RTW attends over the entire input sequence while self-attention is confined to a local window by the cost of the attention matrix — and this difference is credited with RTW's 5% accuracy improvement over Transformers on Something-Something
Load-bearing premise
The claim collapses if either of two premises gives way: that a 0.80 average correlation of weight patterns over the ten smallest canonical angles means RTW's weights operate as self-attention weights rather than merely resembling them, or — more basically for this manuscript — that the submitted full text is actually the RTW paper, since the body supplied is an unrelated contour-learning paper.
Editorial extensions
If this is right
- If the identification is right, RTW joins the attention family: a classical warping-based matcher and the weighting mechanism inside Transformers turn out to be the same operation, connecting two research lines previously treated as separate.
- RTW's global receptive field becomes a concrete design lesson — attention over the entire sequence can beat local-window attention, by roughly 5% accuracy on Something-Something V2.
- Methods for interpreting self-attention, such as weight visualization and importance analysis, transfer directly to RTW, and RTW's weight semantics can explain which frames of an action video drive recognition.
- The 0.80 agreement predicts that RTW and self-attention converge on similar feature-weighting strategies despite starting from entirely different derivations, making RTW a candidate cheap, globally-attentive alternative wherever attention windows are the bottleneck.
Reading between the lines
- Extension the paper leaves implicit: the correlation is reported only over the ten smallest canonical angles; a full analysis over all canonical directions would show whether the agreement is a general property of the two mechanisms or concentrated in a few dominant directions.
- If the global-versus-local receptive field is the true cause of the 5% gain, that gain should widen as input sequences lengthen and Transformers must truncate their attention windows further — a sequence-length sweep would test this directly.
- Because the supplied body text is a different paper, the figures cited above (0.80 correlation, 5% gain) are unverified from this manuscript alone; reading the actual RTW paper and checking how the canonical angles and Transformer baseline were computed is a necessary step before relying on the claim.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The abstract of arXiv:2508.16366 claims that Randomized Time Warping (RTW) can be interpreted as a type of self-attention mechanism, that the two weight patterns agree with average correlation 0.80 over the ten smallest canonical angles, and that RTW achieves a 5% accuracy improvement over Transformers on Something-Something V2. The submitted full text, however, is a completely different manuscript: 'RotaTouille: Rotation Equivariant Deep Learning for Contours' (arXiv:2508.16359v2), which develops complex-valued circular convolutions for contour data. The body contains no definition of RTW, no canonical-angle analysis, no self-attention discussion, and no Something-Something V2 experiments. The central claim of the abstract is therefore not present in the manuscript under review and cannot be checked or reproduced from the submitted artifact.
Significance. If the abstract's claim were supported, connecting RTW's per-element contribution weights to self-attention could be an interesting conceptual bridge between dynamic time warping and transformer attention, with a potentially useful global-receptive-field advantage over local self-attention. However, the submitted manuscript provides no evidence for this claim. The body is an unrelated, self-contained contour-learning paper with its own propositions (e.g., Prop. A.4, Prop. 2.1, Prop. 2.2) and a public code repository, but those contributions do not bear on RTW or attention. The claimed 0.80 correlation and 5% accuracy gain are entirely unsubstantiated in the artifact, so the significance of the paper as submitted is nil.
major comments (2)
- [Entire body vs. abstract] The submitted manuscript is not about RTW or self-attention. The title, abstract, Sections 1–4, and Appendix A describe 'RotaTouille: Rotation Equivariant Deep Learning for Contours' (arXiv:2508.16359v2), which addresses complex-valued circular convolution for contour classification, reconstruction, and regression. There is no definition of Randomized Time Warping, no canonical-angle computation, no interpretation of contribution weights as self-attention weights, and no Something-Something V2 experiment anywhere in the text. The abstract's central claim is therefore entirely unsupported by the manuscript body; this is not a missing derivation but a complete absence of the claimed subject matter.
- [Abstract (canonical-angle evidence)] Even taken on its own terms, the abstract's evidence is insufficient: the only support for 'RTW weights are self-attention weights' is an average correlation of 0.80 computed across the ten smallest canonical angles. No derivation, invariance argument, experimental protocol, error bars, or ablation is provided. The restriction to the ten smallest canonical angles is unexplained and could in principle select the most favorable subspace; the abstract gives no basis for believing this subspace is the relevant one. Because the body does not contain the analysis, this concern cannot be resolved by consulting the manuscript.
minor comments (2)
- [Header and metadata] The arXiv identifiers do not match: the submitted header cites arXiv:2508.16359v2, while the abstract is for arXiv:2508.16366. This reinforces that the manuscript body and the claimed contribution are inconsistent.
- [Abstract precision] The abstract does not state how the 0.80 correlation is computed (which weight patterns, which model, how many seeds), nor how the 5% improvement is measured. These details are essential for any empirical claim of this kind.
Circularity Check
RTW-as-self-attention is entailed by the paper's own definitions; the submitted body is a different paper, so no independent derivation exists.
-
self definitional
[Abstract, first paragraph (arXiv:2508.16366)]
"The self-attention is a mechanism that enables models to identify and weigh the importance of different parts of an input sequential pattern. On the other hand, RTW is a general extension of Dynamic Time Warping (DTW), a technique commonly used for matching and comparing sequential patterns. In essence, RTW searches for optimal contribution weights for each element of the input sequential patterns to produce discriminative features. Although the two approaches look different, these contribution weights can be interpreted as self-attention weights."
The paper's stated definition of self-attention is 'weigh the importance of different parts of an input sequential pattern,' and its stated definition of RTW is 'searches for optimal contribution weights for each element of the input sequential patterns.' Under these definitions, RTW satisfies the definition of self-attention immediately: the conclusion 'these contribution weights can be interpreted as self-attention weights' is a restatement of the premises, not a derived result. The subsequent correlation of 0.80 is presented as confirmation, but the equivalence has already been assumed by the definitions, and no equation or argument connects RTW's DTW-derived weights to the query-key-value mechanism of Transformer self-attention. The central 'reveals' claim therefore reduces by construc
full rationale
The only explicit link between RTW and self-attention in the abstract is definitional: self-attention is characterized as weighing input-part importance, and RTW is characterized as optimizing per-element contribution weights; the claimed interpretation follows immediately from those glosses. The reported correlation over the ten smallest canonical angles is an empirical observation, but it is presented as evidence for an equivalence already entailed by the definitions, and no derivation is supplied. A further, severe problem is that the submitted full text is not the RTW paper: it is 'RotaTouille: Rotation Equivariant Deep Learning for Contours' (arXiv:2508.16359v2), containing no RTW definition, no canonical-angle analysis, no self-attention interpretation, and no Something-Something V2 experiment. That is a missing-support / artifact-mismatch issue rather than a circular step, and it makes the abstract's empirical claims uncheckable. There are no load-bearing self-citations and no imported uniqueness theorems. The score of 6 reflects the partial circularity of the central interpretive claim; the empirical benchmarks, if present, would be independent, but they are absent from the artifact.
Assumptions & free parameters
free parameters (1)
- Number of canonical angles used in the similarity comparison =
10 (the ten smallest)
assumptions (3)
- domain assumption Correlational similarity of RTW and self-attention weight patterns (0.80 over the ten smallest canonical angles) implies RTW is functionally interpretable as self-attention.
- ad hoc to paper The ten smallest canonical angles give a fair measure of weight-pattern agreement.
- ad hoc to paper The submitted full text is the paper described by the abstract.
Cite this review
Pith. "Pith review of Attention Mechanism in Randomized Time Warping." pith.science (2026). https://pith.science/paper/5WWR6CXF
@misc{pith2026250816366,
author = {Pith},
title = {Pith review of: Attention Mechanism in Randomized Time Warping},
year = {2026},
howpublished = {\url{https://pith.science/paper/5WWR6CXF}},
note = {Machine review of arXiv:2508.16366}
}
read the original abstract
This paper reveals that we can interpret the fundamental function of Randomized Time Warping (RTW) as a type of self-attention mechanism, a core technology of Transformers in motion recognition. The self-attention is a mechanism that enables models to identify and weigh the importance of different parts of an input sequential pattern. On the other hand, RTW is a general extension of Dynamic Time Warping (DTW), a technique commonly used for matching and comparing sequential patterns. In essence, RTW searches for optimal contribution weights for each element of the input sequential patterns to produce discriminative features. Although the two approaches look different, these contribution weights can be interpreted as self-attention weights. In fact, the two weight patterns look similar, producing a high average correlation of 0.80 across the ten smallest canonical angles. However, they work in different ways: RTW attention operates on an entire input sequential pattern, while self-attention focuses on only a local view which is a subset of the input sequential pattern because of the computational costs of the self-attention matrix. This targeting difference leads to an advantage of RTW against Transformer, as demonstrated by the 5\% performance improvement on the Something-Something V2 dataset.
Reference graph
Works this paper leans on
-
[1]
Curvature scale space image in shape similarity retrieval
Sadegh Abbasi, Farzin Mokhtarian, and Josef Kittler. “Curvature scale space image in shape similarity retrieval”. In:Multimedia systems7 (1999), pp. 467–476. 2
work page 1999
-
[2]
Unitary evolution recurrent neural net- works
Martin Arjovsky, Amar Shah, and Yoshua Bengio. “Unitary evolution recurrent neural net- works”. In:International conference on machine learning. PMLR. 2016, pp. 1120–1128. 4
work page 2016
-
[3]
Binary shape classification using convolutional neural networks
Habibollah Agh Atabay. “Binary shape classification using convolutional neural networks”. In:IIOAB J7.5 (2016), pp. 332–336. 2
work page 2016
-
[4]
A survey of complex-valued neural networks
Joshua Bassey, Lijun Qian, and Xianfang Li. “A survey of complex-valued neural networks”. In:arXiv preprint arXiv:2101.12249(2021). 1
arXiv 2021
-
[5]
Shape context: A new descriptor for shape matching and object recognition
Serge Belongie, Jitendra Malik, and Jan Puzicha. “Shape context: A new descriptor for shape matching and object recognition”. In:Advances in neural information processing systems13 (2000). 2
work page 2000
-
[6]
A novel 2D contour description generalized curvature scale space
Ameni Benkhlifa and Faouzi Ghorbel. “A novel 2D contour description generalized curvature scale space”. In:Representations, Analysis and Recognition of Shape and Motion from Imaging Data: 6th International Workshop, RFMI 2016, Sidi Bou Said Village, Tunisia, October 27-29, 2016, Revised Selected Papers 6. Springer. 2017, pp. 129–140. 2
work page 2016
-
[7]
Yossi Bokor Bleile, Patrice Koehl, and Florian Rehfeldt. “Persistence diagrams as morphologi- cal signatures of cells: A method to measure and compare cells within a population”. In:arXiv preprint arXiv:2310.20644(2023). 1, 3
-
[8]
G. Bradski. “The OpenCV Library”. In:Dr. Dobb’s Journal of Software Tools(2000). 5
work page 2000
Show all 45 references
-
[9]
Geometric deep learning: Grids, groups, graphs, geodesics, and gauges
Michael M Bronstein et al. “Geometric deep learning: Grids, groups, graphs, geodesics, and gauges”. In:arXiv preprint arXiv:2104.13478(2021). 1–3
2021 arXiv
-
[10]
Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles
James Burgess et al. “Orientation-invariant autoencoders learn robust representations for shape profiling of cells and organelles”. In:Nature Communications15.1 (2024), p. 1022. 1, 3, 7
2024
-
[11]
Deep learning algorithm for predictive maintenance of rotating machines through the analysis of the orbits shape of the rotor shaft
R Caponetto et al. “Deep learning algorithm for predictive maintenance of rotating machines through the analysis of the orbits shape of the rotor shaft”. In:International Conference on Smart Innovation, Ergonomics and Applied Human Factors. Springer. 2019, pp. 245–250. 1, 2
2019
-
[12]
Group equivariant convolutional networks
Taco Cohen and Max Welling. “Group equivariant convolutional networks”. In:International conference on machine learning. PMLR. 2016, pp. 2990–2999. 3, 8
2016
-
[13]
Steerable CNNs
Taco S Cohen and Max Welling. “Steerable CNNs”. In:arXiv preprint arXiv:1612.08498 (2016). 3
2016 arXiv
-
[14]
Analysis of deep complex-valued convolutional neural networks for MRI reconstruction and phase-focused applications
Elizabeth Cole et al. “Analysis of deep complex-valued convolutional neural networks for MRI reconstruction and phase-focused applications”. In:Magnetic resonance in medicine86.2 (2021), pp. 1093–1109. 1
2021
-
[15]
ContourCNN: convolutional neural network for contour data classification
Ahmad Droby and Jihad El-Sana. “ContourCNN: convolutional neural network for contour data classification”. In:2021 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME). IEEE. 2021, pp. 1–7. 2, 7, 8, 17
2021
-
[16]
Enhanced radar imaging using a complex-valued convolutional neural network
Jingkun Gao et al. “Enhanced radar imaging using a complex-valued convolutional neural network”. In:IEEE Geoscience and Remote Sensing Letters16.1 (2018), pp. 35–39. 1
2018
-
[17]
Geometric deep learning and equivariant neural networks
Jan E Gerken et al. “Geometric deep learning and equivariant neural networks”. In:Artificial Intelligence Review56.12 (2023), pp. 14605–14662. 1
2023
-
[18]
On complex valued convolutional neural networks
Nitzan Guberman. “On complex valued convolutional neural networks”. In:arXiv preprint arXiv:1602.09046(2016). 1
2016 arXiv
-
[19]
A neural network approach to robust shape classification
Lalit Gupta, Mohammad R Sayeh, and Ravi Tammana. “A neural network approach to robust shape classification”. In:Pattern Recognition23.6 (1990), pp. 563–568. 2
1990
-
[20]
Complex-Valued Neural Networks: Distinctive Features
Akira Hirose. “Complex-Valued Neural Networks: Distinctive Features”. In:Complex-Valued Neural Networks. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 17–56.ISBN: 978-3-642-27632-3.DOI:���������������������������. 4
2012
-
[21]
Diagnosis of rotating machine unbalance using machine learning algorithms on vibration orbital features
Leonardo S Jablon et al. “Diagnosis of rotating machine unbalance using machine learning algorithms on vibration orbital features”. In:Journal of Vibration and Control27.3-4 (2021), pp. 468–476. 1
2021
-
[22]
Rotating machinery diagnostics using deep learning on orbit plot images
Haedong Jeong et al. “Rotating machinery diagnostics using deep learning on orbit plot images”. In:Procedia Manufacturing5 (2016), pp. 1107–1118. 1, 2 10 Rotation Equivariant Deep Learning for Contours
2016
-
[23]
A rotation robust shape transformer for cartoon character recognition
Qi Jia et al. “A rotation robust shape transformer for cartoon character recognition”. In:The Visual Computer(Oct. 2023).ISSN: 1432-2315.DOI: �������������������������� .URL: ������������������������������������������. 2
2023
-
[24]
Semi-Supervised Classification with Graph Convolutional Networks
TN Kipf. “Semi-Supervised Classification with Graph Convolutional Networks”. In:arXiv preprint arXiv:1609.02907(2016). 7, 17
2016 arXiv
-
[25]
An empirical evaluation of deep architectures on problems with many factors of variation
Hugo Larochelle et al. “An empirical evaluation of deep architectures on problems with many factors of variation”. In:Proceedings of the 24th international conference on Machine learning. 2007, pp. 473–480. 7, 8
2007
-
[26]
Gradient-based learning applied to document recognition
Yann LeCun et al. “Gradient-based learning applied to document recognition”. In:Proceedings of the IEEE86.11 (2002), pp. 2278–2324. 7
2002
-
[27]
Complex-Valued Neural Networks: A Comprehensive Survey
ChiYan Lee, Hideyuki Hasegawa, and Shangce Gao. “Complex-Valued Neural Networks: A Comprehensive Survey”. In:IEEE/CAA Journal of Automatica Sinica9.8 (2022), pp. 1406– 1426.DOI:�����������������������. 1
2022
-
[28]
A new deep convolutional neural network for 2D contour classification
Makrem Mhedhbi, Slim Mhiri, and Faouzi Ghorbel. “A new deep convolutional neural network for 2D contour classification”. In: (2022). 2
2022
-
[29]
Rootstock effects on scion phenotypes in a ‘Chambourcin’ experimental vineyard
Zoë Migicovsky et al. “Rootstock effects on scion phenotypes in a ‘Chambourcin’ experimental vineyard”. In:Horticulture Research6.1 (May 2019), p. 64.ISSN: 2052-7276.DOI: �������� �����������������. 1, 3
2019
-
[30]
Efficient and robust retrieval by shape content through curvature scale space
Farzin Mokhtarian, Sadegh Abbasi, and Josef Kittler. “Efficient and robust retrieval by shape content through curvature scale space”. In:Image databases and multi-media search. World Scientific, 1997, pp. 51–58. 2
1997
-
[31]
DeepGCSS: a robust and explainable contour classifier providing generalized curvature scale space features
Mallek Mziou-Sallami et al. “DeepGCSS: a robust and explainable contour classifier providing generalized curvature scale space features”. In:Neural Computing and Applications35.24 (Aug. 2023), pp. 17689–17700.ISSN: 1433-3058.DOI: �������������������������� . URL:��������������...
2023
-
[32]
A threshold selection method from gray-level histograms
Nobuyuki Otsu et al. “A threshold selection method from gray-level histograms”. In:Automat- ica11.285-296 (1975), pp. 23–27. 5
1975
-
[33]
ShapeEmbed: a self-supervised learning framework for 2D contour quantification
Anna Foix Romero et al. “ShapeEmbed: a self-supervised learning framework for 2D contour quantification”. In:arXiv preprint arXiv:2507.01009(2025). 3
2025 arXiv
-
[34]
Complex-valued neural networks for synthetic aperture radar image classification
Theresa Scarnati and Benjamin Lewis. “Complex-valued neural networks for synthetic aperture radar image classification”. In:2021 IEEE Radar Conference (RadarConf21). IEEE. 2021, pp. 1–6. 1
2021
-
[35]
Learning rotation-aware features: From invariant priors to equivariant descriptors
Uwe Schmidt and Stefan Roth. “Learning rotation-aware features: From invariant priors to equivariant descriptors”. In:2012 IEEE conference on computer vision and pattern recognition. IEEE. 2012, pp. 2050–2057. 8
2012
-
[36]
Bag of shape features with a learned pooling function for shape recognition
Wei Shen et al. “Bag of shape features with a learned pooling function for shape recognition”. In:Pattern Recognition Letters106 (2018), pp. 33–40. 2, 5
2018
-
[37]
Shape recognition by bag of skeleton-associated contour parts
Wei Shen et al. “Shape recognition by bag of skeleton-associated contour parts”. In:Pattern Recognition Letters83 (2016), pp. 321–329. 2
2016
-
[38]
Shape recognition by combining contour and skeleton into a mid-level representation
Wei Shen et al. “Shape recognition by combining contour and skeleton into a mid-level representation”. In:Pattern Recognition: 6th Chinese Conference, CCPR 2014, Changsha, China, November 17-19, 2014. Proceedings, Part I 6. Springer. 2014, pp. 391–400. 2
2014
-
[39]
3D-rotation-equivariant quaternion neural networks
Wen Shen et al. “3D-rotation-equivariant quaternion neural networks”. In:Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16. Springer. 2020, pp. 531–547. 3
2020
-
[40]
Learning invariant representations with local transformations
Kihyuk Sohn and Honglak Lee. “Learning invariant representations with local transformations”. In:arXiv preprint arXiv:1206.6418(2012). 8
2012 arXiv
-
[41]
Better than real: Complex-valued neural nets for MRI fingerprinting
Patrick Virtue, X Yu Stella, and Michael Lustig. “Better than real: Complex-valued neural nets for MRI fingerprinting”. In:2017 IEEE international conference on image processing (ICIP). IEEE. 2017, pp. 3953–3957. 1, 4
2017
-
[42]
World Scientific Singapore, 2023, p
Maurice Weiler et al.Equivariant and coordinate independent convolutional networks. World Scientific Singapore, 2023, p. 110. 3
2023
-
[43]
Harmonic networks: Deep translation and rotation equivariance
Daniel E Worrall et al. “Harmonic networks: Deep translation and rotation equivariance”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, pp. 5028–5037. 4, 8 11 Rotation Equivariant Deep Learning for Contours
2017
-
[44]
3D shapenets: A deep representation for volumetric shapes
Zhirong Wu et al. “3D shapenets: A deep representation for volumetric shapes”. In:Proceed- ings of the IEEE conference on computer vision and pattern recognition. 2015, pp. 1912–
2015
-
[45]
Han Xiao, Kashif Rasul, and Roland V ollgraf.Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. Aug. 28, 2017. arXiv: ���������������� �������. 6 12 Rotation Equivariant Deep Learning for Contours A Appendix A.1 Groups and Group Actions We recal...
2017
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.