REVIEW 5 major objections 5 minor 25 references
Towards Classifying Histopathological Microscope Images as Time Series Data
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Treating microscope histopathology images as time series, aligned by Soft-DTW to a fixed Beta-shaped target, outperforms prior sequence and MIL baselines on two datasets.
desk verdict Time-series framing is new and results look good, but the argmax align loss in Eq. 7 cannot train as written, so the method is not reproducible. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Soft-DTW loss with an implicit time-series target: a differentiable dynamic-time-warping distance that maps each sequence of per-image predictions, of arbitrary length n, onto a fixed reference sequence of length l. The reference is the cumulative distribution function of a Beta(3,20) distribution, a monotone curve that encodes the empirical observation that diagnostically relevant regions tend to appear late in manually captured microscope sequences. Around this sit three supporting mechanisms: wavelet decomposition splits the feature sequence into stable and rapid-change components; gated attention pooling compresses the whole sequence into one prediction under the weak label; and an alignment term, plus a shared dimension-reduction network, coordinates the two prediction paths. The inference-time majority vote across four strategies is what turns complementary predictions into the reported stable results.
What would settle it
Compute, on a held-out set of microscope sequences, the actual index of the first frame a pathologist would flag as diagnostic and compare the empirical cumulative distribution with Beta(3,20); if the empirical curve is flat, early-peaked, or highly variable across cases, the implicit target is misspecified and the Soft-DTW term should measurably hurt accuracy relative to using a target fitted to that distribution.
Extended reading notes
Core claim
The central claim is that the proposed framework classifies weakly labeled microscope image sequences more accurately than existing sequence classifiers and multiple-instance learners. The training signal is a Soft-DTW loss that aligns the per-frame class predictions with a fixed reference sequence shaped like the cumulative distribution function of a Beta(3,20), together with a gated attention pooling loss and an alignment term that forces the pooled prediction and the temporal prediction to agree on the class. At inference, the method combines four strategies—the attention-pooled prediction, the DTW distance to the reference, nearest-neighbor distance in trained prediction space, and a majority vote—and the vote is the best or tied for best in the reported experiments. On the private colon polyp dataset the voting strategy reaches 0.990 F1 and 0.991 accuracy, and on the public benchmark it outperforms all baselines at every magnification. The authors also report ablations showing that removing either the temporal prediction path or the pooled prediction path produces the largest drops in performance.
Load-bearing premise
The training loss assumes that, within each sequence, diagnostically relevant frames follow the cumulative distribution function of a Beta(3,20), a fixed curve that rises late; the paper justifies this choice only by empirical observation and gives no quantitative evidence for the curve's shape.
Editorial extensions
If this is right
- Microscope image sequences, including all four magnifications of the public breast histopathology benchmark, can be classified more accurately by this time-series formulation than by current sequence or MIL baselines.
- Variable-length microscope sequences can be consumed without truncation or fixed-length preprocessing, since Soft-DTW aligns them to a common reference.
- Majority voting across the four inference strategies gives the most stable predictions, so complementary temporal and pooled signals are worth retaining at test time.
- Ablation results imply that both the temporal alignment path and the attention-pooled path contribute, and removing either one degrades performance the most.
Reading between the lines
- Editorial inference: if the Beta(3,20) target were fitted from the data rather than fixed, the framework could adapt to other acquisition styles, such as screening protocols where abnormal regions appear early rather than late.
- Editorial inference: the Soft-DTW-to-implicit-target recipe is not specific to histopathology and could be tested on other weakly labeled ordered image sets, such as endoscopy or dermatoscopy video, where no coordinate information is recorded.
- Editorial inference: the voting gain suggests a testable extension—training the same architecture with different target shapes per site or per stain and letting inference select among them.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a weakly supervised classification framework for histopathology microscope image sequences, treating the order of manually captured images as a time series. The method extracts pre-trained features, removes redundant frames via L2 differencing, applies an unspecified wavelet transform to separate stable and rapid-change components, and feeds the sequence through self- and cross-attention modules. An attention-pooling module produces a whole-sequence embedding, and a shared dimension-reduction network produces both per-frame predictions and a pooled prediction. Training uses Soft-DTW to align per-frame predictions to a fixed-length target shaped as the CDF of a Beta(3,20) distribution, plus cross-entropy on the pooled prediction and an alignment term between the argmax class of the sequence and the argmax class of the pooled prediction. The authors evaluate four inference strategies (pooled prediction, DTW distance, KNN, and voting) on a private colon polyp dataset (SMF) and on BreakHis, reporting F1 and accuracy improvements over LSTM, GRU, Transformer, ABMIL, and TransMIL, with an ablation study attributing performance to the main components.
Significance. The paper addresses a genuinely under-served data modality: low-cost microscope-captured image sequences rather than whole-slide scans, with weak case-level labels and variable sequence lengths. The combination of Soft-DTW alignment to a fixed-length target with attention pooling and voting-based inference is a reasonable and potentially useful design, and the inclusion of a private clinical dataset adds practical value. If the implementation details are clarified and the empirical claims are made statistically robust, the framework could be a useful contribution to weakly labeled histopathology classification. However, the current manuscript is not fully reproducible from the text: the ideal reference sequence in Eq. (6) is undefined, the align loss in Eq. (7) is non-differentiable as written, the implicit Beta(3,20) target is unsupported by quantitative evidence, and the reported test sets are too small to support the strength of the stated superiority claims without error bars or significance tests.
major comments (5)
- [Section 2.5, Eq. (6)] The ideal reference sequence y_ideal is never defined. The text only states that y_ideal ∈ R^{n×C} is set 'to set the minimum value of the term to zero,' but it does not specify its entries, how it is constructed from the weak label, or whether it is fixed or learned. Since L_dtw subtracts D(y_ideal, y_l), the entire alignment loss depends on this undefined quantity, so the training objective cannot be reproduced or even evaluated as written. Please define y_ideal explicitly (for example, as a repetition of the weak-label one-hot vector, or as a constant zero sequence) and state how it is generated for each input sequence of length n.
- [Section 2.5, Eq. (7)] The align loss is written with argmax over predicted class indices, which is piecewise constant and has zero gradient almost everywhere. As a result, L_align as defined in Eq. (7) cannot contribute any gradient during backpropagation, so the total loss in Eq. (8) does not actually train the model through this term. This makes the ablation result in Fig. 4 — where removing L_align causes a performance drop — inconsistent with the stated formulation. Please either specify the differentiable surrogate actually used (e.g., soft-argmax, cross-entropy on softmax probabilities, or a differentiable sorting operator) and justify it, or remove L_align from the method and re-run the ablation accordingly.
- [Section 2.5, implicit target] The implicit time-series target y_l is defined as the CDF of a Beta(3,20) distribution, justified solely by an informal 'empirical observation' that experts find it hard to capture symptomatic regions in the first frames. This assumption is load-bearing: it shapes L_dtw and is also used directly by the 'DTW distance' inference strategy. If the actual temporal pattern of lesion appearance differs from this monotone CDF, the alignment loss will distort per-frame predictions and degrade the method. Please provide quantitative evidence for the assumed progression (e.g., image-level relevance annotation over time, or an analysis of where malignant regions appear in the captured sequences), and include a sensitivity analysis over the Beta parameters or alternative target shapes.
- [Experiment, Table 2] The reported test sets are very small — BreakHis is evaluated with n=28 cases per magnification and SMF with n=186 — and no error bars, confidence intervals, or significance tests are provided. For example, on BreakHis 40× the F1 gap between Ours-AP (0.954) and Transformer (0.928) is about 2.6 percentage points, which is not statistically meaningful with 28 cases. To support the claimed superiority, please report the mean and standard deviation over multiple runs, use case-level bootstrap confidence intervals, and apply an appropriate paired statistical test across methods and inference strategies.
- [Section 2.1, Wavelet transformation] The wavelet transformation is not specified: the paper gives no wavelet family, decomposition level, coefficient selection rule, or reconstruction equations for the low-frequency component X_stb and the high-frequency component X_rpd. Since the ablation in Fig. 4 attributes a performance drop to removing the wavelet component, this missing implementation detail directly blocks replication. Please provide the exact transform used, the thresholding or selection procedure, and the output shapes of X_stb and X_rpd.
minor comments (5)
- [Section 2.1, differencing threshold] The threshold τ is said to be determined via the validation set, and Section 3 states that it removes 25% of SMF duplicates and 5% of BreakHis duplicates, but the actual criterion and numerical values are not given. Please report how τ is selected and what values were used.
- [Section 3, experimental settings] The text says the model adopted 'Tinyrecipes from [16]' — likely a typo for a ViT-based recipe — and gives no number of epochs, early stopping criterion, or hidden dimension settings. Please provide full training hyperparameters for reproducibility.
- [Section 3, dataset preprocessing] The notation 'y25' and 'y75' is used for the target length on BreakHis and SMF, respectively, but the relationship between y_l, the chosen constant l, and the case-average sequence length is not made explicit. Please clarify how l is chosen and whether y_l is shared across classes or class-specific.
- [Section 3, loss weights] The loss weights λ_dtw=1, λ_ap=10, and λ_align=10 are reported without any sensitivity analysis. Given that these weights directly control the balance between alignment, pooled classification, and the align term, a small ablation or a statement of robustness would strengthen the paper.
- [Table 2] Several cells in Table 2 appear as concatenated numbers (e.g., '0.9620.950' for DTW Distance on 40× and '0.9690.957' on 400×). Please fix the formatting so that F1 and accuracy values are clearly separated.
Circularity Check
No significant circularity: the central AP classification path is trained on ground-truth labels; the Beta(3,20) target and DTW-distance inference are hand-designed assumptions, not reductions to the test labels.
full rationale
The paper's central claim is an empirical comparison of a weakly supervised attention-pooling classifier against baselines. The primary prediction path y_ap is trained with cross-entropy (L_ap) on case-level labels, so the main result has independent grounding. The Soft-DTW term L_dtw (Eq. 6) aligns per-image predictions to a fixed Beta(3,20) target chosen by the authors; this is an explicit modeling assumption, not a quantity fitted from the test labels. The 'DTW distance' inference strategy compares test predictions to the same target, which makes that particular strategy dependent on the author-chosen template, but the paper's headline results use majority voting of four strategies and the ablation is based on voting; the central comparison is not forced by construction. No load-bearing self-citations appear in the reference list, and no uniqueness argument is imported from the authors' prior work. One internal-consistency caveat, not circularity: L_align in Eq. 7 uses argmax, whose gradient is zero almost everywhere, so the stated objective cannot train the model as written; this is a reproducibility and technical flaw that should be addressed, but it does not make the derivation circular. Overall, the derivation chain is self-contained relative to its stated assumptions.
Assumptions & free parameters
free parameters (5)
- Implicit target shape parameters =
Beta(3,20)
- Differencing threshold tau =
Tuned to remove 25% (SMF) and 5% (BreakHis) of duplicated images
- Target length l =
y_25 for BreakHis, y_75 for SMF
- Loss weights =
lambda_dtw=1, lambda_ap=10, lambda_align=10
- Wavelet type and decomposition level =
Not specified
assumptions (4)
- ad hoc to paper The temporal order of microscope images contains a monotone progression where the diagnostic class appears gradually; the CDF of Beta(3,20) approximates this progression.
- domain assumption Low-frequency and high-frequency wavelet components of the feature sequence correspond to stable and rapidly changing diagnostically meaningful patterns.
- domain assumption Adjacent-feature L2 distance below tau reliably identifies redundant images without removing informative ones.
- domain assumption The shared dimension reduction network can map both the per-instance sequence and the pooled embedding into the same class space without conflict.
invented entities (2)
-
Implicit time series target y_l (Beta(3,20) CDF profile)
-
Ideal reference sequence y_ideal
Cite this review
Pith. "Pith review of Towards Classifying Histopathological Microscope Images as Time Series Data." pith.science (2026). https://pith.science/paper/2QKB3L7C
@misc{pith2026250615977,
author = {Pith},
title = {Pith review of: Towards Classifying Histopathological Microscope Images as Time Series Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/2QKB3L7C}},
note = {Machine review of arXiv:2506.15977}
}
read the original abstract
As the frontline data for cancer diagnosis, microscopic pathology images are fundamental for providing patients with rapid and accurate treatment. However, despite their practical value, the deep learning community has largely overlooked their usage. This paper proposes a novel approach to classifying microscopy images as time series data, addressing the unique challenges posed by their manual acquisition and weakly labeled nature. The proposed method fits image sequences of varying lengths to a fixed-length target by leveraging Dynamic Time-series Warping (DTW). Attention-based pooling is employed to predict the class of the case simultaneously. We demonstrate the effectiveness of our approach by comparing performance with various baselines and showcasing the benefits of using various inference strategies in achieving stable and reliable results. Ablation studies further validate the contribution of each component. Our approach contributes to medical image analysis by not only embracing microscopic images but also lifting them to a trustworthy level of performance.
Reference graph
Works this paper leans on
-
[1]
Towards Classifying Histopathological Microscope Images as Time Series Data
INTRODUCTION Cancer remains a leading cause of death worldwide, prompt- ing the deep-learning based computer vision community to develop various models to address this critical issue [1]. Histopathology image classification models seek to assist medical professionals in reducing diagnostic errors and improving prognosis effectiveness [2]. However, we note...
work page Pith review arXiv 2025
-
[2]
The overall framework is depicted in Figure 2
METHOD We propose a method for classifying microscopic pathology image-sequences with varying lengths and weak labels. The overall framework is depicted in Figure 2. 2.1. Data processing ∆(xi,x i+1) =||x i+1 −x i||2 2 (1) Time series differencingWe extract a feature sequence X={x i ∈R d |i= 1,2,· · ·, n}, wherenis the length of the sequence anddis feature...
-
[3]
EXPERIMENT Experimental settingsWe adopted theTinyrecipes from
-
[4]
CONCLUSION Focusing on microscope images, a practically significant but under-explored type of visual data, this study formulated au- tomatically captured image sequences as time series data. Our proposed framework preprocesses data using various tech- niques and performs time series analysis and point estima- tion simultaneously. Through extensive experi...
-
[5]
Approval was granted by the Ethics Review Board (SMF-IRB-2020-007) and (KAIST-IRB-22- 335)
COMPLIANCE WITH ETHICAL STANDARDS This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Ethics Review Board (SMF-IRB-2020-007) and (KAIST-IRB-22- 335). Also, this research study was conducted retrospectively using human subject data made available in open access by [18]. Ethical approval was not r...
work page 2020
-
[6]
Research on Develop- ing a Next Generation Medical Diagnosis System Using Deep Learning
ACKNOWLEDGEMENT This research was supported by the Seegene Medical Founda- tion, South Korea, under the project “Research on Develop- ing a Next Generation Medical Diagnosis System Using Deep Learning” (Grant Number: G01180115)
-
[7]
Soft-dtw: a differ- entiable loss function for time-series,
Marco Cuturi and Mathieu Blondel, “Soft-dtw: a differ- entiable loss function for time-series,” inInternational conference on machine learning. PMLR, 2017, pp. 894– 903
work page 2017
-
[8]
Deep learning in cancer pathol- ogy: a new generation of clinical biomarkers,
Amelie Echle, Niklas Timon Rindtorff, Titus Josef Brinker, Tom Luedde, Alexander Thomas Pearson, and Jakob Nikolas Kather, “Deep learning in cancer pathol- ogy: a new generation of clinical biomarkers,”British journal of cancer, vol. 124, no. 4, pp. 686–696, 2021
work page 2021
Show all 25 references
-
[9]
Deep learning in histopathology: the path to the clinic,
Jeroen Van der Laak, Geert Litjens, and Francesco Ciompi, “Deep learning in histopathology: the path to the clinic,”Nature medicine, vol. 27, no. 5, pp. 775–784, 2021
2021
-
[10]
A deep learning approach for colonoscopy pathology wsi analysis: accurate seg- mentation and classification,
Ruiwei Feng, Xuechen Liu, Jintai Chen, Danny Z Chen, Honghao Gao, and Jian Wu, “A deep learning approach for colonoscopy pathology wsi analysis: accurate seg- mentation and classification,”IEEE Journal of Biomed- ical and Health Informatics, vol. 25, no. 10, pp. 3700– 3708, 2020
2020
-
[11]
From whole slide imag- ing to microscopy: Deep microscopy adaptation net- work for histopathology cancer image classification,
Yifan Zhang, Hanbo Chen, Ying Wei, Peilin Zhao, Jiezhang Cao, Xinjuan Fan, Xiaoying Lou, Hailing Liu, Jinlong Hou, Xiao Han, et al., “From whole slide imag- ing to microscopy: Deep microscopy adaptation net- work for histopathology cancer image classification,” in Internationa...
2019
-
[12]
Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) net- work,
Alex Sherstinsky, “Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) net- work,”Physica D: Nonlinear Phenomena, vol. 404, pp. 132306, 2020
2020
-
[13]
Whole slide imaging in pathology: advan- tages, limitations, and emerging perspectives,
Navid Farahani, Anil V Parwani, and Liron Pan- tanowitz, “Whole slide imaging in pathology: advan- tages, limitations, and emerging perspectives,”Pathol- ogy and Laboratory Medicine International, pp. 23–33, 2015
2015
-
[14]
Attention-based deep multiple instance learning,
Maximilian Ilse, Jakub Tomczak, and Max Welling, “Attention-based deep multiple instance learning,” in International conference on machine learning. PMLR, 2018, pp. 2127–2136
2018
-
[15]
Benchmarking self- supervised learning on diverse pathology datasets,
Mingu Kang, Heon Song, Seonwook Park, Donggeun Yoo, and S ´ergio Pereira, “Benchmarking self- supervised learning on diverse pathology datasets,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3344–3354
2023
-
[16]
The model was trained using the Adam optimizer [17] with a learning rate of1e−4and betas of(0.9,0.999)
for the attention module’s parameters. The model was trained using the Adam optimizer [17] with a learning rate of1e−4and betas of(0.9,0.999). All experiments were conducted on a single NVIDIA® RTX 2080 Ti. DatasetWe employ two microscopy datasets: BreakHis [18], a benchmark o...
-
[17]
Attention is all you need,
A Vaswani, “Attention is all you need,”Advances in Neural Information Processing Systems, 2017
2017
-
[18]
Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,”arXiv preprint arXiv:2308.06721, 2023
2023 arXiv
-
[19]
Dynamic-programming approach to continuous speech recognition,
Hiroaki Sakoe, “Dynamic-programming approach to continuous speech recognition,” in1971 Proc. the In- ternational Congress of Acoustics, Budapest, 1971
1971
-
[20]
Long short-term memory,
S Hochreiter, “Long short-term memory,”Neural Com- putation MIT-Press, 1997
1997
-
[21]
Empirical evaluation of gated re- current neural networks on sequence modeling,
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio, “Empirical evaluation of gated re- current neural networks on sequence modeling,”arXiv preprint arXiv:1412.3555, 2014
2014 arXiv
-
[22]
Transmil: Trans- former based correlated multiple instance learning for whole slide image classification,
Zhuchen Shao, Hao Bian, Yang Chen, Yifeng Wang, Jian Zhang, Xiangyang Ji, et al., “Transmil: Trans- former based correlated multiple instance learning for whole slide image classification,”Advances in neural information processing systems, vol. 34, pp. 2136–2147, 2021
2021
-
[23]
An image is worth 16x16 words: Transformers for image recognition at scale,
Alexey Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020
2010 arXiv
-
[24]
Adam: A method for stochastic optimization,
Diederik P Kingma, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[25]
A dataset for breast cancer histopathological image classification,
Fabio A Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte, “A dataset for breast cancer histopathological image classification,”Ieee transac- tions on biomedical engineering, vol. 63, no. 7, pp. 1455–1462, 2015
2015
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.