Pith. sign in

REVIEW 4 major objections 6 minor 18 references

MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a second processing plane and bidirectional time flow to a 3D LSTM improves early prediction of colorectal liver metastases from serial contrast-enhanced CT.

desk verdict A plausible new CRLM prediction model and dataset, but the missing split details could sink the central comparison; worth a careful referee. read the letter →

arxiv 2412.01973 v1 pith:2LNU5DR2 submitted 2024-12-02 eess.IV cs.CV

Xueyang Li , Han Xiao , Weixiang Weng , Xiaowei Xu , Yiyu Shi This is my paper · ORCID
classification eess.IVcs.CV
keywords colorectallivermetastasiscontrast-enhancedCTbidirectionalLSTMmulti-planearchitecturetime-seriespredictionmedicalimageclassification5DdataMPBD-LSTM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Colorectal cancer patients are followed with repeated contrast-enhanced CT scans after surgery, and the paper asks whether a deep learning model can read that scan history to flag patients who will develop liver metastases within two years. The authors assembled a two-cohort dataset of 269 patients (170 with at least three scans) with arterial and portal venous phase volumes at three follow-up times, and compared several video-prediction networks adapted to this five-dimensional data. Their central finding is that treating the two contrast phases as parallel processing planes and letting a 3D LSTM read each plane both forward and backward in time gives the best prediction, AUC 0.790, ahead of PredRNN-V2 (0.765), E3D-LSTM (0.755), SaConvLSTM (0.721), and SimVP (0.662). This matters because earlier detection of liver metastasis is a route to earlier treatment, and because the result identifies two design choices, phase separation and bidirectional temporal reading, that can be tested in other multi-phase imaging problems.

What carries the argument

The load-bearing object is MPBD-LSTM, built from 3D-LSTM modules. Each module contains two E3D-LSTM cells, one forward and one backward, whose hidden states are combined by summation, $y_{v,t} = \overrightarrow{h}_{v,t} \oplus \overleftarrow{h}_{v,t}$. Two such planes process the arterial (A) and portal venous (V) phases separately with the same backbone and hyperparameters, and the hidden states from the two planes are averaged after the LSTM layers before classification. This construction isolates the two hypotheses the paper tests: phase should be a parallel structure rather than a concatenation channel, and temporal dynamics should be read in both directions. The ablation study then removes each component in turn to attribute the AUC gain, including a variant with early inter-plane connections that performs slightly worse and is therefore discarded.

What would settle it

A reader could re-run the same 10-fold protocol on a cohort where every patient has scans at exactly the same post-surgery intervals; if the AUC advantage of MPBD-LSTM over PredRNN-V2 shrinks or vanishes, the claim that the bidirectional multi-plane structure is responsible would be falsified.

Watch

Extended reading notes

Core claim

The central claim is an architectural result: the best way to extend a 4D spatiotemporal model to 5D multi-phase contrast-enhanced CT is not to concatenate the arterial and portal venous phases into channels, but to run one E3D-LSTM backbone per phase and fuse their outputs after the recurrent layers, while replacing the unidirectional temporal pass with a forward-and-backward pair whose hidden states are summed. On the paper's 170-patient evaluation, MPBD-LSTM reaches AUC 0.790, and the ablations attribute the gain to both choices: removing the multi-plane structure and concatenating phases lowers the score to 0.774, and switching the bidirectional connections back to unidirectional lowers it to 0.768. The time-series and multi-phase ablations further show that using all three timestamps beats any single timestamp, and that the venous phase alone (0.752) is much stronger than the arterial phase alone (0.653), so the full five-dimensional input is what the model needs.

Load-bearing premise

The model's evaluation assumes the first three follow-up scans T0, T1, T2 can be treated as the same aligned time series for every patient, even though the dataset allows 2 to 6 scans at irregular post-surgery intervals (Section 2.1, Section 3.1); if scan timing correlates with disease or hospital routine, the temporal signal may be partly scheduling, not biology.

Editorial extensions

If this is right

  • On the reported data, MPBD-LSTM outperforms all four baselines by AUC, with the largest gap over the CNN-only SimVP (0.790 vs 0.662), suggesting recurrent temporal models fit this task better than pure convolutional translators.
  • The ablation results imply that concatenating arterial and venous phases loses slice-position correspondence; using two parallel planes recovers about 0.016 AUC over concatenation.
  • The ablation results imply that bidirectional temporal connections add about 0.022 AUC over the unidirectional version, supporting the value of reading the scan history in both time directions.
  • Single-timestamp performance rises from T0 (0.660) to T2 (0.709), and using all three timestamps (0.790) beats any single time, supporting the clinical value of serial follow-up scans.
  • The portal venous phase alone (0.752) dominates the arterial phase alone (0.653), consistent with the radiological view that venous-phase blood supply is more informative for colorectal liver metastasis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the authors did not run: re-index the time axis by days since surgery rather than visit number. If the AUC drops, the model is partly learning follow-up scheduling; if it holds, the learned features are tied to biological change.
  • The phase-plane design should transfer to other multi-phase or multi-sequence imaging, such as multiparametric MRI, where the same choice, parallel planes versus channel concatenation, can be evaluated directly.
  • The false-negative concentration in small livers suggests a preprocessing remedy: liver-volume normalization, or training with size-stratified losses, might recover a meaningful share of the missed positives.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents MPBD-LSTM, a multi-plane bidirectional 3D LSTM model for predicting colorectal liver metastases (CRLM) from time-series multi-phase contrast-enhanced CT (CECT) scans. The authors construct a retrospective dataset of 269 patients from two cohorts, select 170 patients with at least three follow-up scans, and use the first three timestamps (T0, T1, T2) of the arterial (A) and portal venous (V) phases as input. They compare MPBD-LSTM against E3D-LSTM, PredRNN-V2, SaConvLSTM, and SimVP under 10-fold cross-validation, reporting an AUC of 0.790 for MPBD-LSTM, the highest among the tested models. Ablation studies examine the contributions of the bi-directional connections, the multi-plane structure, individual timestamps, and individual phases. The code is made publicly available.

Significance. If the reported results hold, the paper makes a useful contribution to medical image analysis by demonstrating a concrete architecture for handling 5D spatiotemporal data (time, phase, and 3D volume) in a clinically relevant prediction task. The construction of a private CECT dataset, the public release of code, and the ablation studies on phase and timestamp usage are valuable assets. The central claim is an empirical benchmark result rather than a theoretical derivation, and the comparison across four strong baselines is informative for practitioners. However, the significance is currently limited by the lack of statistical rigor in the evaluation and by unresolved questions about data preprocessing leakage, which could materially affect the reported ranking.

major comments (4)
  1. [Section 3.1] The paper does not specify whether the data augmentation duplication is applied before or after the 10-fold split described in Section 3.2. Section 3.1 states that 'we selected and duplicated 60% of positive cases and 20% of negative cases by applying Standard Scale Jittering (SSJ)' on the full set of 170 patients. If this duplication occurs before splitting, jittered copies of the same patient can appear in both training and test folds, allowing the model to memorize patient-specific liver texture rather than learn generalizable CRLM predictors. Because MPBD-LSTM and the baselines have different parameterizations, such leakage would not affect all models equally, so the reported AUC advantage of 0.790 over 0.765 could be an artifact. The authors must clarify that augmentation is performed only on each training fold after splitting, or alternatively report results with patient-level grouped folds and post-split augmentation.
  2. [Section 4, Table 2] The central comparison reports only point AUC values without confidence intervals, standard deviations across folds, or statistical significance tests. The gap between MPBD-LSTM (0.790) and PredRNN-V2 (0.765) is modest, and on a dataset of only 170 patients this difference may fall within the noise of 10-fold cross-validation. The authors should report per-fold AUC values, confidence intervals (e.g., bootstrap or DeLong), and significance tests (e.g., paired DeLong test or Wilcoxon signed-rank test) for the main comparison and for the ablations in Tables 3 and 4.
  3. [Section 2.1 and Section 3.1] The input construction treats T0, T1, and T2 as aligned timestamps across patients, yet Section 2.1 states that patients have between 2 and 6 follow-up scans at variable intervals after surgery. If the temporal spacing between the first three scans differs across patients, the model may learn patterns related to follow-up scheduling rather than disease progression. The authors should report the distribution of inter-scan intervals and either align scans by actual time or perform a sensitivity analysis restricted to patients with similar follow-up schedules.
  4. [Section 3.1 and Section 3.2] Several key design choices appear to be tuned on the same 170-patient dataset without a nested validation procedure: the number of timestamps (three), the augmentation duplication ratios (60% positive, 20% negative), the input slice depth (64), and the LSTM hidden unit count (16). This creates an overfitting risk that is not addressed by the reported 10-fold cross-validation. The authors should either use a nested cross-validation or an independent validation set for these choices, or explicitly justify each choice by reference to prior work rather than dataset-specific tuning.
minor comments (6)
  1. [Abstract] The word 'bi-directioal' appears to be a typo for 'bi-directional'.
  2. [Table 1] The table caption contains a typo: 'Characreristics' should be 'Characteristics'.
  3. [Section 2.2, Eq. (1)-(2)] Equation (1) uses a general timestamp 't' while the surrounding text and Eq. (2) use 't0'; please make the notation consistent and clarify that Eq. (1) holds for every timestamp.
  4. [Section 3.1] The description of the duplication process is ambiguous: it is not clear whether 'duplicated' means each selected case is copied once or multiple times, nor how the final class counts after augmentation compare to the original 49 positive and 121 negative cases. Please state the exact post-augmentation counts.
  5. [Section 4, Table 2] The text refers to supplementary material for accuracy, sensitivity, and specificity, but no supplementary material is available in the arXiv submission; please include these metrics in the main paper or make the supplement publicly available.
  6. [Section 4, Error Analysis] The claim that 'small liver size is present in most of the false negative cases' is not supported by quantitative data; please provide a statistical comparison of liver sizes between correctly and incorrectly predicted positive cases.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claim is an empirical comparison with measured AUC scores, not a derivation that reduces to its inputs.

full rationale

The paper makes no formal derivation whose conclusion equals an input. MPBD-LSTM is defined as an architectural variant of E3D-LSTM with bi-directional LSTM connections and a multi-plane phase-handling structure, and its performance is measured on 10-fold cross-validation against baselines and ablations. The reported AUC values are empirical outcomes, not quantities forced by model definition: the model architecture does not encode the target labels or the evaluation metric, and no parameter is fitted to the test set in a way that would make the comparison true by construction. The paper contains no load-bearing self-citations; its references are to external prior work and standard components. The augmentation-before-split concern raised in review is a potential methodological leakage risk, but it is not circular reasoning: even if duplication occurred before splitting, it would affect the validity of the empirical comparison without making any prediction equivalent to its input by definition. Accordingly, no circular step can be exhibited with a specific reduction, and the paper is self-contained as an empirical study. Score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on dataset assumptions and hyperparameter choices rather than mathematical axioms. No new physical entities are introduced; MPBD-LSTM is a neural architecture, not a postulated natural object.

free parameters (5)
  • Number of LSTM hidden units = 16
    Set to 16 due to GPU memory constraints; affects model capacity and may influence the reported AUC comparison.
  • Learning rate = 5e-4
    Chosen by the authors for training; no sweep or sensitivity analysis is reported.
  • Number of timestamps used = 3
    Patients with at least three scans are selected, and the input uses exactly three timestamps; patients with more scans are not fully exploited.
  • Input slice depth = 64
    Spline Interpolated Zoom uniformly selects 64 slices per scan, which could discard information outside the selected volume.
  • Augmentation duplication ratios = 60% positive, 20% negative
    Hand-selected duplication of positives and negatives to address imbalance; this choice affects the training distribution and can inflate validation scores if applied before splitting.
assumptions (4)
  • domain assumption The first three follow-up scans are temporally comparable across patients.
    Patients have 2 to 6 scans at irregular intervals (Section 2.1), yet the model uses a fixed 3-timestamp input (Section 3.1). If visit timing differs, the time axis mixes disease progression with scheduling differences.
  • domain assumption The arterial and portal venous phases contain the predictive signal, while the plain phase can be ignored.
    Section 2.2 states the P phase is only relevant when tumors are significantly present, but no experiment is shown to verify this for early CRLM detection.
  • domain assumption The manual label of CRLM within two years is correct ground truth.
    Section 2.1 says cases were manually labeled based on whether liver metastases appeared within two years; there is no independent pathology or imaging central review described.
  • domain assumption Augmentation operations preserve the predictive signal.
    Section 3.1 applies random rotation, mixup, and Standard Scale Jittering to all phases and timestamps; these transformations could alter subtle radiologic features relevant to metastasis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans." pith.science (2026). https://pith.science/paper/2LNU5DR2

@misc{pith2026241201973,
  author       = {Pith},
  title        = {Pith review of: MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2LNU5DR2}},
  note         = {Machine review of arXiv:2412.01973}
}
read the original abstract

Colorectal cancer is a prevalent form of cancer, and many patients develop colorectal cancer liver metastasis (CRLM) as a result. Early detection of CRLM is critical for improving survival rates. Radiologists usually rely on a series of multi-phase contrast-enhanced computed tomography (CECT) scans done during follow-up visits to perform early detection of the potential CRLM. These scans form unique five-dimensional data (time, phase, and axial, sagittal, and coronal planes in 3D CT). Most of the existing deep learning models can readily handle four-dimensional data (e.g., time-series 3D CT images) and it is not clear how well they can be extended to handle the additional dimension of phase. In this paper, we build a dataset of time-series CECT scans to aid in the early diagnosis of CRLM, and build upon state-of-the-art deep learning techniques to evaluate how to best predict CRLM. Our experimental results show that a multi-plane architecture based on 3D bi-directional LSTM, which we call MPBD-LSTM, works best, achieving an area under curve (AUC) of 0.79. On the other hand, analysis of the results shows that there is still great room for further improvement.

Figures

Figures reproduced from arXiv: 2412.01973 by the authors.

Figure 1
Figure 1. Representative slices from 3D CT images of different patients in our dataset, at A/V phases and timestamps T0, T1, T2 (cropped to 256x256 for better view). When patients undergo CECT scans to detect CRLM, typically three phases are captured: the unenhanced plain scan phase (P), the portal venous phase (V), and the arterial phase (A). The P phase provides the basic shape of the liver tissue, while the V and A phases … view at source ↗
Figure 2
Figure 2. (a) The general structure of MPBD-LSTM. The yellow plane is the 1st plane which is used to process the portal venous phase CT scans, and the gray plane is the second one used to process the arterial phase CT scans. µ is the average function. (b) The inner structure of a 3D-LSTM module. Blue arrow stands for the forward pass which generates the output of −→h v,t0 and red arrow indicates the backward pass generating t… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

  1. [1]

    Ba, J., Caruana, R.: Do deep nets really need to be deep? Advances in neural information processing systems 27 (2014)

    work page 2014

  2. [2]

    Ad- vances in neural information processing systems 29 (2016)

    Chen, J., Yang, L., Zhang, Y., Alber, M., Chen, D.Z.: Combining fully convolu- tional and recurrent neural networks for 3d biomedical image segmentation. Ad- vances in neural information processing systems 29 (2016)

    work page 2016

  3. [3]

    arXiv preprint arXiv:1801.02143 (2018)

    Cui, Z., Ke, R., Pu, Z., Wang, Y.: Deep bidirectional and unidirectional lstm recurrent neural network for network-wide traffic speed prediction. arXiv preprint arXiv:1801.02143 (2018)

    arXiv 2018

  4. [4]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Gao, Z., Tan, C., Wu, L., Li, S.Z.: Simvp: Simpler yet better video prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3170–3180 (2022)

    work page 2022

  5. [5]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.Y., Cubuk, E.D., Le, Q.V., Zoph, B.: Simple copy-paste is a strong data augmentation method for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2918–2928 (2021)

    2021

  6. [6]

    World journal of surgical oncology 20(1), 80 (2022)

    Hao, M., Li, H., Wang, K., Liu, Y., Liang, X., Ding, L.: Predicting metachronous liver metastasis in patients with colorectal cancer: development and assessment of a new nomogram. World journal of surgical oncology 20(1), 80 (2022)

    work page 2022

  7. [7]

    In: Proceedings of the AAAI conference on artificial intelligence

    Kim, S., An, S., Chikontwe, P., Park, S.H.: Bidirectional rnn-based few shot learn- ing for 3d medical image segmentation. In: Proceedings of the AAAI conference on artificial intelligence. vol. 35, pp. 1808–1816 (2021)

    work page 2021

  8. [8]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    arXiv 2014

Show all 18 references
  1. [9]

    In: Proceedings of the AAAI conference on artificial intel- ligence

    Lin, Z., Li, M., Zheng, Z., Cheng, Y., Yuan, C.: Self-attention convlstm for spa- tiotemporal prediction. In: Proceedings of the AAAI conference on artificial intel- ligence. vol. 34, pp. 11531–11538 (2020)

    2020

  2. [10]

    In: StatPearls [Internet]

    Patel, P.R., De Jesus, O.: Ct scan. In: StatPearls [Internet]. StatPearls Publishing (2022) 10 X. Li et al

    2022

  3. [11]

    Advances in neural information processing systems 28 (2015)

    Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., Woo, W.c.: Convolutional lstm network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems 28 (2015)

    2015

  4. [12]

    In: International conference on learning representations (2019)

    Wang, Y., Jiang, L., Yang, M.H., Li, L.J., Long, M., Fei-Fei, L.: Eidetic 3d lstm: A model for video prediction and beyond. In: International conference on learning representations (2019)

    2019

  5. [13]

    Advances in neural in- formation processing systems 30 (2017)

    Wang, Y., Long, M., Wang, J., Gao, Z., Yu, P.S.: Predrnn: Recurrent neural net- works for predictive learning using spatiotemporal lstms. Advances in neural in- formation processing systems 30 (2017)

    2017

  6. [14]

    Wang, Y., Wu, H., Zhang, J., Gao, Z., Wang, J., Yu, P.S., Long, M.: PredRNN: A recurrent neural network for spatiotemporal predictive learning (2021)

    2021

  7. [15]

    World journal of gastroenterology: WJG 17(42), 4654 (2011)

    Xu, L.H., Cai, S.J., Cai, G.X., Peng, W.J.: Imaging diagnosis of colorectal liver metastases. World journal of gastroenterology: WJG 17(42), 4654 (2011)

    2011

  8. [16]

    OncoTargets and therapy 13, 11645 (2020)

    Yu, X., Zhu, L., Liu, J., Xie, M., Chen, J., Li, J.: Emerging role of immunotherapy for colorectal cancer with liver metastasis. OncoTargets and therapy 13, 11645 (2020)

    2020

  9. [17]

    arXiv preprint arXiv:1710.09412 (2017)

    Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)

    2017 arXiv

  10. [18]

    In: Predictive Intelligence in Medicine: Third International Workshop, PRIME 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 8, 2020, Proceedings 3

    Zunair, H., Rahman, A., Mohammed, N., Cohen, J.P.: Uniformizing techniques to process ct scans with 3d cnns for tuberculosis prediction. In: Predictive Intelligence in Medicine: Third International Workshop, PRIME 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October...

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.