Pith. sign in

REVIEW 3 major objections 6 minor 16 references

Oriented histogram-based vector field embedding for characterizing 4D CT data sets in radiotherapy

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that breathing motion captured in 4D CT scans can be compressed into compact oriented histograms that cluster patients by breathing-pattern similarity, without any supervised labels.

desk verdict A tidy, honestly-scoped feasibility study: the new encoding is sensible, but the clustering claim is only supported by anecdote and repeat-scan grouping, not by quantitative validation. read the letter →

arxiv 2411.16314 v1 pith:NLYDO6IA submitted 2024-11-25 physics.med-ph cs.CV

Frederic Madesta , Lukas Wimmert , Tobias Gauer , René Werner , Thilo Sentker This is my paper · ORCID
classification physics.med-phcs.CV
keywords 4DCTdeformableimageregistrationorientedhistogramUMAPlungradiotherapyvectorfieldembeddingunsupervisedclusteringbreathingmotion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that high-dimensional deformation vector fields from deformable image registration of 4D CT scans can be reduced to compact oriented histograms, one per motion phase, without losing the patient-specific breathing signature. It reports that after a further autoencoder compression and a UMAP embedding into two dimensions, patients cluster by breathing-pattern similarity rather than by lung size, reconstruction binning, or even the absence of one lung. If this holds, a clinician could compare a new patient's pre-treatment motion data against a database of previous patients, flag unusual motion, or link similar patients to treatment outcomes. The workflow is unsupervised throughout and is demonstrated on 71 in-house and 33 external 4D CT data sets; repeat scans of the same patient mostly landing in the same cluster is the central internal validation.

What carries the argument

The load-bearing object is the oriented histogram, an extension of histograms of oriented optical flow (a discretized tally of motion directions originally used for action recognition) to 3D displacement fields. The spherical transform $r,\theta,\phi$ followed by $r$-weighted binning into an $8\times16$ array per phase is what converts a vector field of size $[I_x,I_y,I_z,3]$ into a patient-level summary; the autoencoder and UMAP then operate on these summaries. The machinery deliberately discards spatial localization while preserving the distribution of motion directions and magnitudes, and it is this distribution that the cluster analysis links to breathing-pattern similarity.

What would settle it

Use the 33 external 4D CTs to compute embedding distances between repeat scans of the same patient and scans of different patients: if same-patient pairs are not systematically closer than different-patient pairs, the meaningful-cluster claim collapses. A second check is to halve the histogram bin size (from $\pi/8$ to $\pi/16$) and see whether cluster separation improves; if it does not, the $8\times16$ representation has already saturated the available motion information.

Watch

Extended reading notes

Core claim

The central claim is that a weighted 2D oriented histogram built from the spherical coordinates of each voxel displacement can serve as a compact summary statistic of breathing-motion similarity in lung 4D CT data. Each displacement vector $v=(x,y,z)$ is converted to radius $r$, polar angle $\theta=\arccos(z/r)$, and azimuthal angle $\phi=\operatorname{arctan2}(y/x)$; vectors inside the lung mask are tallied into bins of size $\pi/8$ over $\theta\in[0,\pi]$ and $\phi\in(-\pi,\pi]$, each bin is weighted by $r$, and the result is normalized by lung voxel count. Repeating this for the nine phase-to-reference registrations yields a $9\times8\times16$ code per patient. A lightweight unsupervised autoencoder densifies the code, UMAP embeds it in 2D, and the resulting clusters show similar breathing patterns, with most repeat 4D CTs of the same patient clustering together and two post-pneumonectomy patients still falling into interpretable places.

Load-bearing premise

The method assumes that a fixed $8\times16$ histogram of displacement directions, weighted by vector length and normalized by lung voxel count, keeps enough patient-specific motion information that distances in the UMAP embedding reflect true breathing similarity; this is checked only by visual cluster inspection, with no quantitative separation measure.

Editorial extensions

If this is right

  • A new patient's 4D CT can be embedded into the same UMAP space using only pre-treatment planning data, enabling prospective comparison with previously treated patients without any supervised training.
  • The histogram encoding is not lung-specific, so the same reduction and embedding pipeline applies to other registration tasks, anatomies, or physiological processes.
  • Because repeat 4D CTs of the same patient mostly cluster, the embedding is stable enough to serve as a longitudinal check on whether a patient's breathing motion is changing between scans.
  • Any deformable registration algorithm's output can be fed into the pipeline, and the paper reports using the embedding to select patient-specific registration hyperparameters based on nearest neighbors in the UMAP space.
  • Clustering appears independent of whether the 4D CT was reconstructed by phase-based or amplitude-based binning, so the representation captures motion characteristics beyond the reconstruction protocol.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer the same histogram representation could serve as a retrieval index for treatment-plan libraries, ranking past patients by motion similarity before any dose simulation is run; the paper names this as motivation but does not test retrieval performance.
  • A natural extension the authors do not report is correlating cluster membership with dosimetric or clinical endpoints; if clusters track motion similarity, they may also track motion-induced target-miss risk, but this connection is my inference, not their result.
  • Because localization is discarded, I expect the method to be blind to spatial rearrangements of motion (for instance, a whole-tumor translation versus a local diaphragm spike), so a multiscale version that bins displacement fields in subregions is a direct testable upgrade.
  • The visual cluster analysis suggests an obvious quantitative check, such as repeat-scan nearest-neighbor accuracy or silhouette score, which the paper leaves for future work; I infer such a score would make the method easier to trust and tune.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an unsupervised dimensionality-reduction pipeline for deformable image registration (DIR) vector fields from 4D CT lung data. Each vector field is transformed to a patient-specific lung basis, converted to spherical coordinates, and binned into a radius-weighted 8×16 oriented histogram per phase; the nine histograms of a 10-phase 4D CT are then compressed with a lightweight autoencoder and embedded in 2D with UMAP (n_neighbors=5, cosine metric, DensMAP). The authors demonstrate the approach on 71 in-house and 33 external 4D-Lung data sets, and argue from visual inspection of the UMAP plots, and from repeat 4D CTs of the same patient clustering together, that the embedding captures breathing-pattern similarity. They also state that the method is a general, computationally efficient way to reduce DIR vector fields for patient similarity analysis in radiotherapy.

Significance. If the central claim holds, the paper would provide a practical, fully unsupervised tool for comparing lung-motion patterns across patients using only planning 4D CT data, with potential applications in treatment-plan review, outcome analysis, and even hyperparameter selection for DIR (as claimed in the Learn2Reg context). The methodological idea of encoding directional motion statistics as oriented histograms is a reasonable and computationally attractive extension of 2D histogram-of-optical-flow methods to 3D registration fields, and the authors are right that this specific combination is not common in the radiotherapy literature. The paper is written clearly and the pipeline is straightforward to implement. However, the evidential basis for the main claim is currently qualitative: the paper relies on visual cluster inspection and on same-patient repeat scans, neither of which isolates breathing-motion similarity from anatomical or acquisition-related confounds. The absence of quantitative cluster-quality metrics, baseline comparisons, or stability analyses means the 'meaningful embedding' claim is not yet distinguished from clustering artifacts of a small dataset.

major comments (3)
  1. [Section 3, Fig. 2, and Section 4] The central claim that clusters reflect breathing-pattern similarity is supported only by qualitative visual inspection ('patients with vector fields with similar characteristics ... tend to form a cluster') and by the observation that repeat 4D CTs of the same patient in the 4D-Lung set mostly cluster together. This is load-bearing because the abstract and the Discussion ('the embedding appears to be meaningful') assert exactly this claim. No quantitative cluster-quality metric (e.g., silhouette score, Davies-Bouldin index, gap statistic), no comparison to baseline embeddings (e.g., PCA, t-SNE, or a simple distance on the histograms without the autoencoder), and no permutation/randomization test are provided. With only n=71 and n=33 data points and UMAP n_neighbors=5, apparent clusters can easily arise from sampling noise. I request a quantitative evaluation that separates motion-specific structure from noise, for example by comparing cluster separation of real histograms against histograms with randomly permuted directions, and by reporting a cluster validity index for several hyperparameter choices.
  2. [Section 2.3, Eqs. (1)-(2), and Section 3] The pipeline does not isolate breathing motion from correlated nuisance structure. The spherical coordinate frame is defined by the PCA-based lung basis of each patient's lung mask, so the representation is patient-specific in anatomy; the 4D-Lung repeat scans share not only the patient but also scanner, reconstruction protocol, and mask geometry. Therefore, same-patient clustering or visually similar clusters could be driven by these confounds even if the histograms carried zero breathing information. To support the motion-specificity claim, the authors should include a control analysis, for instance an ablation in which the histogram is computed from a randomized or patient-averaged vector field, or an explicit comparison of clustering behavior across different reconstruction binning (phase- vs. amplitude-based) with a statistical test of separation, rather than the current 'was not observed' statement in Section 3.
  3. [Section 2.3-2.4 and Section 3] The method contains several free parameters whose influence on the clustering is not investigated: the bin width π/8 for θ and φ, the radius weighting and lung-voxel normalization, the choice of the reference phase j=3, the autoencoder architecture and training hyperparameters, and UMAP's n_neighbors=5, cosine metric, and DensMAP. The Discussion explicitly delegates the angular-resolution question to future research, but the central claim of meaningful clustering is conditional on these choices. At minimum, a sensitivity analysis over a few reasonable bin sizes (e.g., π/4, π/8, π/12) and reference phases, with a cluster-quality metric, is needed to demonstrate that the reported clusters are not an artifact of one arbitrary setting. In addition, the autoencoder is described only as 'lightweight'; its architecture, number of parameters, training epochs, and reconstruction error are not given, which harms reproducibility.
minor comments (6)
  1. [Section 2.1] The text says '71 lung cancer patients' for the in-house data set and later '33 4D CT data sets' for 4D-Lung; please clarify whether the in-house number refers to patients or to 4D CT data sets, since the 4D-Lung subset comprises 33 data sets from only 8 patients and the two cohorts are not directly comparable in that respect.
  2. [Section 2.3] The spherical-coordinate equations use x, y, z without defining that these are the components of Vlung; please make the notation explicit (e.g., v = (x, y, z)^T ∈ Vlung) to avoid confusion.
  3. [Section 3 and Fig. 2] The caption and text do not specify the color coding for the in-house UMAP scatter plot; if the top panel is also colored by patient, state this, otherwise describe the markers used. This matters because the reader cannot otherwise distinguish whether the in-house clusters are patient-related or motion-related.
  4. [Section 3] The claim that clustering is independent of phase- vs. amplitude-based reconstruction is stated as 'was not observed' without any quantitative measure. If this claim is retained, it should be supported by the same cluster-quality analysis requested in the major comments.
  5. [Section 4, Learn2Reg paragraph] The final paragraph of the Discussion introduces a claimed prior application of the embedding concept to DIR-hyperparameter selection during the Learn2Reg challenges, but gives no reference to the authors' own prior work or challenge report. Please provide a citation or clarify the status of that claim, since it is material to the novelty statement that 'the present publication is the first to explain the underlying methodology.'
  6. [Abstract and Section 2.3] The abstract says 'voxel-wise spherical coordinate transformation'; since the transformation is applied to each displacement vector at each voxel (not to the voxel coordinate itself), consider rewording to 'per-vector spherical coordinate transformation' to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the embedding is unsupervised, no parameter is fitted to the claimed outcome, and the self-cited registration method is a preprocessing tool rather than a load-bearing premise.

full rationale

The paper's derivation chain is not circular. The oriented-histogram representation is computed directly from DIR displacement vectors by spherical coordinate transformation and r-weighted (theta, phi) binning (Section 2.3); no target labels are used and no parameter is fitted to the clustering result. The autoencoder is trained solely with an L2 reconstruction loss, and UMAP uses standard parameters, so the 2D embedding is an unsupervised function of the histograms. The only self-citation is the variational registration algorithm of Ref. [6] used to generate the input vector fields; that is a preprocessing tool with external evaluation, not a premise whose truth is assumed from the present authors. The validation in Section 3 is based on visual inspection of clusters and on repeat 4D CTs of the same patient clustering together. This evidence is self-referential in the sense that UMAP places similar histogram encodings nearby, but it is not a reduction of a predicted quantity to an input by construction; it is an observational consistency check. The concerns about confounds (shared anatomy, scanner, reconstruction, small sample size) and the absence of quantitative cluster metrics are validity and interpretability limitations, not circularity. Therefore no circular step is identified.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical entities. It uses several hand-chosen design parameters (bin width, weighting, UMAP settings, reference phase, autoencoder architecture) and relies on two domain assumptions: the lung-mask principal axes form a comparable coordinate system, and a coarse direction histogram preserves motion information relevant for clustering. These assumptions are not quantitatively validated.

free parameters (5)
  • Angular bin width π/8 for θ and φ = π/8
    Chosen in Section 2.3; resolution of oriented histogram is a free design choice and authors state effect of smaller bins is future work.
  • Radius weighting r and normalization by lung voxel count = weight = r; normalize by lung voxels
    Section 2.3 design choices that alter histogram shape and comparability across lung sizes.
  • UMAP hyperparameters: n_neighbors=5, cosine metric, DensMAP = n_neighbors=5, metric=cosine
    Set in Section 2.4; no sensitivity analysis provided.
  • Fixed reference phase j=3 (mid-ventilation) = phase index 3
    Section 3; chosen to obtain positive and negative z-displacements, affects which 9 histograms are generated.
  • Autoencoder architecture and training hyperparameters = unspecified
    Section 2.3; a lightweight convolutional autoencoder with L2 loss is described but architecture details and training settings are deferred to code.
assumptions (5)
  • domain assumption Lung-mask principal axes form a stable patient-specific anatomical basis for comparing vector fields across patients.
    Section 2.3: Blung is computed from eigenvectors of lung mask covariance; if this basis is unstable, histograms are not comparable.
  • domain assumption The r-weighted (θ, φ) histogram retains sufficient motion information for clustering meaningful breathing patterns.
    Section 2.3: localization information is discarded by design; the paper assumes direction and magnitude distribution suffices.
  • standard math UMAP embedding of autoencoded histograms preserves neighborhood structure relevant to breathing similarity.
    Section 2.4 relies on UMAP theory; DensMAP regularizes density.
  • domain assumption Repeat 4D CTs of the same patient should cluster together, used as a proxy for embedding quality.
    Section 3: patient-wise clustering of repeat scans is treated as evidence that embedding is meaningful.
  • domain assumption Vector fields from deformable registration accurately represent lung motion from 4D CT.
    Section 2.2 uses a demons-based variational registration; errors in DIR propagate to histograms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Oriented histogram-based vector field embedding for characterizing 4D CT data sets in radiotherapy." pith.science (2026). https://pith.science/paper/NLYDO6IA

@misc{pith2026241116314,
  author       = {Pith},
  title        = {Pith review of: Oriented histogram-based vector field embedding for characterizing 4D CT data sets in radiotherapy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NLYDO6IA}},
  note         = {Machine review of arXiv:2411.16314}
}
read the original abstract

In lung radiotherapy, the primary objective is to optimize treatment outcomes by minimizing exposure to healthy tissues while delivering the prescribed dose to the target volume. The challenge lies in accounting for lung tissue motion due to breathing, which impacts precise treatment alignment. To address this, the paper proposes a prospective approach that relies solely on pre-treatment information, such as planning CT scans and derived data like vector fields from deformable image registration. This data is compared to analogous patient data to tailor treatment strategies, i.e., to be able to review treatment parameters and success for similar patients. To allow for such a comparison, an embedding and clustering strategy of prospective patient data is needed. Therefore, the main focus of this study lies on reducing the dimensionality of deformable registration-based vector fields by employing a voxel-wise spherical coordinate transformation and a low-dimensional 2D oriented histogram representation. Afterwards, a fully unsupervised UMAP embedding of the encoded vector fields (i.e., patient-specific motion information) becomes applicable. The functionality of the proposed method is demonstrated with 71 in-house acquired 4D CT data sets and 33 external 4D CT data sets. A comprehensive analysis of the patient clusters is conducted, focusing on the similarity of breathing patterns of clustered patients. The proposed general approach of reducing the dimensionality of registration vector fields by encoding the inherent information into oriented histograms is, however, applicable to other tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 15 canonical work pages

  1. [1]

    Oriented histogram-based vector field embedding for characterizing 4D CT data sets in radiotherapy

    INTRODUCTION In current lung cancer radiotherapy (RT), advanced imaging techniques play a pivotal role in the treatment process to op- timize patient outcomes [1]. The main objective remains to minimize the exposure of healthy tissue while ensuring that the planned dose is delivered to the target volume. Considera- tion of the physiological motion of lung...

    work page Pith review arXiv 2024

  2. [2]

    Datasets We utilized anonymized 4D CT data from 71 lung cancer patients (in-house data set collected between 2015 and 2020)

    METHODS 2.1. Datasets We utilized anonymized 4D CT data from 71 lung cancer patients (in-house data set collected between 2015 and 2020). 4D CT image reconstruction was performed using phase- based (34 cases) and amplitude-based (37 cases) binning techniques, with the resulting 4D CT dataset comprising nph = 10 3D CT phase images. Additionally, we employe...

    work page 2015

  3. [3]

    RESULTS All patient datasets underwent the registration and dimension- ality reduction process outlined in fig. 1. We selected the mid ventilation phase as the fixed image, i.e.,IF = Ij=3, to obtain both positive and negative z-displacements, thus utilizing the full θ-range of the oriented histograms. Afterward, the en- coded data was subjected to analysi...

    work page

  4. [4]

    DISCUSSION AND CONCLUSIONS The presented embedding of vector fields computed by DIR of 4D CT data by applying oriented histograms to reduce di- mensionality shows that the methodological approach is fea- sible to allow for analyses of clusters in the UMAP space. The clusters exhibited similar breathing patterns of the pa- tients, illustrating that the emb...

    work page 2022

  5. [5]

    ACKNOWLEDGMENTS This work was funded by DFG research grant WE 6197/2-2 (project number 390567362)

    work page

  6. [6]

    The data evaluation was approved by the local ethics board, and the requirement to obtain written informed consent was waived [WF-82/18] between 2015 and 2020

    COMPLIANCE WITH ETHICAL STANDARDS The study was conducted retrospectively using human subject data. The data evaluation was approved by the local ethics board, and the requirement to obtain written informed consent was waived [WF-82/18] between 2015 and 2020

    work page 2015

  7. [7]

    Technological quality requirements for stereotac- tic radiotherapy,

    D. Schmitt, O. Blanck, T. Gauer, M. K. Fix, T. B. Brunner, J. Fleckenstein, B. Loutfi-Krauss, P. Manser, R. Werner, M. L. Wilhelm, W. W. Baus, and C. Mous- takis, “Technological quality requirements for stereotac- tic radiotherapy,” Strahlenther Onkol , vol. 196, no. 5, pp. 421–43, 2020

    work page 2020

  8. [8]

    Cor- respondence model-based 4d vmat dose simulation for analysis of local metastasis recurrence after extracranial sbrt,

    T. Sothmann, T. Gauer, M. Wilms, and R. Werner, “Cor- respondence model-based 4d vmat dose simulation for analysis of local metastasis recurrence after extracranial sbrt,” Phys Med Biol, vol. 62, no. 23, p. 9001, 2017

    work page 2017

Show all 16 references
  1. [9]

    Evaluation of 4-dimensional computed tomography to 4-dimensional cone-beam computed to- mography deformable image registration for lung can- cer adaptive radiation therapy,

    S. Balik, E. Weiss, N. Jan, N. Roman, W. C. Sleeman, M. Fatyga, G. E. Christensen, C. Zhang, M. J. Mur- phy, J. Luet al., “Evaluation of 4-dimensional computed tomography to 4-dimensional cone-beam computed to- mography deformable image registration for lung can- cer adaptive ...

    2013

  2. [10]

    Data from 4D lung imaging of NSCLC patients,

    G. D. Hugo, E. Weiss, W. C. Sleeman, S. Balik, P. J. Keall, J. Lu, and J. F. Williamson, “Data from 4D lung imaging of NSCLC patients,” Cancer Imaging Arch , vol. 10, p. K9, 2016

    2016

  3. [11]

    A longitudinal four-dimensional computed to- mography and cone beam computed tomography dataset for image-guided radiation therapy research in lung can- cer,

    ——, “A longitudinal four-dimensional computed to- mography and cone beam computed tomography dataset for image-guided radiation therapy research in lung can- cer,” Med Phys, vol. 44, no. 2, pp. 762–771, 2017

    2017

  4. [12]

    Estimation of lung motion fields in 4D CT data by variational non-linear intensity-based registra- tion: A comparison and evaluation study

    R. Werner, A. Schmidt-Richberg, H. Handels, and J. Ehrhardt, “Estimation of lung motion fields in 4D CT data by variational non-linear intensity-based registra- tion: A comparison and evaluation study.” Phys Med Biol, vol. 59, no. 15, pp. 4247–60, 2014

    2014

  5. [13]

    Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recog- nition of human actions,

    R. Chaudhry, A. Ravichandran, G. Hager, and R. Vidal, “Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recog- nition of human actions,” in 2009 IEEE conference on computer vision and pattern recognition . IEEE, 2009, pp. 1932–1939

    2009

  6. [14]

    N2d: (not too) deep clustering via clustering the local manifold of an autoencoded embed- ding,

    R. McConville, R. Santos-Rodriguez, R. J. Piechocki, and I. Craddock, “N2d: (not too) deep clustering via clustering the local manifold of an autoencoded embed- ding,” in 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, Jan. 2021

    2020

  7. [15]

    Umap: Uniform manifold approximation and projection for dimension reduction,

    L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” arXiv preprint arXiv:1802.03426, 2018

    2018 arXiv

  8. [16]

    Assessing single-cell transcriptomic variability through density- preserving data visualization,

    A. Narayan, B. Berger, and H. Cho, “Assessing single-cell transcriptomic variability through density- preserving data visualization,” Nat Biotechnol, vol. 39, no. 6, pp. 765–774, 2021

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.