Pith. sign in

REVIEW 5 major objections 6 minor 3 cited by

Medical Video Generation for Disease Progression Simulation

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a two-stage generative pipeline that turns a single medical image and a text description of a future disease state into a realistic disease-progression video.

desk verdict Useful new application of known diffusion/video components to a real medical gap, but the core claim of clinically ordered progression is asserted rather than demonstrated, and the evaluation needs major strengthening before the stated conclusions can hold. read the letter →

arxiv 2411.11943 v1 pith:US4MEG5B submitted 2024-11-18 cs.CV cs.AI

Xu Cao , Kaizhao Liang , Kuei-Da Liao , Tianren Gao , Wenqian Ye , Jintai Chen , Zhiguang Ding , Jianguo Cao
show 2 more authors
James M. Rehg Jimeng Sun
This is my paper · ORCID
classification cs.CVcs.AI
keywords medicalvideogenerationdiseaseprogressionsimulationdiffusionmodelsimage-to-videochestX-rayfundusphotographyskinlesionlongitudinalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a two-stage generative pipeline, Medical Video Generation (MVG), can take one patient image and a written description of a future disease state, and produce a short video showing an intermediate disease trajectory that is coherent, preserves the patient's unrelated features, and is clinically plausible. The motivation is that longitudinal imaging data for individual patients is scarce, so a model that fabricates realistic progression could fill missing follow-up images, support prognosis modeling, and aid medical education. The paper argues that its first stage, Progressive Image Editing (PIE), moves the image along the intended disease manifold by repeatedly inverting and denoising it under the same text prompt, and that its second stage interpolates between those states with a video diffusion model. Evidence offered includes disease-classification confidence scores that rise toward the target disease, a clinician A/B preference study over three video generation baselines, and a small comparison against real longitudinal COVID-19 chest X-rays.

What carries the argument

The load-bearing mechanism is recursive masked DDIM inversion followed by text-conditioned denoising, with a human-refined region-of-interest mask. Each PIE round takes the previous output image, partially inverts it to a noisy latent, denoises it under the fixed recaptioned prompt for the terminal disease state, and then blends the edited and original images inside the mask with hyperparameters $\beta_1$ and $\beta_2$. This is repeated $N$ times to produce discrete disease states, and a video transition model generates the frames between consecutive states. The paper's Proposition 1 identifies the recursion with maximization of $p(x^0_N \mid x^0_0, y)$ and its Proposition 2 asserts convergence with exponentially shrinking step size; both proofs are deferred to a supplementary document. The same mask controls the video transition model, so the pathological region is the only place where content is allowed to change.

What would settle it

Take a set of patients with real longitudinal images at two or more time points, run MVG from the first time point with the later report as prompt, and compare the intermediate generated states with the real intermediate follow-up images. If the monotone-progression claim is right, classification confidence should rise monotonically across generated stages and the generated states should be closer to the real follow-ups than states produced with a shuffled or unrelated prompt of equal length; if unrelated prompts produce the same trajectory, the movement is text-conditioned editing rather than disease progression.

Watch

Extended reading notes

Core claim

The central claim is that disease progression can be simulated without any longitudinal training video by chaining two off-the-shelf generative components. First, a large language model recaptions a clinician-provided terminal report into a prompt; then PIE repeatedly applies masked denoising diffusion inversion to the same image, with the same text conditioning, so that each round nudges the image a bounded step toward the target disease state. The paper states that this recursion maximizes the posterior probability of the final state given the initial image and text prompt, and that the step size decays exponentially, so the sequence converges. The second stage uses a video latent diffusion model to interpolate smooth transitions between neighboring states. The claimed outcome is a video that edits only the disease-relevant region, preserves the patient identity encoded in the rest of the image, and matches what physicians expect of real progression.

Load-bearing premise

The argument rests on the premise that repeatedly inverting and denoising an image with the same future-state text prompt moves it monotonically along a clinically meaningful disease trajectory rather than merely applying arbitrary text-conditioned edits; the proof of this premise is stated in the paper but deferred to a supplementary that is not included.

Editorial extensions

If this is right

  • If MVG is right, a hospital with only a single historical image and a future clinical report can synthesize the missing intermediate images for that patient, effectively augmenting sparse longitudinal records.
  • Generated disease-state sequences can be used to visually preview a predicted trajectory before it happens, giving clinicians and patients a concrete look at how a condition such as diabetic retinopathy or cardiomegaly may evolve.
  • Because PIE is prompt-controlled, the same framework can produce alternative trajectories for the same starting image, supporting what-if comparisons of different disease courses or treatment regimens.
  • The framework's reported convergence means long progressions can be generated in many small steps without accumulating unbounded distortion, keeping unrelated anatomy stable while the disease feature develops.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit, testable corollary of the paper's theory is that classifier confidence for the target disease should be monotone nondecreasing in the progression step $n$; the paper reports endpoint confidence but not per-step monotonicity, so a per-step curve would be a direct check.
  • If the progression-direction claim holds, the generated intermediate states could serve as pseudo-longitudinal training pairs for prognosis models, effectively converting cross-sectional datasets into synthetic cohorts; that use goes beyond what the paper validates.
  • The reliance on a human-edited ROI mask suggests the method's ceiling may be set by localization quality; an automatic mask from segmentation models would need to be as reliable as the human mask for the method to scale.
  • The same two-stage recipe might transfer to other imaging modalities with a text interface, such as OCT or pathology, but the paper only demonstrates chest X-ray, fundus, and skin images.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Medical Video Generation (MVG), a two-stage framework for simulating disease progression videos from a single medical image and a clinical report. Stage 1, Progressive Image Editing (PIE), iteratively applies DDIM inversion and denoising with a fixed target text condition and a human-refined ROI mask to generate a discrete sequence of disease-state images. Stage 2 uses the SEINE video diffusion model to interpolate smooth transitions between these states. The authors claim theoretical support (Propositions 1 and 2) that recursive PIE maximizes the posterior p(x0_N | x0_0, y) and converges to a bounded edit, and they report experiments on chest X-ray, fundus, and skin imaging comparing against Stable Video Diffusion and a GAN-based extrapolation method, plus clinician preference studies and a small real-longitudinal comparison on the BrixIA COVID dataset.

Significance. If the central claims were established, MVG would be a notable contribution to medical image generation: it addresses the realistic and important problem of zero-shot disease-progression simulation, combines LLM-based recaptioning with diffusion-based editing and video interpolation, and includes both automated metrics and clinician evaluations. The paper also makes a useful effort to evaluate against real longitudinal data, which is rare in this literature. However, the load-bearing premise that repeated masked DDIM inversion/denoising with a fixed terminal prompt produces a clinically meaningful progression trajectory is not proven in the preprint, and the quantitative evaluation is weakened by a partly circular classifier-confidence metric, missing error bars, and an underpowered longitudinal validation. The strengths are the problem framing and the breadth of the proposed pipeline; the weaknesses currently prevent the claims of clinical plausibility from being fully supported.

major comments (5)
  1. [Sec. 4.1, Eq. (3)-(6), Proposition 1] Proposition 1 is load-bearing for the central claim that PIE produces disease progression, but its proof is deferred to a supplementary appendix that is not included in this preprint. Moreover, Eq. (5) applies the same terminal text conditioning y at every recursive step and Eq. (6) blends with a fixed ROI mask; there is no stage-dependent conditioning or explicit severity schedule. Even if the claimed posterior maximization were true, it would establish convergence to some text-conditioned edit, not a clinically ordered trajectory. The authors should provide the proof in the main text or an accessible supplementary and add an experiment demonstrating monotonic movement along a clinically meaningful severity axis, for example by tracking a severity score or anatomical measurement over the generated states.
  2. [Sec. 5.2, Table 2 and Sec. 5.4, Tables 5-7] No error bars or significance tests are reported for the Table 2 results despite the statement in Sec. 5.2 that experiments were run with 5 random seeds. In addition, the main classification-confidence metric uses a DenseNet121 trained on the same label space (CheXpert, ISIC, Diabetic Retinopathy) that is used to construct the text prompts, making the metric partly circular; the hyperparameters N, gamma, beta1, and beta2 are then selected on this same metric in the ablations (Tables 5-7), which risks overfitting to the metric. The paper should report variance and statistical tests, and use independent evaluation such as distance to real follow-up images or clinician ratings of progression order rather than relying on the classifier confidence alone.
  3. [Sec. 5.5, BrixIA comparison] The only direct longitudinal validation reports a single MAE of approximately 0.0658 for 10 patients, with no baseline comparator, no per-patient error bars, and no statistical test; Figure 7 visualizes classifier confidence rather than distance to the real Day-7 image. This evidence is too weak to support the claim that MVG matches real disease trajectories, especially given the possibility that the generated sequence is a generic text-guided edit rather than true progression. A stronger comparison with at least one baseline method, patient-level metrics, and uncertainty quantification is needed.
  4. [Sec. 5.3, Table 3] Table 3 is confusingly formatted and its numbers do not match the prose. The prose reports average win rates of 79%, 70%, and 66% for Cardiomegaly, diabetic retinopathy, and benign skin lesion, respectively, but the table rows for MVG versus the three baselines show values such as 0.80, 0.67, and 0.63 for X-ray, which are not obviously reconciled with those averages. The column labels 'Method A' and 'Method B' and the absence of a clear comparison direction make the preference values hard to interpret, and no significance test is applied to the A/B preference results. The authors should restructure the table, reconcile it with the text, and provide confidence intervals or a paired significance test.
  5. [Sec. 5.4, Table 4] The ROI-mask ablation in Table 4 shows that removing the mask increases the confidence score for chest X-ray (0.729 vs 0.712) and for skin lesion (0.666 vs 0.453) while decreasing CLIP-I; the claim that MROI is a 'good controller to balance the alignment with the intended disease features and preservation of patient identity' is therefore not consistently supported by the reported numbers. The authors should explain these discrepancies or provide a more appropriate evaluation that accounts for the trade-off between the two metrics.
minor comments (6)
  1. [Abstract] The phrase 'Two user studies by veteran physicians, provide' contains a comma error and should be 'Two user studies by veteran physicians provide'.
  2. [Table 2] The baseline name 'Sable Video Diffusion' should be 'Stable Video Diffusion'.
  3. [Sec. 5.5] The text contains the typo 'seqeuence' for 'sequence', and 'longitude' should be 'longitudinal' throughout the paper.
  4. [Sec. 4.1] The sentence 'The proof of Proposition 1 and Proposition 2 are shown in the supplementary material' has subject-verb agreement issues; it should be 'is shown'.
  5. [References] Reference [45] is incorrectly attributed to Midjourney; the text refers to 'Video generation models as world simulators', which appears to describe Sora, and the URL does not match the title.
  6. [Figure 3] The labels 'PIE0' and 'PIEn' in Figure 3 are unclear; the figure would benefit from a consistent notation matching Eq. (3)-(6).

Circularity Check

1 steps flagged · score 5.0 of 10

Partial circularity: the disease-confidence metric verifies the same label that is embedded in the generation prompt, and the load-bearing progression guarantee is deferred to an absent supplementary.

  1. self definitional [Sec. 5.1 'Evaluation Metrics' with the conditioning input defined in Sec. 4.1 'Progressive Image Editing (PIE)']
    "The classification confidence score is derived from a supervised deep network trained for binary classification between negative (healthy) and positive (disease) samples. ... measures how well the generated images align with the target disease state. / The inputs to PIE are ... a corresponding terminal stage clinical report yN ... The Latent y will be the text conditioning of the diffusion model."

    The 'target disease state' used in the confidence metric is the same disease label that is embedded into the text conditioning y used to generate each image. The reported Conf therefore checks whether the generator obeyed its own prompt (label-consistency), not whether the sequence is a clinically valid progression. Since both the diffusion model and the classifier are trained on the same image-label datasets, high Conf is largely forced by construction once the text-to-image model respects its conditioning. This is partial, not total, circularity: the generation can still fail, and the physician-preference and BrixIA comparisons are not closed loops, but the headline Table 2 numbers cannot independently establish disease progression.

full rationale

The core algorithm — repeated masked DDIM inversion/denoising with a fixed text condition y — is not itself circular: it is a concrete generative procedure, and its output is not an algebraic relabeling of its input. The main partial circularity is in the headline quantitative evaluation. The classification-confidence metric (Sec. 5.1) uses a classifier trained on disease labels that are the same labels embedded in the GPT-4 re-captioned terminal prompt y (Sec. 4.1). A high Conf score therefore largely certifies that the generator followed its own text condition, not that the trajectory is clinically valid; this is a closed evaluation loop for the 'target disease state' component of the claim. The physician preference study (Sec. 5.3, 5.6) and the small BrixIA comparison (Sec. 5.5) provide some independent evidence, but the BrixIA result is a single MAE without a baseline comparator or error bars. Separately, the load-bearing Proposition 1 (posterior maximization) and Proposition 2 (convergence) are deferred to a supplementary not present in the preprint, so the 'guaranteed' movement along a disease manifold is currently an unverified assertion rather than a demonstrated step. Because the central 'clinically plausible trajectory' claim is only partially closed-loop (the confidence metric) and still has independent human-preference support, the circularity score is 5, not higher.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the assumption that text-guided diffusion edits follow a clinically meaningful progression direction, on the unproven Proposition 1, on the boundedness assumptions of Proposition 2, and on the transferability of SEINE to medical frames. The only hand-set quantities are the ablation-selected hyperparameters and the human-edited ROI mask.

free parameters (5)
  • N (number of PIE stages) = 10
    Chosen from ablation Table 6 to balance Conf, CLIP-I, and KID; not derived from first principles.
  • gamma (noise strength) = 0.6
    Chosen from ablation Table 5; at gamma=0.6 Conf=0.995 in the table but KID worsens; selected as the main setting.
  • beta1 (identity blending weight) = 0.01
    Chosen from ablation Table 7; used in Eq. 6 to control interpolation between generated and original content outside the ROI.
  • beta2 (ROI blending weight) = 0.75
    Chosen from ablation Table 7; controls the strength of generated disease features inside the ROI.
  • ROI mask M_ROI = per-image, human-edited after Med-SAM
    The mask localizes editing; in the experiments it is 'slightly edit[ed] by human' (Sec 4.1), so it is a manual per-input choice that affects results and is not fully reproducible.
assumptions (6)
  • domain assumption DDIM inversion and denoising are invertible and provide meaningful gradient information toward the text-conditioned distribution (Sec 4.1 Eq. 5).
    The PIE loop assumes that partially denoising an inverted latent toward prompt y moves the image along the disease manifold rather than introducing arbitrary changes.
  • domain assumption CLIP text embeddings of clinical reports align with visual disease features for all three modalities (Sec 4.1).
    The text conditioning is produced by recaptioning clinical reports with GPT-4 and encoding with CLIP; if this alignment fails, the edits are not clinically directed.
  • domain assumption The finetuned Stable Diffusion model covers the relevant disease manifolds for CXR, fundus, and skin images (Sec 5.1).
    The central experiment finetunes CompVis SD 1.4 on public datasets and assumes this is sufficient for realistic disease progression edits.
  • ad hoc to paper Proposition 1: the recursive composition of PIE maximizes p(x0_N | x0_0, y) (Sec 4.1).
    This is an unproven assertion in the main text; the proof is deferred to supplementary, which is not included.
  • ad hoc to paper Boundedness assumptions of Proposition 2, ||x0_0|| <= C1 and ||epsilon_theta|| <= C2, and the algebraic form of the bound (Eq. 7-8) (Sec 4.1).
    The proof of Proposition 2 is not included, and the displayed bound appears garbled, so the convergence statement is taken as an assumption rather than a demonstrated theorem.
  • domain assumption SEINE pretrained on general video data can interpolate medical frames when conditioned on ROI masks (Sec 4.2, Eq. 9-10).
    The transition generation assumes a general video diffusion model transfers to medical imaging without medical video finetuning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Medical Video Generation for Disease Progression Simulation." pith.science (2026). https://pith.science/paper/US4MEG5B

@misc{pith2026241111943,
  author       = {Pith},
  title        = {Pith review of: Medical Video Generation for Disease Progression Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/US4MEG5B}},
  note         = {Machine review of arXiv:2411.11943}
}
read the original abstract

Modeling disease progression is crucial for improving the quality and efficacy of clinical diagnosis and prognosis, but it is often hindered by a lack of longitudinal medical image monitoring for individual patients. To address this challenge, we propose the first Medical Video Generation (MVG) framework that enables controlled manipulation of disease-related image and video features, allowing precise, realistic, and personalized simulations of disease progression. Our approach begins by leveraging large language models (LLMs) to recaption prompt for disease trajectory. Next, a controllable multi-round diffusion model simulates the disease progression state for each patient, creating realistic intermediate disease state sequence. Finally, a diffusion-based video transition generation model interpolates disease progression between these states. We validate our framework across three medical imaging domains: chest X-ray, fundus photography, and skin image. Our results demonstrate that MVG significantly outperforms baseline models in generating coherent and clinically plausible disease trajectories. Two user studies by veteran physicians, provide further validation and insights into the clinical utility of the generated sequences. MVG has the potential to assist healthcare providers in modeling disease trajectories, interpolating missing medical image data, and enhancing medical education through realistic, dynamic visualizations of disease progression.

Figures

Figures reproduced from arXiv: 2411.11943 by the authors.

Figure 1
Figure 1. Illustrative examples of video-based disease progres [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Visualization for cardiomegarly disease state absolute [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the MVG inference pipeline. The above blue part denotes the single step of PIE. For any given step [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Editing path of PIE, SVD, and Extrapolation in the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Disease Progression Simulation of MVG. The top progres [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Using MVG, SVD, Extrapolation to simulate Edema [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Evaluating the confidence scores of PIE (stage 1 of MVG) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLM4SG: Adapting Large Language Model for Scatterer Generation via Synesthesia of Machines

    eess.SP 2025-05 conditional novelty 5.0 of 10

    Fine-tuning a small GPT-2 with LoRA on a new synthetic V2V dataset lets it predict ray-tracing scatterer grids from LiDAR point clouds, outperforming a ResNet baseline.

  2. Foundation Model Empowered Synesthesia of Machines (SoM): AI-native Intelligent Multi-Modal Sensing-Communication Integration

    eess.SP 2025-06 conditional novelty 4.0 of 10

    The paper proposes a systematic classification and two roadmaps for using foundation models (LLMs and wireless foundation models) to design Synesthesia of Machines systems for 6G, with preliminary case-study evidence ...

  3. From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine

    cs.AI 2025-02 conditional novelty 3.0 of 10

    A PRISMA-ScR scoping review of 144 studies finds the field shifting from text-only LLMs to multimodal AI in medicine, with evaluation and data diversity still the main bottlenecks.

Reference graph

Works this paper leans on

79 extracted references · 49 canonical work pages · cited by 3 Pith papers

  1. [1]

    Diabetic Retinopathy Detection, howpublished= https: //www.kaggle.com/c/diabetic-retinopathy- detection, 2015. 5

    work page 2015

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 2, 3

    arXiv 2023

  3. [3]

    Attentive state- space modeling of disease progression

    Ahmed M Alaa and Mihaela van der Schaar. Attentive state- space modeling of disease progression. Advances in neural information processing systems, 32, 2019. 2

    work page 2019

  4. [4]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 2

    2023

  5. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 6, 7

    arXiv 2023

  6. [6]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 6

    work page 2023

  7. [7]

    A vision–language foundation model for the generation of realistic chest x-ray images

    Christian Bluethgen, Pierre Chambon, Jean-Benoit Del- brouck, Rogier van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tanishq Mathew Abraham, Shivanshu Purohit, Curtis P Langlotz, and Akshay S Chaud- hari. A vision–language foundation model for the generation of realistic chest x-ray images. Nature Biomedical Engineer- ing, pages 1–13, 2024. 2

    work page 2024

  8. [8]

    Base- lines for chest x-ray report generation

    William Boag, Tzu-Ming Harry Hsu, Matthew McDermott, Gabriela Berner, Emily Alesentzer, and Peter Szolovits. Base- lines for chest x-ray report generation. In Machine learning for health workshop, pages 126–140. PMLR, 2020. 7

    work page 2020

Show all 79 references
  1. [9]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. arXiv preprint arXiv:2211.09800, 2022. 2

    2022 arXiv

  2. [10]

    Chexpert plus: Hundreds of thousands of aligned radiology texts, im- ages and patients

    Pierre Chambon, Jean-Benoit Delbrouck, Thomas Sounack, Shih-Cheng Huang, Zhihong Chen, Maya Varma, Steven QH Truong, Chu The Chuong, and Curtis P Langlotz. Chexpert plus: Hundreds of thousands of aligned radiology texts, im- ages and patients. arXiv preprint arXiv:2405.19538, 2024. 5

    2024 arXiv

  3. [11]

    Seine: Short-to-long video diffusion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. In The Twelfth International Conference on Learning Representations,...

    2023

  4. [12]

    Skin lesion analysis toward melanoma detection 2018: A chal- lenge hosted by the international skin imaging collaboration (isic)

    Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al. Skin lesion analysis toward melanoma detection 2018: A chal- lenge hosted by the international skin imagin...

    2018 arXiv

  5. [13]

    Disease progression model- ing: key concepts and recent developments

    Sarah F Cook and Robert R Bies. Disease progression model- ing: key concepts and recent developments. Current pharma- cology reports, 2:221–230, 2016. 1

    2016

  6. [14]

    End-to-end adversarial retinal image syn- thesis

    Pedro Costa, Adrian Galdran, Maria Ines Meyer, Meindert Niemeijer, Michael Abràmoff, Ana Maria Mendonça, and Aurélio Campilho. End-to-end adversarial retinal image syn- thesis. IEEE transactions on medical imaging, 37(3):781–791,

  7. [15]

    Investigating data memorization in 3d latent diffusion models for medical image synthesis

    Salman Ul Hassan Dar, Arman Ghanaat, Jannik Kahmann, Isabelle Ayx, Theano Papavassiliu, Stefan O Schoenberg, and Sandy Engelhardt. Investigating data memorization in 3d latent diffusion models for medical image synthesis. In International Conference on Medical Image Computing ...

    2023

  8. [16]

    Scaling rectified flow trans- formers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim En- tezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  9. [17]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming- Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Comp...

    2023

  10. [18]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communi- cations of the ACM, 63(11):139–144, 2020. 2

    2020

  11. [19]

    Biomedjourney: Counterfactual biomedical image genera- tion by instruction-learning from multimodal patient journeys

    Yu Gu, Jianwei Yang, Naoto Usuyama, Chunyuan Li, Sheng Zhang, Matthew P Lungren, Jianfeng Gao, and Hoifung Poon. Biomedjourney: Counterfactual biomedical image genera- tion by instruction-learning from multimodal patient journeys. arXiv preprint arXiv:2310.10765, 2023. 2

    arXiv

  12. [20]

    Image prediction of disease progres- sion for osteoarthritis by style-based manifold extrapolation

    Tianyu Han, Jakob Nikolas Kather, Federico Pedersoli, Markus Zimmermann, Sebastian Keil, Maximilian Schulze- Hagen, Marc Terwoelbeck, Peter Isfort, Christoph Haarburger, Fabian Kiessling, et al. Image prediction of disease progres- sion for osteoarthritis by style-based manifo...

    2022

  13. [21]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 2

    2020

  14. [22]

    Densely connected convolutional networks

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017. 6

    2017

  15. [23]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Sil- viana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAA...

    2019

  16. [24]

    Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de- identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317, 2019. 5

    2019

  17. [25]

    Conditional gan with 3d discriminator for mri generation of alzheimer’s disease progression

    Euijin Jung, Miguel Luna, and Sang Hyun Park. Conditional gan with 3d discriminator for mri generation of alzheimer’s disease progression. Pattern Recognition, 133:109061, 2023. 2

    2023

  18. [26]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. arXiv preprint arXiv:2206.00364, 2022. 2

    2022 arXiv

  19. [27]

    Diffusion models in medical imaging: A comprehensive survey

    Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Moein Heidari, Reza Azad, Mohsen Fayyaz, Ilker Hacihaliloglu, and Dorit Merhof. Diffusion models in medical imaging: A comprehensive survey. Medical Image Analysis, 88:102846,

  20. [28]

    Denoising diffusion probabilistic models for 3d medical image generation

    Firas Khader, Gustav Müller-Franzes, Soroosh Tayebi Arasteh, Tianyu Han, Christoph Haarburger, Maximilian Schulze-Hagen, Philipp Schad, Sandy Engel- hardt, Bettina Baeßler, Sebastian Foersch, et al. Denoising diffusion probabilistic models for 3d medical image generation. Scie...

    2023

  21. [29]

    Auto-encoding varia- tional bayes

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  22. [30]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. arXiv preprint arXiv:2304.02643, 2023. 7

    2023 arXiv

  23. [31]

    Anyv2v: A plug-and-play framework for any video-to- video editing tasks

    Max Ku, Cong Wei, Weiming Ren, Huan Yang, and Wenhu Chen. Anyv2v: A plug-and-play framework for any video-to- video editing tasks. arXiv preprint arXiv:2403.14468, 2024. 3

    2024 arXiv

  24. [32]

    The slice-3d dataset: 400,000 skin lesion image crops extracted from 3d tbp for skin cancer detection

    Nicholas R Kurtansky, Brian M D’Alessandro, Maura C Gillis, Brigid Betz-Stablein, Sara E Cerminara, Rafael Gar- cia, Marcela Alves Girundi, Elisabeth Victoria Goessinger, Philippe Gottfrois, Pascale Guitera, et al. The slice-3d dataset: 400,000 skin lesion image crops extracte...

    2024

  25. [33]

    Towards predicting temporal changes in a patient’s chest x-ray images based on electronic health records

    Daeun Kyung, Junu Kim, Tackeun Kim, and Edward Choi. Towards predicting temporal changes in a patient’s chest x-ray images based on electronic health records. arXiv preprint arXiv:2409.07012, 2024. 2

    2024 arXiv

  26. [34]

    Predicting alzheimer’s disease pro- gression using multi-modal deep learning approach

    Garam Lee, Kwangsik Nho, Byungkon Kang, Kyung-Ah Sohn, and Dokyoon Kim. Predicting alzheimer’s disease pro- gression using multi-modal deep learning approach. Scientific reports, 9(1):1952, 2019. 1

    1952

  27. [35]

    Llm-cxr: Instruction-finetuned llm for cxr image understand- ing and generation

    Suhyeon Lee, Won Jun Kim, Jinho Chang, and Jong Chul Ye. Llm-cxr: Instruction-finetuned llm for cxr image understand- ing and generation. arXiv preprint arXiv:2305.11490, 2023. 2

    2023 arXiv

  28. [36]

    Endora: Video generation models as endoscopy simu- lators

    Chenxin Li, Hengyu Liu, Yifan Liu, Brandon Y Feng, Wuyang Li, Xinyu Liu, Zhen Chen, Jing Shao, and Yixuan Yuan. Endora: Video generation models as endoscopy simu- lators. arXiv preprint arXiv:2403.11050, 2024. 3

    2024 arXiv

  29. [37]

    Artificial intelligence for biomedical video generation, 2024

    Linyuan Li, Jianing Qiu, Anujit Saha, Lin Li, Poyuan Li, Mengxian He, Ziyu Guo, and Wu Yuan. Artificial intelligence for biomedical video generation, 2024. 1

    2024

  30. [38]

    Efficient learning of continuous-time hidden markov models for disease progression

    Yu-Ying Liu, Shuang Li, Fuxin Li, Le Song, and James M Rehg. Efficient learning of continuous-time hidden markov models for disease progression. Advances in neural informa- tion processing systems, 28, 2015. 1, 2

    2015

  31. [39]

    Learning to generate clinically coherent chest x-ray reports

    Justin Lovelace and Bobak Mortazavi. Learning to generate clinically coherent chest x-ray reports. In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 1235–1243, 2020. 7

    2020

  32. [40]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471, 2022. 7

    2022

  33. [41]

    Segment anything in medical images

    Jun Ma and Bo Wang. Segment anything in medical images. arXiv preprint arXiv:2304.12306, 2023. 7

    2023 arXiv

  34. [42]

    Segment anything in medical images

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1):654, 2024. 4

    2024

  35. [43]

    Chest x-ray generation and data aug- mentation for cardiovascular abnormality classification

    Ali Madani, Mehdi Moradi, Alexandros Karargyris, and Tan- veer Syeda-Mahmood. Chest x-ray generation and data aug- mentation for cardiovascular abnormality classification. In Medical imaging 2018: Image processing , pages 415–420. SPIE, 2018. 2

    2018

  36. [44]

    Midjourney – home

    Midjourney. Midjourney – home. https : / / www . midjourney.com/home, 2024. Accessed: 2024-07-30. 2

    2024

  37. [45]

    Video generation models as world simulators

    Midjourney. Video generation models as world simulators. https://www.midjourney.com/home , 2024. Ac- cessed: 2024-07-30. 2

    2024

  38. [46]

    Sybil: a validated deep learning model to predict future lung can- cer risk from a single low-dose chest computed tomography

    Peter G Mikhael, Jeremy Wohlwend, Adam Yala, Ludvig Karstens, Justin Xiang, Angelo K Takigami, Patrick P Bour- gouin, PuiYee Chan, Sofiane Mrah, Wael Amayri, et al. Sybil: a validated deep learning model to predict future lung can- cer risk from a single low-dose chest compute...

    2023

  39. [47]

    A multimodal comparison of latent denois- ing diffusion probabilistic models and generative adversarial networks for medical image synthesis

    Gustav Müller-Franzes, Jan Moritz Niehues, Firas Khader, Soroosh Tayebi Arasteh, Christoph Haarburger, Christiane Kuhl, Tianci Wang, Tianyu Han, Teresa Nolte, Sven Nebelung, et al. A multimodal comparison of latent denois- ing diffusion probabilistic models and generative adve...

    2023

  40. [48]

    Medical im- age synthesis with context-aware generative adversarial net- works

    Dong Nie, Roger Trullo, Jun Lian, Caroline Petitjean, Su Ruan, Qian Wang, and Dinggang Shen. Medical im- age synthesis with context-aware generative adversarial net- works. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Conferenc...

    2017

  41. [49]

    Editing implicit assumptions in text-to-image diffusion models

    Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. Editing implicit assumptions in text-to-image diffusion models. arXiv preprint arXiv:2303.08084, 2023. 2

    2023 arXiv

  42. [50]

    Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality clas- sification systems

    Kai Packhäuser, Lukas Folle, Florian Thamm, and Andreas Maier. Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality clas- sification systems. In 2023 IEEE 20th International Sympo- sium on Biomedical Imaging (ISBI), pages 1–...

    2023

  43. [51]

    Zero-shot image-to-image translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot image-to-image translation. arXiv preprint arXiv:2302.03027, 2023. 2

    2023 arXiv

  44. [52]

    Pika art – home

    Pika. Pika art – home. https://pika.art/home, 2024. Accessed: 2024-07-30. 2, 6, 7

    2024

  45. [53]

    Pixverse

    PixVerse. Pixverse. https://app.pixverse.ai/ ,

  46. [54]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  47. [55]

    Degenerative adversarial neuroimage nets: generating images that mimic disease progression

    Daniele Ravi, Daniel C Alexander, Neil P Oxtoby, and Alzheimer’s Disease Neuroimaging Initiative. Degenerative adversarial neuroimage nets: generating images that mimic disease progression. In International Conference on Medical Image Computing and Computer-Assisted Interventi...

    2019

  48. [56]

    Degenerative adversar- ial neuroimage nets for brain scan simulations: Application in ageing and dementia

    Daniele Ravi, Stefano B Blumberg, Silvia Ingala, Frederik Barkhof, Daniel C Alexander, Neil P Oxtoby, Alzheimer’s Disease Neuroimaging Initiative, et al. Degenerative adversar- ial neuroimage nets for brain scan simulations: Application in ageing and dementia. Medical Image An...

  49. [57]

    Stable diffusion videos

    Nathan Raw. Stable diffusion videos. https://github. com/nateraw/stable-diffusion-videos , 2023. 6

    2023

  50. [58]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 2, 3, 4

    2022

  51. [59]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven gen- eration. arXiv preprint arXiv:2208.12242, 2022. 5

    2022 arXiv

  52. [60]

    Disease trajectory maps

    Peter Schulam and Raman Arora. Disease trajectory maps. Advances in neural information processing systems, 29, 2016. 2

    2016

  53. [61]

    Personalized input-output hidden markov models for disease progression modeling

    Kristen A Severson, Lana M Chahine, Luba Smolensky, Ken- ney Ng, Jianying Hu, and Soumya Ghosh. Personalized input-output hidden markov models for disease progression modeling. In Machine Learning for Healthcare Conference, pages 309–330. PMLR, 2020. 1

    2020

  54. [62]

    Bs-net: Learning covid-19 pneumonia severity on a large chest x-ray dataset

    Alberto Signoroni, Mattia Savardi, Sergio Benini, Nicola Adami, Riccardo Leonardi, Paolo Gibellini, Filippo Vaccher, Marco Ravanelli, Andrea Borghesi, Roberto Maroldi, et al. Bs-net: Learning covid-19 pneumonia severity on a large chest x-ray dataset. Medical Image Analysis , ...

  55. [63]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792,

    arXiv

  56. [64]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502,

    2010 arXiv

  57. [65]

    Conformal time-series forecasting

    Kamile Stankeviciute, Ahmed M Alaa, and Mihaela van der Schaar. Conformal time-series forecasting. Advances in Neural Information Processing Systems, 34:6216–6228, 2021. 2

    2021

  58. [66]

    From sora what we can see: A survey of text-to-video generation

    Rui Sun, Yumin Zhang, Tejal Shah, Jiahao Sun, Shuoying Zhang, Wenqi Li, Haoran Duan, Bo Wei, and Rajiv Ran- jan. From sora what we can see: A survey of text-to-video generation. arXiv preprint arXiv:2405.10674, 2024. 2

    2024 arXiv

  59. [67]

    Bora: Biomedical generalist video generation model

    Weixiang Sun, Xiaocao You, Ruizhe Zheng, Zhengqing Yuan, Xiang Li, Lifang He, Quanzheng Li, and Lichao Sun. Bora: Biomedical generalist video generation model. arXiv preprint arXiv:2407.08944, 2024. 3

    2024 arXiv

  60. [68]

    A roadmap for the de- velopment of human body digital twins

    Chenyu Tang, Wentian Yi, Edoardo Occhipinti, Yanning Dai, Shuo Gao, and Luigi G Occhipinti. A roadmap for the de- velopment of human body digital twins. Nature Reviews Electrical Engineering, 1(3):199–207, 2024. 1

    2024

  61. [69]

    The ham10000 dataset, a large collection of multi-source dermato- scopic images of common pigmented skin lesions

    Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multi-source dermato- scopic images of common pigmented skin lesions. Scientific data, 5(1):1–9, 2018. 5

    2018

  62. [70]

    Envisioning the future of personalized medicine: Role and realities of digital twins

    Alexandre Vallée. Envisioning the future of personalized medicine: Role and realities of digital twins. Journal of Medical Internet Research, 26:e50204, 2024. 1

    2024

  63. [71]

    Unsupervised learning of disease progression models

    Xiang Wang, David Sontag, and Fei Wang. Unsupervised learning of disease progression models. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 85–94, 2014. 2

    2014

  64. [72]

    Lavie: High-quality video gener- ation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video gener- ation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 2

    2023 arXiv

  65. [73]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Weixian Lei, Yuchao Gu, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. arXiv preprint arXiv:2212.11565, 2022. 6

    2022 arXiv

  66. [74]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Sys- tems, 36, 2024. 6

    2024

  67. [75]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 6, 7

    2024 arXiv

  68. [76]

    Generative adversar- ial network in medical imaging: A review

    Xin Yi, Ekta Walia, and Paul Babyn. Generative adversar- ial network in medical imaging: A review. Medical image analysis, 58:101552, 2019. 2

    2019

  69. [77]

    Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification

    Zhuoning Yuan, Yan Yan, Milan Sonka, and Tianbao Yang. Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3040–3049, 2021. 6

    2021

  70. [78]

    Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network

    Zizhao Zhang, Lin Yang, and Yefeng Zheng. Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network. InProceed- ings of the IEEE conference on computer vision and pattern Recognition, pages 9242–9251, 2018. 2

    2018

  71. [2024]

    Accessed: 2024-11-10. 6, 7

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.