EndoGen: Conditional Autoregressive Endoscopic Video Generation

Cheng Wang; Hengyu Liu; Tianming Liu; Xinyu Liu; Yixuan Yuan

REVIEW 4 major objections 5 minor 31 references

EndoGen: Conditional Autoregressive Endoscopic Video Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An autoregressive transformer that packs video frames into a grid and masks low-information tokens can generate condition-specific endoscopic videos, beating prior baselines on all eight HyperKvasir findings and improving semi-supervised…

desk verdict First credible conditional endoscopic video generator, but the baseline-conditioning gap undermines the SOTA claim until fixed. read the letter →

arxiv 2507.17388 v1 pith:4SJ3V5KD submitted 2025-07-23 cs.CV eess.IV

Xinyu Liu , Hengyu Liu , Cheng Wang , Tianming Liu , Yixuan Yuan This is my paper

classification cs.CVeess.IV

keywords conditionalvideogenerationendoscopyautoregressivemodelsspatiotemporalgrid-framepatterningsemantic-awaretokenmaskingsemi-supervisedlearningpolypsegmentationFréchetDistance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces EndoGen as the first conditional endoscopic video generation framework: instead of producing arbitrary or static endoscopic content, it takes a condition label for a pathological finding or surgical task and generates a temporally coherent video clip for that condition. The central claim is that an autoregressive transformer can do this well if the video frames are rearranged into a single grid image, so the model attends over space and time in one pass, and if training stochastically masks low-information tokens so the model concentrates on diagnostically meaningful content. On HyperKvasir, the authors report the best Fréchet Video Distance on all eight pathological findings, with an average of 507.2 versus 1548.6 for the best prior baseline, and on SurgVisdom they report the best results on all three surgical tasks. They further show that synthetic polyp frames generated by EndoGen can replace real unlabeled data in semi-supervised polyp segmentation and improve it further when combined with real data. If these results hold, the framework offers a practical route to on-demand, pathology-specific endoscopic video for training simulators, diagnostic-system development, and data augmentation in low-annotation regimes.

What carries the argument

The load-bearing machinery has two parts. Spatiotemporal Grid-Frame Patterning (SGP) converts a video clip into one composite image by arranging its frames row by row, then treats that grid as an image for the VQGAN encoder and the autoregressive transformer; the transformer's attention can therefore model spatial detail and temporal continuity simultaneously, and the decoder re-splits the generated grid into a video. Semantic-Aware Token Masking (SAT) partitions the tokenized features into segments, computes each segment's variance across the channel dimension, and applies a Bernoulli mask with ratio $p_i = \mathrm{Clamp}\big((1 - \sigma_i^2/\max_i \sigma_i^2)\,p_{\max},\,0,\,p_{\max}\big)$, where $\sigma_i^2$ is the segment variance, so low-variance, information-poor tokens are masked during training. A learnable condition embedding is prepended as the starting token, and training uses a standard cross-entropy next-token loss. The ablation study shows SGP carries most of the gain: replacing the grid layout with a plain 2D reshaping of the video raises FVD from 507.2 to 2617.5, while removing SAT raises it to 562.0.

What would settle it

Encode real HyperKvasir frames through the same frozen VQGAN grid-patterning path EndoGen uses, decode immediately, and compare the reconstruction with the original frames using FID or LPIPS; if that reconstruction gap is as large as the quality gap between EndoGen and VideoGPT, then the tokenizer, not the autoregressive model, sets the quality ceiling.

Watch

Extended reading notes

Core claim

The discovery the paper argues for is that the long-range conditional modeling of autoregressive transformers transfers from images to endoscopic video without specialized temporal modules, provided the video is re-represented as a grid. EndoGen stitches the frames of a clip into a grid image, encodes it with a frozen VQGAN, prepends a learnable condition token that encodes the desired class, and generates the remaining grid tokens autoregressively before decoding and re-splitting them into frames. Semantic-Aware Token Masking modifies the training objective so that segments with low channel variance are masked with an adaptive probability, steering capacity toward tokens that carry clinically relevant structure such as lesions or instruments. The paper reports that this system outperforms diffusion-based and autoregressive baselines on every condition tested: all eight pathological findings of HyperKvasir, with an average FVD of 507.2 compared with 1548.6 for VideoGPT, and all three surgical tasks of SurgVisdom; it also reports lower FID and LPIPS. In the downstream study, generated polyp frames serve as unlabeled data for semi-supervised segmentation, matching or improving on real unlabeled data, with 70.96% versus 70.80% Dice for FixMatch when synthetic data replaces real data.

Load-bearing premise

If the frozen image tokenizer trained on ordinary photographs distorts endoscopy textures once frames are packed into a grid, then every generated video is capped at that distortion, and the paper does not measure the tokenizer's reconstruction fidelity on real endoscopic frames.

Editorial extensions

If this is right

Supplying only a condition token lets EndoGen generate video for a specified finding or surgical task, so clinicians or training systems can request a particular pathology instead of searching through unconditionally generated footage.
Synthetic frames can substitute for real unlabeled data in semi-supervised polyp segmentation: with FixMatch, synthetic-only unlabeled data reaches 70.96% Dice versus 70.80% with real unlabeled data, and combining both reaches 71.03%.
Because the pipeline reuses a frozen VQGAN and an ImageNet-pretrained autoregressive image generator, the framework inherits the scalability of transformer image generation and is reported to extend to 64-frame clips and 256×256 resolution in the supplementary material.
The reported margins over diffusion baselines on FVD, CD-FVD, FID, and LPIPS suggest that, for endoscopic video, an autoregressive generator with a grid-frame representation is a competitive alternative to video diffusion models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The grid ordering in SGP is fixed as row-by-row, which determines which frames are spatial neighbors for the transformer; varying or learning that ordering is a plausible way to improve temporal coherence further, but the paper does not explore it.
Because the VQGAN is frozen and pretrained on general images, its codebook is the one component not adapted to endoscopy; if it is the bottleneck, an endoscopy-tuned tokenizer should lower FVD further, which would also clarify how much of the reported gain is due to SGP and SAT rather than tokenizer capacity.
SAT uses channel variance as a proxy for semantic richness; masking guided by segmentation maps or pathology annotations could push generation toward clinically critical regions even more directly, at the cost of needing labels during training.
The downstream evaluation covers only polyp segmentation, so the same synthetic-data substitution logic remains untested for other findings, for detection, and for classification; those tests would show whether the benefit generalizes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real step forward—nobody else has conditioned endoscopic video generation on pathology labels—and the SGP grid-frame trick is a clean way to reuse an autoregressive image model for video. The downstream segmentation experiment is also the right kind of evidence. But the experimental section has a hole the authors need to fix before the headline claim can stand: the per-class FVD comparison doesn't tell us how any of the baselines were conditioned, and VideoGPT is unconditional in its original form. If VideoGPT and the others were not given class labels, the comparison is between conditional generation and unconditional generation, and the reported margin mostly reflects that difference. That's not a fatal flaw in the method, but it is a fatal flaw in the evidence as presented.

What's genuinely new: SGP maps a video to a grid image and runs a LlamaGen-style autoregressive model over the tokens, which is simple and effective; the ablation shows it is doing heavy lifting. SAT—variance-based masking—is a minor embellishment but seems to help modestly. The paper is honest about the frozen VQGAN and reports code release. The citation pattern looks fine; self-citations are to relevant prior work.

Soft spots, in order of severity. First, no error bars or seeds anywhere; FVD differences of 500–1000 points could be noise with small per-class test sets. Second, the downstream evaluation uses synthetic unlabeled frames from a model trained on the same HyperKvasir data the segmentation test split is drawn from, so distribution overlap could inflate the gains. Fix: split the generator's training subjects from the test subjects. Third, the frozen ImageNet VQGAN is a plausible quality cap; a quick codebook analysis on endoscopic frames would settle whether it captures endoscopy-specific textures.

Bottom line: the method is promising, the writing is clear, and the limitations are addressable. The paper deserves a serious referee, but the referee should demand controlled baseline conditioning, repeated-run statistics, and a leakage-free downstream setup.

Referee Report

4 major / 5 minor

Summary. EndoGen proposes a conditional autoregressive video generation framework for endoscopy. Videos are converted into grid-frame images via the proposed Spatiotemporal Grid-Frame Patterning (SGP), tokenized by a frozen VQGAN, and modeled autoregressively from a learnable condition token; a variance-based Semantic-Aware Token Masking (SAT) mechanism is used during training. The paper reports FVD, CD-FVD, FID, and LPIPS comparisons against SimDA, VDM, and VideoGPT on HyperKvasir and SurgVisdom, ablations of SGP/SAT and the masking threshold pmax, and a downstream semi-supervised polyp segmentation experiment in which EndoGen-generated synthetic frames are used as unlabeled data.

Significance. If the reported results are reproducible, EndoGen would be a useful step toward controllable endoscopic video synthesis and a practical source of synthetic training data for downstream tasks. The method is clearly described, the code release is announced, and the ablation study in Table 4 isolates the main components. However, the evidence for the state-of-the-art claim is currently incomplete: the conditioning of the baselines is not specified, no uncertainty estimates are given, and the frozen VQGAN's fidelity on endoscopic imagery is unexamined. These gaps are fixable with additional experiments and reporting, so the manuscript would benefit from a major revision rather than being rejected outright.

major comments (4)

[Sec. 3.2, Tables 1-3] The manuscript does not state how the three baselines were conditioned on the eight pathological findings or the three surgical tasks. VideoGPT is originally an unconditional video model, while SimDA and VDM are diffusion models whose conditioning mechanisms (e.g., class embeddings, cross-attention, classifier-free guidance) are not described. If VideoGPT was run in its standard unconditional form, its per-class FVD values in Table 1 are not conditional-generation metrics; they are FVDs of unlabeled samples against class-specific reference sets, and the comparison conflates the ability to use the condition with video quality. Please specify the exact conditioning setup for every baseline, and ideally include an unconditional variant of EndoGen as a control; without this, the state-of-the-art claim is not established.
[Sec. 3.2, Tables 1-4] No error bars, standard deviations, or repeated-run statistics are reported for any metric. The FVD gap between EndoGen (507.2) and VideoGPT (1548.6) in Table 1 is large, but the ablation gap between pmax=0.2 (514.8) and pmax=0.3 (507.2) in Table 4 is small; without variance estimates the reader cannot judge whether either difference is reliable. Please report at least three random seeds with mean and standard deviation, or equivalent interval estimates, for the main comparisons and ablations.
[Sec. 3.1, implementation details and Sec. 2.1] All generation quality is capped by a frozen VQGAN pretrained on ImageNet, yet the paper provides no evaluation of how faithfully this encoder-decoder reconstructs endoscopic grid-frame images. This concern is amplified because SGP may produce large grid images (e.g., 16 frames at 128x128 arranged in a grid) that differ in resolution and content from the VQGAN's training distribution. Please report reconstruction FID/LPIPS on endoscopic grids, codebook usage statistics, and either finetune the tokenizer on endoscopic data or justify the frozen choice with concrete evidence.
[Sec. 3.3, downstream polyp segmentation] The HyperKvasir split for polyp segmentation is described as 1,000 images with an 8:2 train-test ratio, but no information is given about whether the videos used to train EndoGen overlap with the segmentation test images. If the generator saw the test-distribution frames, the synthetic unlabeled data could leak information and inflate the segmentation gains. In addition, only EndoGen synthetic frames are tested; to support the claim that the generated data are useful, include synthetic data from at least one baseline generator (e.g., VDM or VideoGPT) under the same semi-supervised protocol.

minor comments (5)

[Sec. 2.2, Eq. (1)] The notation max(sigma^2_i) is ambiguous: specify whether the maximum is taken over segments in the batch, over the dataset, or over channels, and define sigma^2_i consistently.
[Table 1] The abbreviations Bar., Ecta., and Perf. are not expanded in the caption or the text; please define them in the caption or use the full condition names.
[Sec. 3.1] The sentence 'SurgVisdom [31] contains surgical videos on porcine model' contains a grammatical error; also consider giving the official challenge name and a brief description of the video content.
[Fig. 1] Figure 1 is referenced but not analyzed in the text; please add a sentence describing what the reader should observe in the generated frames at different resolutions and frame counts.
[Table 4] The 'w/o SGP' ablation is described only as 'a simple 2D reshaping of the video sequence'; please specify the exact reshaping operation (e.g., concatenating frames along width or height) so the ablation is reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EndoGen's derivation is self-contained and its claims are measured against external baselines, not its own inputs.

full rationale

EndoGen's generation pipeline contains no load-bearing circular step. The SGP grid-frame reformulation is a modeling choice, not a definitional equivalence: video frames are arranged into a grid and tokenized by a frozen ImageNet-pretrained VQGAN, and the autoregressive model is initialized with ImageNet class-conditional weights, both external. The condition tokens are learned embeddings, and the masked-token cross-entropy loss supervises the generated tokens against real video tokens, so the training signal is not generated by the model itself. The only tuned hyperparameter, pmax, is selected by validation on FVD (Table 4), which is a standard fitting procedure rather than a renamed prediction. The downstream polyp-segmentation experiment uses EndoGen-generated frames as unlabeled data, but the reported improvement is measured on a real 8:2 split of HyperKvasir, so the semi-supervised gain is an external benchmark, not an identity. Self-citations in the introduction (Endora, DiffRect, and prior CUHK medical-imaging work) motivate the gap and prior methods, but none supplies a load-bearing premise nor a uniqueness theorem that forces the design. The strongest unresolved objection, that baselines VideoGPT, VDM, and SimDA may not have been conditioned in the same way, concerns experimental comparability, not circularity, and per the rules is not counted here.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard machine-learning design choices (VQGAN and LlamaGen-style AR initialization), a fitted masking threshold pmax, and a heuristic assumption that variance equates to semantic importance. No new physical or anatomical entities are introduced.

free parameters (2)

pmax = 0.3
Maximum masking ratio threshold in Eq. (1), tuned on HyperKvasir via ablation (Table 4); the final model uses pmax=0.3.
H = 8
Segment length for variance computation in SAT, set to 8 without reported sensitivity analysis.

assumptions (4)

domain assumption Frozen VQGAN pretrained on general domain data (ImageNet) is an adequate tokenizer for endoscopic frames
Used in Section 3.1 without endoscopy-specific fine-tuning; if the codebook fails on endoscopy, generation quality is capped.
domain assumption ImageNet-pretrained class-conditional autoregressive model provides a useful initialization for medical video generation
Used in Section 3.1; transfer from natural images to endoscopy is assumed to hold.
ad hoc to paper Arranging frames in a row-by-row grid (SGP) preserves temporal consistency
The paper claims SGP preserves temporal dynamics, but no theoretical or empirical analysis of positional continuity relative to other layouts is provided beyond an ablation against a simple 2D reshaping.
ad hoc to paper Variance-based masking selects tokens with semantic and clinical relevance
Eq. (1) equates high feature variance with informativeness; the link to clinical relevance (lesions, tools) is asserted rather than demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EndoGen: Conditional Autoregressive Endoscopic Video Generation." pith.science (2026). https://pith.science/paper/4SJ3V5KD

@misc{pith2026250717388,
  author       = {Pith},
  title        = {Pith review of: EndoGen: Conditional Autoregressive Endoscopic Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4SJ3V5KD}},
  note         = {Machine review of arXiv:2507.17388}
}

read the original abstract

Endoscopic video generation is crucial for advancing medical imaging and enhancing diagnostic capabilities. However, prior efforts in this field have either focused on static images, lacking the dynamic context required for practical applications, or have relied on unconditional generation that fails to provide meaningful references for clinicians. Therefore, in this paper, we propose the first conditional endoscopic video generation framework, namely EndoGen. Specifically, we build an autoregressive model with a tailored Spatiotemporal Grid-Frame Patterning (SGP) strategy. It reformulates the learning of generating multiple frames as a grid-based image generation pattern, which effectively capitalizes the inherent global dependency modeling capabilities of autoregressive architectures. Furthermore, we propose a Semantic-Aware Token Masking (SAT) mechanism, which enhances the model's ability to produce rich and diverse content by selectively focusing on semantically meaningful regions during the generation process. Through extensive experiments, we demonstrate the effectiveness of our framework in generating high-quality, conditionally guided endoscopic content, and improves the performance of downstream task of polyp segmentation. Code released at https://www.github.com/CUHK-AIM-Group/EndoGen.

Figures

Figures reproduced from arXiv: 2507.17388 by the authors.

**Figure 1.** Endoscopic frames and videos with different resolutions generated by EndoGen. or pathological conditions when needed by doctors [29]. These limitations hinder their practical utility in scenarios requiring targeted outputs, such as generating videos of specific pathologies or tailoring simulations for surgical training. Thus, there is an urgent need for a conditional endoscopy video generation framework that can pro… view at source ↗

**Figure 2.** Illustration of the EndoGen framework. During training, each input video undergoes (a) Spatiotemporal Grid-Frame Patterning (SGP), (b) Semantic-Aware Token Masking (SAT), and (c) Autoregressive Generation. During inference, video tokens are generated autoregressively based on the provided condition token and then reconstructed into video format. ation of temporally consistent and detail preserved endoscopic sequen… view at source ↗

**Figure 3.** Qualitative comparison on the HyperKvasir [1] and SurgVisdom [31] datasets with different conditions. state-of-the-art results across various key metrics. These results validate that EndoGen could effectively leverage the long-range dependency modeling of autoregressive models in diverse scenarios in endoscopic video generation. Ablation Studies. In [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Qualitative results of semi-supervised polyp segmentation. 3.3 Downstream Task: Semi-supervised Polyp Segmentation Semi-supervised medical image segmentation is an essential approach that reduces the labeling cost for improved performance [14]. To evaluate the fidelit…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 19 canonical work pages

[1]

Sci- entific data 7(1), 283 (2020)

Borgli, H., Thambawita, V., Smedsrud, P.H., Hicks, S., Jha, D., Eskeland, S.L., Randel, K.R., Pogorelov, K., Lux, M., Nguyen, D.T.D., et al.: Hyperkvasir, a com- prehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci- entific data 7(1), 283 (2020)

work page 2020
[2]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion mod- els. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7310–7320 (2024)

2024
[3]

arXiv preprint arXiv:2412.14169 (2024)

Deng, H., Pan, T., Diao, H., Luo, Z., Cui, Y., Lu, H., Shan, S., Qi, Y., Wang, X.: Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412.14169 (2024)

arXiv 2024
[4]

In: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP)

Diamantis, D.E., Gatoula, P., Iakovidis, D.K.: Endovae: Generating endoscopic images with a variational autoencoder. In: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP). pp. 1–5. IEEE (2022)

work page 2022
[5]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12873–12883 (2021)

2021
[6]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Ge, S., Mahapatra, A., Parmar, G., Zhu, J.Y., Huang, J.B.: On the content bias in fréchet video distance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7277–7288 (2024)

work page 2024
[7]

Advances in neural information processing systems30 (2017) 10 Liu et al

Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems30 (2017) 10 Liu et al

work page 2017
[8]

Advances in Neural Information Processing Systems35, 8633– 8646 (2022)

Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Advances in Neural Information Processing Systems35, 8633– 8646 (2022)

work page 2022

Show all 31 references

[9]

Computers in Biology and Medicine170, 108006 (2024)

Jia, X., Shen, Y., Yang, J., Song, R., Zhang, W., Meng, M.Q.H., Liao, J.C., Xing, L.: Polypmixnet: Enhancing semi-supervised polyp segmentation with polyp-aware augmentation. Computers in Biology and Medicine170, 108006 (2024)

2024
[10]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Lee, D., Kim, C., Kim, S., Cho, M., Han, W.S.: Autoregressive image genera- tion using residual quantization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11523–11532 (2022)

2022
[11]

arXiv preprint arXiv:2403.11050 (2024)

Li, C., Liu, H., Liu, Y., Feng, B.Y., Li, W., Liu, X., Chen, Z., Shao, J., Yuan, Y.: Endora: Video generation models as endoscopy simulators. arXiv preprint arXiv:2403.11050 (2024)

2024 arXiv
[12]

Advances in Neural Information Processing Systems37, 56424–56445 (2025)

Li, T., Tian, Y., Li, H., Deng, M., He, K.: Autoregressive image generation with- out vector quantization. Advances in Neural Information Processing Systems37, 56424–56445 (2025)

2025
[13]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Li, W., Liu, X., Yang, Q., Yuan, Y.: From static to dynamic diagnostics: Boosting medical image analysis via motion-informed generative videos. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 195–205. Springer (2024)

2024
[14]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Liu, X., Li, W., Yuan, Y.: Diffrect: Latent diffusion label rectification for semi- supervised medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 56–66. Springer (2024)

2024
[15]

IEEE Transactions on Medical Imaging41(7), 1897–1908 (2022)

Liu, X., Yuan, Y.: A source-free domain adaptive polyp detection framework with style diversification flow. IEEE Transactions on Medical Imaging41(7), 1897–1908 (2022)

2022
[16]

arXiv preprint arXiv:2401.03048 (2024)

Ma, X., Wang, Y., Jia, G., Chen, X., Liu, Z., Li, Y.F., Chen, C., Qiao, Y.: Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048 (2024)

2024 arXiv
[17]

Journal of Systems Architecture115, 102041 (2021)

Mittal, S., et al.: A survey of accelerator architectures for 3d convolution neural networks. Journal of Systems Architecture115, 102041 (2021)

2021
[18]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Sharma, V., Kumar, A., Jha, D., Bhuyan, M.K., Das, P.K., Bagci, U.: Con- trolpolypnet: towards controlled colon polyp synthesis for improved polyp segmen- tation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2325–2334 (2024)

2024
[19]

arXiv preprint arXiv:2209.14792 (2022)

Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al.: Make-a-video: Text-to-video generation without text- video data. arXiv preprint arXiv:2209.14792 (2022)

2022 arXiv
[20]

Advances in neural information processing systems33 (2020)

Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C.A., Cubuk, E.D., Kurakin, A., Li, C.L.: Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems33 (2020)

2020
[21]

arXiv preprint arXiv:2406.06525 (2024)

Sun, P., Jiang, Y., Chen, S., Zhang, S., Peng, B., Luo, P., Yuan, Z.: Autoregres- sive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525 (2024)

2024 arXiv
[22]

Tian, K., Jiang, Y., Yuan, Z., Peng, B., Wang, L.: Visual autoregressive modeling: Scalableimagegenerationvianext-scaleprediction.Advancesinneuralinformation processing systems 37, 84839–84865 (2025)

2025
[23]

arXiv preprint arXiv:2302.13971 (2023) EndoGen: Conditional Autoregressive Endoscopic Video Generation 11

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023) EndoGen: Conditional Autoregressive Endoscopic Vi...

2023 arXiv
[24]

arXiv preprint arXiv:1812.01717 (2018)

Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717 (2018)

2018 arXiv
[25]

In: International Conference on Med- ical Image Computing and Computer-Assisted Intervention

Wang, Z., Liu, C., Zhang, S., Dou, Q.: Foundation model for endoscopy video anal- ysis via large-scale self-supervised pre-train. In: International Conference on Med- ical Image Computing and Computer-Assisted Intervention. pp. 101–111. Springer (2023)

2023
[26]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Xing, Z., Dai, Q., Hu, H., Wu, Z., Jiang, Y.G.: Simda: Simple diffusion adapter for efficient video generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7827–7839 (2024)

2024
[27]

IEEE transactions on multi- media 22(1), 229–241 (2019)

Yan, C., Tu, Y., Wang, X., Zhang, Y., Hao, X., Zhang, Y., Dai, Q.: Stat: Spatial- temporal attention mechanism for video captioning. IEEE transactions on multi- media 22(1), 229–241 (2019)

2019
[28]

arXiv preprint arXiv:2104.10157 (2021)

Yan, W., Zhang, Y., Abbeel, P., Srinivas, A.: Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157 (2021)

2021 arXiv
[29]

In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision

Yellapragada, S., Graikos, A., Prasanna, P., Kurc, T., Saltz, J., Samaras, D.: Pathldm: Text conditioned latent diffusion model for histopathology. In: Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 5182–5191 (2024)

2024
[30]

In: Proceedings of the IEEE conference on computer vision and pattern recognition

Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018)

2018
[31]

arXiv preprint arXiv:2102.13644 (2021)

Zia, A., Bhattacharyya, K., Liu, X., Wang, Z., Kondo, S., Colleoni, E., van Amster- dam, B., Hussain, R., Hussain, R., Maier-Hein, L., et al.: Surgical visual domain adaptation: Results from the miccai 2020 surgvisdom challenge. arXiv preprint arXiv:2102.13644 (2021)

2021 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Sci- entific data 7(1), 283 (2020)

Borgli, H., Thambawita, V., Smedsrud, P.H., Hicks, S., Jha, D., Eskeland, S.L., Randel, K.R., Pogorelov, K., Lux, M., Nguyen, D.T.D., et al.: Hyperkvasir, a com- prehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci- entific data 7(1), 283 (2020)

work page 2020

[2] [2]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Chen, H., Zhang, Y., Cun, X., Xia, M., Wang, X., Weng, C., Shan, Y.: Videocrafter2: Overcoming data limitations for high-quality video diffusion mod- els. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7310–7320 (2024)

2024

[3] [3]

arXiv preprint arXiv:2412.14169 (2024)

Deng, H., Pan, T., Diao, H., Luo, Z., Cui, Y., Lu, H., Shan, S., Qi, Y., Wang, X.: Autoregressive video generation without vector quantization. arXiv preprint arXiv:2412.14169 (2024)

arXiv 2024

[4] [4]

In: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP)

Diamantis, D.E., Gatoula, P., Iakovidis, D.K.: Endovae: Generating endoscopic images with a variational autoencoder. In: 2022 IEEE 14th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP). pp. 1–5. IEEE (2022)

work page 2022

[5] [5]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12873–12883 (2021)

2021

[6] [6]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Ge, S., Mahapatra, A., Parmar, G., Zhu, J.Y., Huang, J.B.: On the content bias in fréchet video distance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7277–7288 (2024)

work page 2024

[7] [7]

Advances in neural information processing systems30 (2017) 10 Liu et al

Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems30 (2017) 10 Liu et al

work page 2017

[8] [8]

Advances in Neural Information Processing Systems35, 8633– 8646 (2022)

Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Advances in Neural Information Processing Systems35, 8633– 8646 (2022)

work page 2022