ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming

Bowei Xu; Hao Chen; Xun Cao; Zehao Cao; Zhan Ma

REVIEW 3 major objections 4 minor 54 references

ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming

T0 review · 3 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rank-ordered prompts cut loss-induced quality drops by up to 95%

desk verdict The dropout-trained rank-ordered prompt idea is genuinely useful and the prefix-truncation result holds, but the headline robustness claim outruns the evidence because packet-loss simulation doesn't match the prefix-only training. read the letter →

arxiv 2607.26106 v1 pith:RA3MZJ6U submitted 2026-07-28 eess.IV cs.CVcs.MM

Zehao Cao , Bowei Xu , Xun Cao , Zhan Ma , Hao Chen This is my paper · ORCID

classification eess.IVcs.CVcs.MM

keywords prompt-basedvideostreaminggenerativecodingdropouttrainingrank-orderedrepresentationpacketlossresiliencelow-rankpromptembeddingsphericallinearinterpolation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that prompt-based video streaming—transmitting compact semantic embeddings that a generative model turns back into frames—can be made resilient to network loss, the main obstacle to real deployment. The core idea is a dropout training strategy that forces a low-rank prompt to store the most critical visual information in its leading dimensions. Once prompt dimensions are rank-ordered, a receiver can use just the first few received dimensions and still produce meaningful frames, and extra dimensions can arrive later as a quality upgrade without retraining. Under packet loss, the authors report the quality degradation is reduced by 82–95% relative to the baseline, while under stable networks the additions give modest gains. They also add semantic and color alignment losses and spherical interpolation to improve fidelity.

What carries the argument

The mechanism is the rank-ordered prompt representation learned by prefix-preserving dropout. In each training step the method truncates the low-rank factors U,V to their first k components, reconstructs the prompt as U_{:,:k}V_{:k,:}/√k, and back-propagates only through the kept prefix, while auxiliary regularization keeps the unselected trailing dimensions small. The resulting ordering makes c=U_{:,:k}V_{:k,:}/√k a valid reconstruction for any k at inference. Secondary components—a semantic cosine loss from a frozen pretrained image encoder, per-channel mean/variance color alignment, and spherical linear interpolation between keyframes—improve fidelity and intermediate frames but do not ca

What would settle it

Run the packet-loss experiment with the first dimension (or first k dimensions) forced to be lost and all trailing dimensions present. If LPIPS degrades sharply in that case, the rank-ordered representation is only robust to clean prefix truncation, and the 82–95% degradation-reduction figure does not extend to arbitrary packet loss.

Watch

Extended reading notes

Core claim

The central claim is that a single trained pair of low-rank prompt factors can serve every effective rank at once. During training, each iteration randomly keeps only a prefix of the dimensions; the reconstruction uses the kept prefix scaled by 1/√k, and gradients flow only through that prefix. This teaches dimensions to be ordered by importance: leading dimensions carry coarse, essential visual content; trailing dimensions add fine detail. At inference, if only k dimensions arrive, the receiver reconstructs the prompt from the prefix and generates normally, no adaptation needed. The paper reports that this turns truncation from catastrophic failure into graceful degradation, cutting the per

Load-bearing premise

The method's robustness rests on the assumption that the first (most important) dimensions are reliably delivered, since training only ever drops trailing suffixes and the evaluation never tests what happens when a leading dimension is lost.

Editorial extensions

If this is right

A single dropout-trained prompt model can serve any effective rank from 4 to 16, so the sender no longer needs to store or transmit multiple rank-specific prompts.
Mid-stream bandwidth increases become incremental: the sender transmits only additional trailing dimensions, and the receiver appends them to the already-received prefix.
Under bursty packet loss between 5% and 20%, reconstruction quality stays near its stable-network level, while the fixed-rank baseline degrades sharply.
The cost is a small full-rank fidelity loss at the trained rank, traded for robustness at lower ranks.
Receiver-side decoding remains unchanged, just prefix reconstruction followed by the normal generation path.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A decisive stress test goes beyond the paper: force the first dimension (or first few) to be lost while all later dimensions arrive; if quality collapses, the robustness claim is really about prefix truncation, not arbitrary packet loss.
The same prefix-preserving dropout idea could be applied to other latent or token-based generative codecs, turning layered/truncatable representations into a general property of the training objective rather than a special case.
Because prompt bitrate is independent of output resolution, the rank-ordered prompts could be combined with unequal error protection: send leading dimensions over a protected channel and trailing dimensions opportunistically, fully exploiting the graceful-degradation property.
A practical deployment would need a rate-control policy that decides when to send the base prefix and when to append refinement dimensions; the paper does not design that policy, but its representation makes it possible.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know: this paper has a solid core idea. Training prompt factors with suffix-only dropout so that a single model supports arbitrary prefix ranks at inference is a practicul integration, and the truncation curves directly show it works. The semantic/color losses and Slerp are individually known, but they're sensibly combined, and the ablation at r=2 is a nice touch. The code link is a plus.

The main soft spot is real. Algorithm 1 always truncates to a prefix, so the learned property is specifically that leading dimensions alone are self-sufficient and trailing ones refine. But Sec 4.3 simulates packet loss by zeroing lost dimensions, which can create non-prefix holes. The paper never evaluates the worst case where dimension 1 is lost, and Sec 5 implicitly concedes the point by saying leading dimensions must be reliably delivered and proposing FEC. That means the 'arbitrarily truncated' claim in the abstract is too strong; it holds for clean prefix truncation only.

The 82–95% figure is also not backed by the packet-loss tables. That number comes from the rank-truncation experiment (Fig 4), not the Gilbert-Elliott tables, which show smaller reductions—roughly 55% at 5% loss and 75% at 20% in the main GE table. A reader could be misled into thinking packet-loss resilience is stronger than it is. Minor: main tables lack error bars, and the full model slightly trails Promptus under stable networks, which is expected but should be stated more prominently.

Overall, the central mechanism is plausible and the prefix-truncation result is real. The paper is for people in generative compression and adaptive streaming. It deserves a serious referee, but the authors should be asked to align claims with evidence and either evaluate non-prefix loss or clearly scope the method to truncation. My recommendation: send for peer review with major revision.

Referee Report

3 major / 4 minor

Summary. The paper presents ScalablePromptus, an extension of the Promptus prompt-based video streaming framework. The authors add (i) a semantic similarity loss and a color statistics alignment loss to the prompt inversion objective, (ii) spherical linear interpolation (Slerp) for intermediate frames, and (iii) a dropout training strategy intended to produce rank-ordered prompt representations so that a single trained prompt can be truncated to any lower effective rank at inference without retraining. The paper reports modest gains over Promptus under stable network conditions and claims an 82--95% reduction in truncation-induced degradation under lossy conditions, evaluated on QST and UVG at 512x512, 30 fps, with Stable Diffusion v1.5 as the generative backbone.

Significance. If the central mechanism holds, the dropout-based rank ordering is a genuinely useful idea for generative streaming: it promises incremental quality refinement and graceful degradation from a single trained model, avoiding the storage and re-download costs of pre-fitting multiple ranks. The paper ships code, presents a clear algorithm, and evaluates against standard codecs and Promptus. The semantic/color losses and Slerp are well motivated, and the ablations in Table 3 show small but consistent gains from each component. However, the evidence currently supports a narrower claim than the abstract makes: the training distribution is prefix-only, while the packet-loss experiments allow arbitrary dimension loss, and the headline 82--95% reduction is not supported by the numerical tables in the lossy-channel experiment.

major comments (3)

[§3.3 (Algorithm 1), §4.3, §5] Algorithm 1 always selects a prefix length k and truncates to U[:,:k], V[:k,:], with gradients flowing only through that prefix. The packet-loss experiment in §4.3, however, states that lost dimensions are 'simply set to zero', which can delete leading dimensions. Because training never presents non-prefix holes, nothing in the method guarantees graceful behavior if, e.g., the first dimension is lost. The Gilbert-Elliott tables report only average LPIPS and do not condition on which dimensions were lost, so the claim that 'arbitrarily truncated prompts' are handled is not established. Section 5 implicitly concedes this by stating that leading dimensions 'must be reliably delivered' and proposing forward error correction. Please either (a) evaluate non-prefix loss conditioning on the position of lost dimensions and show bounded degradation, or (b) restrict the claims and the packet-loss p
[§4.2 and Abstract] The headline '82%–95% reduction in degradation' is not tied to a numerical table. Section 4.2 reports only 'an absolute degradation reduction of approximately 0.35–0.37' and Figure 4 shows curves without per-rank LPIPS values for both methods. Moreover, the Gilbert-Elliott experiments in Table 1 yield reductions of roughly 55%, 63%, and 75% at 5%, 10%, and 20% loss (computed from the reported LPIPS: at 20% loss Promptus degrades by 0.1584 vs. 0.0402 for ScalablePromptus, a 74.6% reduction), not 82–95%. Since the abstract and conclusion state 'under lossy conditions' with the 82–95% figure, the reader cannot tell whether this claim refers to clean rank truncation or to packet loss, and the packet-loss table does not reproduce it. Please provide per-rank LPIPS for both methods and state explicitly which experiments support the 82–95% claim.
[§4.2 and Table 3] The paper attributes truncation robustness to dropout training, but no experiment isolates the dropout mechanism. Table 3 ablates Lsem, Lcolor, and Slerp on top of Promptus, but not dropout; Section 4.2 compares the full method with Promptus, which differs by all proposed components. A controlled comparison between 'Ours w/o dropout' and 'Ours (full)' under rank truncation at k=4,6,...,14 is needed to show that dropout, and not the semantic/color/Slerp enhancements, is responsible for the reported degradation reduction. Similarly, Table 1 compares the full method to Promptus only, so the causal role of dropout in packet-loss resilience is not directly demonstrated.

minor comments (4)

[§1 and Algorithm 1] The introduction says 'a random subset of prompt dimensions is dropped', but Algorithm 1 always drops a trailing suffix after a random prefix length. The wording should say 'a random suffix is dropped (i.e., a random prefix is kept)', because this distinction is central to what the training actually teaches.
[§3.2 and Algorithm 1] Equation/algorithm cross-references are inconsistent: the text refers to 'composite loss (Equation 4)' and Algorithm 1 computes L 'using Equation 4', but the composite loss is Equation (3). Section 3.3's description of auxiliary regularization cites 'line 11' while Algorithm 1 places it on line 14.
[Figure 4 caption] The caption contains a typo: 'Our method remains resonable results' should be 'reasonable'.
[Tables 1, 2, 4] Only Table 3 reports mean±std; the main quantitative tables report point estimates without error bars or a statement about number of seeds. Please add error bars or state explicitly that results are single-seed, so the magnitude of the reported differences can be assessed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the rank-ordered property is trained directly, and the headline comparisons are against an external baseline (Promptus).

full rationale

The central claim is that dropout training produces rank-ordered prompts. This is engineered into the objective: Algorithm 1 truncates to prefixes and back-propagates through them; inference reconstructs c = U_{:,:k} V_{:k,:}/sqrt(k), the exact operator used in training. Evaluating that property at k=4..16 is a design validation, not a prediction derived from an input by construction — the paper does not claim the ordering emerges from an unrelated first-principles theorem. The stable-network and lossy comparisons are against Promptus (Wu et al. 2026), an external baseline with no author overlap, and against H.265/H.266, so no self-citation chain carries the result. The only caveat is a coverage gap, not circularity: training only sees prefix truncation, whereas Sec 4.3 says 'The lost dimensions are simply set to zero,' which can delete leading dimensions; Sec 5 concedes 'these leading dimensions must be reliably delivered.' This limits the generality of the 'arbitrarily truncated' claim but does not make any derivation circular. The semantic/color losses and Slerp are grounded in external cited geometry findings. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' own prior work.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-chosen hyperparameters, differentiable-generator assumptions, and a geometric assumption about prompt embeddings. No new physical or mathematical entities are introduced. The main liability is that the rank-ordering property, which is the core of the method, is assumed to follow from suffix-dropout training and is only measured indirectly through downstream quality.

free parameters (3)

Loss weights (alpha, beta, lambda_reg, w_sem, w_color) = alpha=0.2, beta=0.8, lambda_reg=0.1, w_sem=0.5, w_color=0.3
Hand-set and tuned via the r=2 ablation in Sec A.3; stable-network quality claims depend on these choices, and the ablations show sensitivity to w_sem and w_color.
Dropout training hyperparameters (r_min, r_max, T_w, total iterations, eta, gamma) = r_min=4, r_max=16, T_w=500/100, total=10000/1500, eta=1e-3, gamma=0.7
Chosen by the authors with no sensitivity analysis; the claimed rank-ordered behavior is a direct function of these values.
Keyframe interval K = 10 frames (3 keyframes/s)
Selected from the ablation in Table 5; K controls the bitrate/interpolation tradeoff and shapes all reported bitrate comparisons.

assumptions (5)

domain assumption Stable Diffusion v1.5 with single-step denoising is differentiable w.r.t. prompt embeddings and can be optimized through CLIP and pixel losses.
Sec 3.2 and 3.3 rely on gradients flowing through SD generation back to U and V; no formal guarantee or memory analysis is given.
domain assumption Prompt embeddings live on a hyperspherical manifold where Slerp is the correct interpolation and preserves conditioning when norms are equal.
Sec 3.2, Eq 4 (Slerp). Relies on cited CLIP geometry work; antipodal or near-zero-angle cases (sin theta = 0) are not addressed.
domain assumption Nested-dropout-style suffix truncation induces a rank-ordered representation with leading dimensions carrying coarse critical content.
Sec 3.3 / Algorithm 1, citing Rippel et al. 2014. The paper does not measure ordering directly, only downstream truncation performance.
domain assumption LPIPS, MSE, CLIP cosine, and color-statistics losses are adequate proxies for perceptual fidelity in the low-bitrate regime.
Used in the composite loss Eq 3; no user study is reported, and all conclusions rest on these metrics.
domain assumption 8-bit quantization of U and V does not materially change the rank-ordered property or truncation behavior.
The transmission section quantizes U and V to 8-bit integers but provides no quantization-aware analysis or tolerance experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming." pith.science (2026). https://pith.science/paper/RA3MZJ6U

@misc{pith2026260726106,
  author       = {Pith},
  title        = {Pith review of: ScalablePromptus: Scalable and High-Fidelity Prompt-Based Video Streaming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RA3MZJ6U}},
  note         = {Machine review of arXiv:2607.26106}
}

read the original abstract

Prompt-based video streaming transmits compact semantic prompts instead of pixel-level content for generative reconstruction, enabling ultra-low-bitrate communication. However, the state-of-the-art Promptus framework is vulnerable to network fluctuation, where partially received prompts lead to catastrophic quality collapse. We propose ScalablePromptus, which enhances Promptus with semantic and color-aware prompt inversion, spherical linear interpolation for intermediate frames, and--most critically--a dropout training strategy that produces rank-ordered prompt representations. This allows the receiver to reconstruct meaningful video from arbitrarily truncated prompts without any adaptation. Under stable networks, ScalablePromptus achieves modest quality gains. Under lossy conditions, it reduces the performance degradation caused by truncation by 82%-95% compared to the baseline, making prompt-based streaming robust enough for real-world deployment.

Figures

Figures reproduced from arXiv: 2607.26106 by the authors.

**Figure 2.** The ScalablePromptus pipeline. At the sender, video frames are inverted into rank-ordered low-rank prompt factors [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Illustration of linear interpolation (Lerp) and spher [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Scalability experiments. Left: LPIPS vs. rank. For comparison, we also include results of non-prompt methods under [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 6.** Figure 6: Qualitative ablation of individual components. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 5.** Figure 5: Qualitative comparison under 70 kbps. (full), which incorporates dropout training, performs comparably to Promptus at the low bitrate and slightly trails it at the medium bitrate. This is expected: dropout training forces the model to sacrifice some representational c…

**Figure 8.** Figure 8: Qualitative ablation of [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

**Figure 7.** Figure 7: Qualitative ablation of wsem (wcolor = 0.3 fixed) at r = 2. Left to right: wsem = 0.25, 0.50, 0.75, 1.00. Increasing semantic weight sharpens structure but gradually desaturates color. and the generated frames gradually desaturate, trending toward a grayish appearanc…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 1 canonical work pages

[1]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

Promptus: Can prompt streaming replace video streaming , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
[2]

International Conference on Machine Learning , pages=

Learning ordered representations with nested dropout , author=. International Conference on Machine Learning , pages=. 2014 , organization=

2014
[4]

Scandinavian Conference on Image Analysis , pages=

Addressing degeneracies in latent interpolation for diffusion models , author=. Scandinavian Conference on Image Analysis , pages=. 2025 , organization=

2025
[5]

IEEE Transactions on Circuits and Systems for Video Technology , volume=

Neural video coding using multiscale motion compensation and spatiotemporal context model , author=. IEEE Transactions on Circuits and Systems for Video Technology , volume=. 2020 , publisher=

2020
[6]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Mokady, Ron and Hertz, Amir and Aberman, Kfir and Pritch, Yael and Cohen-Or, Daniel , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2023 , pages =

2023
[7]

2022 , eprint=

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion , author=. 2022 , eprint=

2022
[8]

and Cohen, Taco S

Habibian, Amirhossein and Rozendaal, Ties van and Tomczak, Jakub M. and Cohen, Taco S. , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =
[9]

and Liu, Xue and Kuo, Tei-Wei and Xue, Chun Jason , journal=

Cui, Yufei and Mao, Yu and Liu, Ziquan and Li, Qiao and Chan, Antoni B. and Liu, Xue and Kuo, Tei-Wei and Xue, Chun Jason , journal=. Variational Nested Dropout , year=

Show all 54 references

[10]

Multiuser Content-Style Adaptive Semantic Communication for Image Transmission , year=

Song, Mengshu and Ma, Nan and Liang, Haotai and Dong, Chen and Li, Weizhi and Chen, Jianqiao and Lin, Yijing and Zhang, Ping , journal=. Multiuser Content-Style Adaptive Semantic Communication for Image Transmission , year=
[11]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Wang, Ting-Chun and Mallya, Arun and Liu, Ming-Yu , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2021 , pages =

2021
[12]

2024 , eprint=

Reparo: Loss-Resilient Generative Codec for Video Conferencing , author=. 2024 , eprint=

2024
[13]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\"orn , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2022 , pages =

2022
[14]

and Sullivan, G.J

Wiegand, T. and Sullivan, G.J. and Bjontegaard, G. and Luthra, A. , journal=. Overview of the H.264/AVC video coding standard , year=
[15]

and Ohm, Jens-Rainer and Han, Woo-Jin and Wiegand, Thomas , journal=

Sullivan, Gary J. and Ohm, Jens-Rainer and Han, Woo-Jin and Wiegand, Thomas , journal=. Overview of the High Efficiency Video Coding (HEVC) Standard , year=
[16]

and Ohm, Jens-Rainer , journal=

Bross, Benjamin and Wang, Ye-Kui and Ye, Yan and Liu, Shan and Chen, Jianle and Sullivan, Gary J. and Ohm, Jens-Rainer , journal=. Overview of the Versatile Video Coding (VVC) Standard and its Applications , year=
[17]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Li, Jiahao and Li, Bin and Lu, Yan , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2023 , pages =

2023
[18]

Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , month =

Pourreza, Reza and Le, Hoang and Said, Amir and Sauti\`ere, Guillaume and Wiggers, Auke , title =. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , month =. 2023 , pages =

2023
[19]

Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , month =

Fathima, Noor and Petersen, Jens and Sauti\`ere, Guillaume and Wiggers, Auke and Pourreza, Reza , title =. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , month =. 2023 , pages =

2023
[20]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Li, Jiahao and Li, Bin and Lu, Yan , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2024 , pages =

2024
[21]

A slimmable framework for practical neural video compression , journal =

Zhaocheng Liu and Fei Yang and Defa Wang and Marc. A slimmable framework for practical neural video compression , journal =. 2024 , issn =. doi:https://doi.org/10.1016/j.neucom.2024.128525 , url =

2024
[22]

2023 , eprint=

CaDM: Codec-aware Diffusion Modeling for Neural-enhanced Video Streaming , author=. 2023 , eprint=

2023
[23]

He, Zhaoyuan and Yang, Yifan and Qiu, Lili and Park, Kyoungjun and Yang, Yuqing , title =. Proc. ACM Netw. , month = mar, articleno =. 2024 , issue_date =. doi:10.1145/3649472 , abstract =

2024 doi
[24]

REM: Enabling Real-Time Neural-Enhanced Video Streaming on Mobile Devices Using Macroblock-Aware Lookup Table , year=

Chai, Baili and Wu, Di and Chen, Jinyu and Yang, Mengyu and Wang, Zelong and Hu, Miao , journal=. REM: Enabling Real-Time Neural-Enhanced Video Streaming on Mobile Devices Using Macroblock-Aware Lookup Table , year=
[25]

DoMo: Rethinking Downscaling For Mobile Neural-Enhanced Video Streaming , year=

Zhu, Zhui and Wang, Xu and Xu, Jingao and Zhang, Weichen and Yuan, Yankun and Wang, Lin and Dang, Fan and Liu, Yunhao , booktitle=. DoMo: Rethinking Downscaling For Mobile Neural-Enhanced Video Streaming , year=
[26]

DTVNet: Dynamic Time-Lapse Video Generation via Single Still Image

Zhang, Jiangning and Xu, Chao and Liu, Liang and Wang, Mengmeng and Wu, Xia and Liu, Yong and Jiang, Yunliang. DTVNet: Dynamic Time-Lapse Video Generation via Single Still Image. Computer Vision -- ECCV 2020. 2020

2020
[27]

Proceedings of the 11th ACM Multimedia Systems Conference , pages =

Mercat, Alexandre and Viitanen, Marko and Vanne, Jarno , title =. Proceedings of the 11th ACM Multimedia Systems Conference , pages =. 2020 , isbn =. doi:10.1145/3339825.3394937 , abstract =

2020
[28]

Elliott, E. O. , journal=. Estimates of error rates for codes on burst-noise channels , year=
[29]

J.; and Ohm, J.-R

Bross, B.; Wang, Y.-K.; Ye, Y.; Liu, S.; Chen, J.; Sullivan, G. J.; and Ohm, J.-R. 2021. Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology, 31(10): 3736--3764

2021
[30]

Chai, B.; Wu, D.; Chen, J.; Yang, M.; Wang, Z.; and Hu, M. 2025. REM: Enabling Real-Time Neural-Enhanced Video Streaming on Mobile Devices Using Macroblock-Aware Lookup Table. IEEE Transactions on Mobile Computing, 24(3): 2085--2097

2025
[31]

B.; Liu, X.; Kuo, T.-W.; and Xue, C

Cui, Y.; Mao, Y.; Liu, Z.; Li, Q.; Chan, A. B.; Liu, X.; Kuo, T.-W.; and Xue, C. J. 2023. Variational Nested Dropout. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8): 10519--10534

2023
[32]

Elliott, E. O. 1963. Estimates of error rates for codes on burst-noise channels. The Bell System Technical Journal, 42(5): 1977--1997

1963
[33]

Eslami, S.; and de Melo, G. 2024. Mitigate the gap: Investigating approaches for improving cross-modal alignment in clip. arXiv preprint arXiv:2406.17639

2024 arXiv
[34]

Fathima, N.; Petersen, J.; Sauti\`ere, G.; Wiggers, A.; and Pourreza, R. 2023. A Neural Video Codec With Spatial Rate-Distortion Control. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 5365--5374

2023
[35]

H.; Chechik, G.; and Cohen-Or, D

Gal, R.; Alaluf, Y.; Atzmon, Y.; Patashnik, O.; Bermano, A. H.; Chechik, G.; and Cohen-Or, D. 2022. An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. arXiv:2208.01618

2022 arXiv
[36]

v.; Tomczak, J

Habibian, A.; Rozendaal, T. v.; Tomczak, J. M.; and Cohen, T. S. 2019. Video Compression With Rate-Distortion Autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

2019
[37]

He, Z.; Yang, Y.; Qiu, L.; Park, K.; and Yang, Y. 2024. NERVE: Real-Time Neural Video Recovery and Enhancement on Mobile Devices. Proc. ACM Netw., 2(CoNEXT1)

2024
[38]

Landolsi, E.; and Kahl, F. 2025. Addressing degeneracies in latent interpolation for diffusion models. In Scandinavian Conference on Image Analysis, 16--29. Springer

2025
[39]

Li, J.; Li, B.; and Lu, Y. 2023. Neural Video Compression With Diverse Contexts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 22616--22626

2023
[40]

Li, J.; Li, B.; and Lu, Y. 2024. Neural Video Compression with Feature Modulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 26099--26108

2024
[41]

Li, T.; Sivaraman, V.; Karimi, P.; Fan, L.; Alizadeh, M.; and Katabi, D. 2024. Reparo: Loss-Resilient Generative Codec for Video Conferencing. arXiv:2305.14135

2024 arXiv
[42]

Liu, Z.; Yang, F.; Wang, D.; G \'o rriz Blanch , M.; Murn, L.; Wan, S.; Zhang, S.; Mrak, M.; and Herranz, L. 2024. A slimmable framework for practical neural video compression. Neurocomputing, 610: 128525

2024
[43]

Mercat, A.; Viitanen, M.; and Vanne, J. 2020. UVG dataset: 50/120fps 4K sequences for video codec analysis and development. In Proceedings of the 11th ACM Multimedia Systems Conference, MMSys '20, 297--302. New York, NY, USA: Association for Computing Machinery. ISBN 9781450368452

2020
[44]

Mokady, R.; Hertz, A.; Aberman, K.; Pritch, Y.; and Cohen-Or, D. 2023. NULL-Text Inversion for Editing Real Images Using Guided Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 6038--6047

2023
[45]

Pourreza, R.; Le, H.; Said, A.; Sauti\`ere, G.; and Wiggers, A. 2023. Boosting Neural Video Codecs by Exploiting Hierarchical Redundancy. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 5355--5364

2023
[46]

Rippel, O.; Gelbart, M.; and Adams, R. 2014. Learning ordered representations with nested dropout. In International Conference on Machine Learning, 1746--1754. PMLR

2014
[47]

Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10684--10695

2022
[48]

Song, M.; Ma, N.; Liang, H.; Dong, C.; Li, W.; Chen, J.; Lin, Y.; and Zhang, P. 2025. Multiuser Content-Style Adaptive Semantic Communication for Image Transmission. IEEE Internet of Things Journal, 12(17): 36063--36078

2025
[49]

J.; Ohm, J.-R.; Han, W.-J.; and Wiegand, T

Sullivan, G. J.; Ohm, J.-R.; Han, W.-J.; and Wiegand, T. 2012. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology, 22(12): 1649--1668

2012
[50]

Wang, T.-C.; Mallya, A.; and Liu, M.-Y. 2021. One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10039--10049

2021
[51]

Wiegand, T.; Sullivan, G.; Bjontegaard, G.; and Luthra, A. 2003. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7): 560--576

2003
[52]

Wu, J.; Liu, L.; Tan, Y.; Hao, J.; Zhang, L.; and Zhang, X. 2026. Promptus: Can prompt streaming replace video streaming. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 10664--10672

2026
[53]

Zhang, J.; Xu, C.; Liu, L.; Wang, M.; Wu, X.; Liu, Y.; and Jiang, Y. 2020. DTVNet: Dynamic Time-Lapse Video Generation via Single Still Image. In Vedaldi, A.; Bischof, H.; Brox, T.; and Frahm, J.-M., eds., Computer Vision -- ECCV 2020, 300--315. Cham: Springer International Pu...

2020
[54]

Zhou, Q.; Li, R.; Guo, S.; Dong, P.; Liu, Y.; Guo, J.; and Xu, Z. 2023. CaDM: Codec-aware Diffusion Modeling for Neural-enhanced Video Streaming. arXiv:2211.08428

2023 arXiv
[55]

Zhu, Z.; Wang, X.; Xu, J.; Zhang, W.; Yuan, Y.; Wang, L.; Dang, F.; and Liu, Y. 2025. DoMo: Rethinking Downscaling For Mobile Neural-Enhanced Video Streaming. In IEEE INFOCOM 2025 - IEEE Conference on Computer Communications, 1--10

2025

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

Promptus: Can prompt streaming replace video streaming , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

[2] [2]

International Conference on Machine Learning , pages=

Learning ordered representations with nested dropout , author=. International Conference on Machine Learning , pages=. 2014 , organization=

2014

[3] [4]

Scandinavian Conference on Image Analysis , pages=

Addressing degeneracies in latent interpolation for diffusion models , author=. Scandinavian Conference on Image Analysis , pages=. 2025 , organization=

2025

[4] [5]

IEEE Transactions on Circuits and Systems for Video Technology , volume=

Neural video coding using multiscale motion compensation and spatiotemporal context model , author=. IEEE Transactions on Circuits and Systems for Video Technology , volume=. 2020 , publisher=

2020

[5] [6]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

Mokady, Ron and Hertz, Amir and Aberman, Kfir and Pritch, Yael and Cohen-Or, Daniel , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2023 , pages =

2023

[6] [7]

2022 , eprint=

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion , author=. 2022 , eprint=

2022

[7] [8]

and Cohen, Taco S

Habibian, Amirhossein and Rozendaal, Ties van and Tomczak, Jakub M. and Cohen, Taco S. , title =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , month =

[8] [9]

and Liu, Xue and Kuo, Tei-Wei and Xue, Chun Jason , journal=

Cui, Yufei and Mao, Yu and Liu, Ziquan and Li, Qiao and Chan, Antoni B. and Liu, Xue and Kuo, Tei-Wei and Xue, Chun Jason , journal=. Variational Nested Dropout , year=