{"id":"d77a8022-4ab5-43f3-851e-cf049ab9d685","arxiv_id":"2608.08460","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"InstructionCrafter generates consistent, step-faithful visual instruction sequences by freezing spatial layers of a video diffusion backbone and adding instruction-aware adapters, improving consistency and per-frame quality over prior methods.","lead":"This paper presents InstructionCrafter, a diffusion model that turns step-by-step text instructions into a sequence of matching pictures, keeping objects and scenes consistent while preserving image quality. It freezes the image-detail layers of a video model and adds two small adapters to carry instruction context across steps, reducing artifacts and training cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'best overall' claim is not robust to the alpha parameter in SC-F: with alpha=1, Stable Diffusion beats InstructionCrafter on ShowHowTo (supplementary Table 1), and alpha=2 was chosen post hoc in Section 4.1.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the SC-F composite with post hoc alpha=2 drives the SOTA ranking, and supplementary Table 1 shows the ranking flips on ShowHowTo at alpha=1. This is not a manufactured objection; the paper's own numbers and supplementary text expose it. The concern is central because the abstract and Section 4.2 claim state-of-the-art overall performance, and the method's main empirical advantage over the fully finetuned Baseline is a consistency-versus-faithfulness trade-off rather than a clear win on all components. The paper has genuine supporting evidence: spatial-freeze training plausibly preserves per-frame quality, the adapters are lightweight and zero-initialized, the parameter reduction from 1.4B to 0.7B is concrete, and the ablations show consistent perceptual improvements. Those contributions can stand, but the headline ranking should be presented as conditional on the chosen weighting and should include error bars and significance tests. Since the reader already returned CONDITIONAL, my read does not change the verdict.","tokens_in":28025,"tokens_out":4931,"duration_ms":58008,"concrete_test":"Fit alpha to human preference on a held-out sample: collect paired human comparisons between InstructionCrafter and the fully finetuned Baseline and Stable Diffusion on ShowHowTo sequences, then estimate the alpha in the SC-F formula that maximizes agreement with human choices, with bootstrap confidence intervals. If the fitted alpha is not near 2, or if InstructionCrafter is not the SC-F winner at the fitted alpha, the 'best overall' claim is conditional on an arbitrary weighting rather than a robust property of the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 4.2 that InstructionCrafter 'achieves the best overall performance on both datasets' rests on the SC-F metric defined in Section 4.1 with alpha=2, justified as 'to balance the empirical ranges' of SF and CIC observed in the authors' own experiments. The supplementary material (B.2) explicitly reports that Stable Diffusion achieves the best SC-F with alpha=1 on ShowHowTo (0.468 vs. 0.435 for InstructionCrafter), so the headline ranking is not invariant to the metric's free parameter. Moreover, on ShowHowTo, InstructionCrafter is not best on the component metrics most directly tied to faithfulness: its SF is 0.455 versus the Baseline's 0.464, and its VLM Semantic score is 0.537 versus the Baseline's 0.555; it wins only on CIC (0.525 vs. 0.510) and on the alpha-weighted aggregate. The paper acknowledges this trade-off in the supplementary, arguing that larger alpha 'better reflects the overall performance,' but that argument is made after observing the rankings. The user study is too small (15 participants, 53% average preference over the Baseline, no significance testing) to independently establish that alpha=2 matches human priorities. Thus the load-bearing premise of the SOTA claim is a post hoc weighting choice, not a settled property of the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InstructionCrafter, a diffusion-based framework for generating step-by-step visual instructions from textual procedure descriptions. The key ideas are (i) spatial-freeze training, which freezes spatial layers of a pretrained video diffusion backbone (VideoCrafter2) while finetuning temporal, cross-attention, and text-conditioning pathways, and (ii) two lightweight adapters: a Consistent Adapter that aggregates global and adjacent step-text context, and a Context-Aware Temporal Adapter that injects step-conditioned biases into temporal self-attention. The method is evaluated on ShowHowTo and WikiHow-VGSI using CLIP-based Step Faithfulness (SF), DINOv3-based Cross-Image Consistency (CIC), a weighted SF-CIC F-score (SC-F), and VLM-based VIEScore semantic/perceptual components. The paper claims state-of-the-art overall performance on both datasets and includes ablations on training strategy, adapter design, and cross-attention finetuning, plus a small user study.","tokens_in":28260,"tokens_out":4132,"duration_ms":42232,"significance":"If the claimed results hold, the paper offers a practical text-only pipeline for generating coherent, faithful, and artifact-free visual instructions, and the spatial-freeze idea is a plausible way to preserve pretrained per-frame quality while adapting temporal and text pathways. The ablations are informative and the adapters are lightweight and zero-initialized. However, the central state-of-the-art claim is not robustly supported: the SC-F metric weighting (alpha=2) is selected after observing the empirical score ranges, and with alpha=1 the paper's own supplementary results show Stable Diffusion ahead on ShowHowTo. In addition, the method does not dominate on the component faithfulness metrics on ShowHowTo, and the most directly competing prior method (Souček et al./ShowHowTo) is not included in the quantitative comparison. These issues collectively mean the paper's headline contribution needs revision rather than acceptance as-is.","major_comments":[{"comment":"The SC-F composite metric uses alpha=2, justified as balancing the empirical ranges of SF and CIC observed in the authors' own experiments. Supplementary Table 1 shows that with alpha=1, Stable Diffusion achieves the best SC-F on ShowHowTo (0.468 vs. 0.435 for InstructionCrafter). Since the headline claim in Section 4.2 that InstructionCrafter 'achieves the best overall performance on both datasets' rests on this weighting, the ranking is not robust to the metric's free parameter. The authors should either justify alpha independently of the observed results, report the full alpha-sweep prominently as the primary result, or temper the claim to a weighted-aggregate statement.","section":"Section 4.1, SC-F definition, and Supplementary Table 1 (B.2)"},{"comment":"Even under the chosen alpha=2, the claimed 'best overall' is not a domination: on ShowHowTo, the Baseline has higher SF (0.464 vs. 0.455) and higher Semantic score (0.555 vs. 0.537), while InstructionCrafter wins only on CIC and Perceptual. The SC-F gap over the Baseline is also small (0.460 vs. 0.455). The manuscript should explicitly state this trade-off structure and avoid implying that the method is uniformly best on faithfulness, consistency, and quality; the strengths are consistency and perceptual quality at a small cost in step-faithfulness on ShowHowTo.","section":"Section 4.2, Table 1"},{"comment":"The paper lists Souček et al. [40] (ShowHowTo) as a compared method category but does not include it in Table 1; instead, it introduces a self-defined Baseline (fully finetuned VideoCrafter2 with per-frame text conditioning). Since Souček et al. use an image-to-video model with first-frame reference, the absence of this directly competing prior method or a faithful adaptation of it leaves the claim of outperforming the actual state-of-the-art visual-instruction method unsubstantiated. At minimum, the authors should either incorporate the original ShowHowTo method (with its reference-image setting) or clearly argue why the setting difference makes it non-comparable, and avoid saying the method outperforms [40] on the basis of Table 1.","section":"Section 4.1, Compared Methods, and Section 4.2"},{"comment":"The user study involves only 15 participants, reports no confidence intervals or significance tests, and the average preference over the Baseline is 53%. This is too weak to independently validate the alpha=2 weighting or the claim of overall superiority in human preference. The authors should add statistical testing (e.g., binomial tests or confidence intervals per baseline) or present the study as a pilot with corresponding caveats.","section":"Section 4.5, Table 7"}],"minor_comments":[{"comment":"The trainable parameter counts are inconsistent: Section 3.2 says '1.4B vs 0.7B', the Introduction says 'from 1.4B to 0.6B', and Table 4 lists 0.6B for Spatial-Freeze and 0.7B for the full model. The authors should unify these numbers and clarify whether 0.6B refers to a specific ablation or a typo.","section":"Section 3.2, Section 1, Table 4"},{"comment":"The modifications from Open-Sora to Open-Sora++ are described only as 'several efforts' and 'adjusting several hyperparameters'; for reproducibility, the authors should list the exact architectural changes and hyperparameter settings.","section":"Section 4.3, Open-Sora comparison"},{"comment":"The statement that improvements 'may seem marginal' in the supplementary is rather important and should be reflected in the main text's discussion of effect sizes and practical significance, rather than only appearing in the appendix.","section":"Supplementary Section B.2"},{"comment":"The caption mentions red/blue marking for best/second-best values, which is conventional, but the actual table formatting in the PDF must be checked so that the colors are distinguishable in grayscale printing; adding bold or symbols would improve clarity.","section":"Table 1 caption"},{"comment":"The CIC metric uses DINOv3 features but the reference [36] is cited as 'Siméoni et al.' with a different spelling in the reference list; please ensure the spelling and citation are consistent.","section":"Section 4.1, CIC metric"}],"recommendation":"major_revision","confidential_remarks":"The core technical idea (spatial-freeze training plus instruction-aware adapters) is interesting and the ablations are well designed, but the paper's central 'state-of-the-art overall' claim is not yet supported given the post hoc alpha=2 choice and the missing direct comparison with the original ShowHowTo method. The authors should be given the opportunity to reframe the claims as consistency/quality improvements under a transparently reported weighting, and to add the missing comparison. I do not see a fatal flaw in the method itself, but the evaluation protocol needs substantive revision before the paper can make its headline claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely useful: freeze the spatial layers of a video diffusion backbone, finetune only temporal and cross-attention pathways, and add two lightweight adapters. That recipe preserves per-frame quality while learning step transitions, cuts trainable parameters roughly in half, and the qualitative artifact reduction is visible. The Consistent Adapter and Context-Aware Temporal Adapter are modest but sensible, and the ablations—causal vs. adjacent attention, cross-attention freeze vs. finetune—are properly done. The paper is honest about limitations and failure cases, which I respect.\n\nThe soft spot is the headline claim. Section 4.2 says InstructionCrafter 'achieves the best overall performance on both datasets,' but that rests on the SC-F composite with alpha=2, chosen after looking at the empirical ranges. The supplementary's own Table 1 shows that with alpha=1, Stable Diffusion wins on ShowHowTo (0.468 vs. 0.435). The method is best only for alpha>=2. Moreover, on ShowHowTo it loses to the Baseline on step faithfulness (0.455 vs. 0.464) and on the VLM semantic score (0.537 vs. 0.555); it wins on the weighted aggregate and on perceptual quality. So the SOTA claim is not a settled property of the method—it is a consequence of the chosen weighting. The authors disclose the alpha sensitivity in the supplementary, which is to their credit, but the main text states the claim without that caveat.\n\nThere are two smaller issues. First, the strongest prior method, ShowHowTo, is not directly compared; the authors use their own 'Baseline' (full finetune of VideoCrafter2) as a stand-in, which is not the same. Second, the user study is small (15 participants) with no significance testing, and there are no error bars anywhere. Code is promised but not released, so the quantitative numbers are not independently checkable yet.\n\nThe method itself is sound enough: on WikiHow-VGSI it beats the Baseline on all three metrics, and the perceptual gains are consistent across ablations. This is a solid systems paper for people working on procedural image generation or step-by-step visual instruction. It deserves peer review, but the authors should be pushed to either report the alpha sensitivity honestly in the main text, add error bars or paired significance tests, and preferably compare against the actual ShowHowTo model, or soften the 'best overall' claim to something like 'best under consistency-weighted evaluation.' With those revisions it would be a good contribution.","headline":"Practical, well-ablated method for visual instruction generation, but the 'best overall' claim rides on a post hoc metric weighting and does not survive alpha=1.","tokens_in":28822,"tokens_out":3058,"would_cite":true,"duration_ms":31207,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Freezing spatial layers of a video diffusion model yields sharper, consistent step-by-step picture guides from text alone.","keywords":["visual instruction generation","video diffusion models","spatial-freeze training","consistent adapter","context-aware temporal adapter","step faithfulness","cross-image consistency","parameter-efficient finetuning"],"falsifier":"Re-run the full evaluation with SC-F computed at $\\alpha=1$ or with a pre-registered user study that weights step faithfulness and cross-image consistency equally; the paper's supplementary Table 1 shows that under $\\alpha=1$ Stable Diffusion, not InstructionCrafter, achieves the best SC-F on ShowHowTo.","tokens_in":27788,"feed_emoji":"🖼️","tokens_out":3593,"duration_ms":33953,"temperature":0.7,"pith_summary":"InstructionCrafter claims that the best way to generate a coherent sequence of step-by-step pictures from text is to keep the spatial layers of a pretrained video diffusion model frozen and train only the temporal and text-conditioning pathways. The paper argues that step faithfulness, cross-image consistency, and per-frame visual quality need not be a fixed trade-off when the model's spatial prior is protected from low-quality training data. Two lightweight adapters then inject instructional context into text and temporal attention, so the model can learn multi-step relations without absorbing dataset artifacts. This matters because it offers a practical text-only route to automatic visual manuals, which could help anyone following procedural instructions without needing reference images.","feed_headline":"Freezing spatial layers yields sharper, consistent visual instructions","feed_subtitle":"A text-only pipeline keeps per-frame quality while aligning each step to its caption.","key_machinery":"The load-bearing mechanisms are spatial-freeze training plus two instruction-aware adapters. Spatial-freeze training keeps the frozen spatial convolutions and spatial self-attention layers that control local texture and high-frequency detail, so the pretrained image-quality prior stays intact while temporal and cross-attention layers adapt to procedural semantics. The Consistent Adapter is a stack of four transformer layers with global attention over all step tokens and adjacent attention over neighboring steps, capturing both task-level dependencies and local transitions. The Context-Aware Temporal Adapter forms a Gram matrix $G_s = C_s C_s^\\top$ from a spatial slice of cross-attention outputs and adds a per-head learnable bias $B_s = \\beta G_s$ to temporal self-attention, so frames with similar step semantics attend to each other; both adapters are zero-initialized to behave as identity at the start of training.","core_discovery":"The central discovery is that separating the optimization of temporal and instructional alignment from per-frame visual quality lets a single model achieve state-of-the-art overall performance on both ShowHowTo and WikiHow-VGSI. Building on VideoCrafter2, the authors freeze all spatial convolutional blocks and spatial self-attention layers, finetune temporal attention and cross-attention on per-step texts, and add the Consistent Adapter and Context-Aware Temporal Adapter. The Consistent Adapter aggregates textual cues from the whole instruction sequence and from neighboring steps so object identity and attributes persist across frames, while the Context-Aware Temporal Adapter converts cross-attention outputs into a step-conditioned bias for temporal self-attention, propagating inter-step relations. This combination reduces trainable parameters from 1.4B to 0.7B while improving VIEScore perceptual quality from 0.662 to 0.742 and achieving the best SC-F, Semantic, and overall balance of scores on both benchmarks.","pith_inferences":["The SC-F meta-metric's weighting is a free parameter, and the paper's own supplementary results show that with $\\alpha=1$, Stable Diffusion leads SC-F on ShowHowTo; a different principled weighting could therefore change the headline ranking.","The Gram-matrix bias mechanism is generic enough to be reused in other multi-condition sequence generation tasks such as storyboards, comic panels, or medical procedure illustrations, where per-frame text conditions vary while visual identity must persist.","A user study that weights step faithfulness more heavily than cross-image consistency might shift preferences toward zero-shot methods, whose main weakness in the current protocol is low consistency rather than low visual quality.","The spatial-freeze recipe could be tested as a general recipe for adapting video diffusion models to any task with per-frame text conditioning, not just visual instructions, whenever the training data is visually noisier than the pretraining data."],"forward_implications":["Text-only visual instruction generation without any reference image can outperform pipelines that tile frames, generate autoregressively, or finetune the full backbone.","Freezing spatial layers reduces trainable parameters by about half and suppresses artifacts such as noise, blur, watermarks, and spurious subtitles learned from low-quality web video.","The two adapters work best together: ablations show that either adapter alone underperforms the full combination on both semantic and perceptual scores.","Diffusion Transformer backbones with 3D VAE temporal compression are less suited to per-step text conditioning, since their compressed latents make step-specific semantics harder to preserve.","The approach generalizes across diverse procedural domains including cooking, DIY, electronics repair, and crafts, as shown on the WikiHow-VGSI zero-shot evaluation."],"supporting_citations":[{"why":"Provides the pretrained VideoCrafter2 backbone whose spatial layers are frozen and whose temporal and cross-attention pathways are finetuned.","marker":"[8]"},{"why":"Contributes the ShowHowTo dataset, the full-finetuning baseline protocol, and the evaluation setting used for training and testing.","marker":"[40]"},{"why":"StackedDiffusion is a key baseline that tile-generates all frames jointly and is compared against on both benchmarks.","marker":"[25]"},{"why":"Bordalo et al. is the zero-shot autoregressive baseline that preserves visual quality but struggles with step faithfulness.","marker":"[5]"},{"why":"VIEScore supplies the VLM-based evaluation protocol with Qwen3-VL as the judge for semantic and perceptual scores.","marker":"[20]"},{"why":"Stable Diffusion is the finetuned text-to-image baseline that generates each step independently without temporal modeling.","marker":"[33]"},{"why":"Prior work reporting the trade-off between image quality and instruction consistency motivates the spatial-freeze design.","marker":"[44]"}],"fun_headline_variants":["Freeze spatial, train temporal: sharper visual steps","Consistent visual instructions with half the parameters","Spatial-freeze trick for high-fidelity step images","Adapters keep object identity across instruction steps","Sharp steps, consistent objects: InstructionCrafter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The state-of-the-art claim depends on the chosen SC-F weighting, where a neutral choice ($\\alpha=1$) changes the ShowHowTo ranking and would put Stable Diffusion ahead.","fun_headline_variants_meta":{"raw":{"variants":["Freeze spatial, train temporal: sharper visual steps","Consistent visual instructions with half the parameters","Spatial-freeze trick for high-fidelity step images","Adapters keep object identity across instruction steps","Sharp steps, consistent objects: InstructionCrafter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1305,"prompt_tokens":1015,"completion_tokens":290,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":217}},"tokens_in":631,"tokens_out":290,"duration_ms":3647,"temperature":1.0,"reasoning_tokens":217,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:35:15.960953+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full evaluation with SC-F computed at $\\alpha=1$ or with a pre-registered user study that weights step faithfulness and cross-image consistency equally; the paper's supplementary Table 1 shows that under $\\alpha=1$ Stable Diffusion, not InstructionCrafter, achieves the best SC-F on ShowHowTo.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pretrained VideoCrafter2 backbone whose spatial layers are frozen and whose temporal and cross-attention pathways are finetuned."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Bordalo et al. is the zero-shot autoregressive baseline that preserves visual quality but struggles with step faithfulness."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VIEScore supplies the VLM-based evaluation protocol with Qwen3-VL as the judge for semantic and perceptual scores."}],"review_version":1}