{"id":"039e00b3-1bc5-4299-a2b1-5c3a76aab06f","arxiv_id":"2603.12252","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Iterative latent thought refinement plus terminal text grounding lets diffusion models solve multi-step visual reasoning tasks at 92.1% average accuracy, beating DiffThinker by 8.3 points.","lead":"EndoCoT trains diffusion image models to reason step by step by repeatedly updating continuous latent “thought” vectors inside a multimodal language model and conditioning the image generator on those states. On synthetic visual puzzles (mazes, TSP, Sudoku, spatial planning) it reaches 92.1% average accuracy and shows intermediate solution steps that prior static-encoder methods lack.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Claim of genuine endogenous CoT hinges on GT intermediate visual targets that risk inducing multi-stage trajectory imitation rather than abstract latent logical refinement.","rationale":"The reader's weakest_assumption correctly isolates the precise load-bearing premise for the 'genuine endogenous CoT' claim. The paper supplies strong synthetic results, public code/data claims, layer-wise analysis (Sec. 3), and supportive ablations (semantic loss, joint training, τ scaling), with no internal contradiction or formal error. However, the supervision design (visual intermediates + terminal text only) does not yet rule out multi-stage imitation, and no latent probing, intermediate-ablation, or non-decomposable OOD logical task is present. This keeps the verdict CONDITIONAL exactly as the reader stated; the concrete test would falsify or corroborate the concern without new benchmarks. No stronger soundness issue found, so no verdict shift.","tokens_in":19129,"tokens_out":656,"duration_ms":21017,"concrete_test":"Retrain Maze (task-specific, same LoRA/hyperparams) under Stage-1 objective but replace every intermediate I*_τ with either (a) the final GT image for all τ or (b) randomly permuted partial paths; retain terminal ℒ_align and Stage-2. Evaluate Maze-32 accuracy and path-repetition at τ=50. If either variant falls to ≤65% accuracy (DiffThinker level) or path-repetition collapses, intermediate visual GT is load-bearing for imitation rather than latent logical CoT.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim (first genuine CoT in diffusion via iterative latent refinement, Sec. 1/4.2, yielding 92.1% avg and large hard-instance gains) requires that recursive latent updates h_τ = e_L+1^T f_φ([P; h_τ-1]) (Eq. 4) plus DiT conditioning (Eq. 5) produce multi-step logical reasoning. Yet each step is supervised by sequential ground-truth visual intermediates I*_τ obtained via path/fill decomposition (partial maze segments, progressive Sudoku cells, etc.; Sec. 4.2.1) and only the terminal state receives L2 text alignment (Eq. 8, ℒ_align). Progressive training (Stage 1 supervises all τ; Stage 2 freezes intermediates) further embeds these visual trajectories. This leaves open that the model primarily learns multi-stage visual imitation of the provided GT path rather than endogenous logical CoT in latent space; the MLLM latents may simply track the supervised sequence while DiT renders progressive images. Existing ablations (Tab. 2: w/o semantic loss; Tab. 3: MLLM/DiT-only) and inference-time τ scaling (Tab. 4) do not isolate intermediate visual supervision necessity or probe whether h_τ encode logic independent of the visual GT trajectory. Unified-training drops reinforce limited abstraction. The conclusion itself flags reliance on high-quality intermediate supervision.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes EndoCoT, a framework that couples an MLLM text encoder with a Diffusion Transformer so that latent thought states are refined iteratively (Iterative Thought Guidance; Eq. 4–6) and the terminal latent is L2-aligned to ground-truth text embeddings (Terminal Thought Grounding; Eqs. 7–8). A two-stage progressive training schedule first supervises all intermediate visual targets obtained by sequential ground-truth decomposition, then consolidates the final output. Built on Qwen-Image-Edit-2511 with LoRA, the method is evaluated on Maze, TSP, Sudoku, and VSP (task-specific and unified settings). Task-specific training reports 92.1% average accuracy, +8.3 pp over DiffThinker, with large gains on hard instances (Maze-32 90% vs 65%; Sudoku-35 95% vs 55%). Supporting analyses include layer-wise sensitivity, attention entropy, ablations of semantic loss and implicit vs explicit tokens, MLLM/DiT-only baselines, and inference-time scaling of the reasoning budget τ.","tokens_in":19559,"tokens_out":870,"duration_ms":15013,"significance":"If the iterative latent mechanism truly induces multi-step logical planning rather than multi-stage visual imitation of provided trajectories, the work would be a meaningful step toward endogenous reasoning inside diffusion generators, with practical value for spatial planning and controllable progressive editing. Strengths that should be credited regardless of interpretation: multi-task tables under both task-specific and unified regimes, systematic ablations (Tabs. 2–4, 6), inference-time CoT scaling curves with latency trade-offs, qualitative intermediate trajectories, layer-wise sensitivity motivating joint MLLM–DiT adaptation, and public code plus a large constructed dataset (182.4K). These make the empirical contribution solid even if the strongest ‘genuine CoT’ framing requires tighter evidence.","major_comments":[{"comment":"Sec. 4.2.1 and Eqs. (4)–(6): each reasoning step is supervised by sequential ground-truth visual intermediates I*_τ (partial maze paths, progressive Sudoku fills, etc.). Terminal text alignment (Eqs. 7–8) is applied only at τ=𝒯. The central claim of ‘genuine endogenous CoT via iterative latent refinement’ (Abstract, Sec. 1, contribution 1) is therefore not isolated from multi-stage visual trajectory imitation. Existing ablations (Tab. 2 semantic loss; Tab. 3 MLLM/DiT-only; Tab. 4 τ scaling) do not remove intermediate visual targets while retaining terminal grounding. A load-bearing experiment is needed: train with only final-image (and/or terminal text) supervision, or probe whether h_τ encode constraint-satisfying logic when the visual GT trajectory is withheld or scrambled.","section":null},{"comment":"Table 1, unified-training block: accuracy drops sharply relative to task-specific training (e.g., Maze-32 90%→52%, TSP-18 73%→46%, Sudoku-35 95%→80%). The paper still frames EndoCoT as learning transferable reasoning (Sec. 5.1). These drops are large enough that claims of abstract, cross-task endogenous CoT need either stronger analysis of what transfers or explicit tempering of the abstraction narrative; otherwise the gains look closer to task-specialized multi-stage imitation.","section":null},{"comment":"Sec. 1 / conclusion: the manuscript asserts the ‘first’ diffusion framework for genuine CoT and contrasts with DiffThinker as ‘superficial alignment.’ Given that both methods use powerful MLLM+DiT backbones and heavy task-specific supervision, and that EndoCoT’s intermediates are algorithmically generated GT decompositions (Appendix B), the novelty claim should be restated operationally (what is endogenous, what is measured) and supported by the isolation experiment above. Without that, the contribution remains a strong empirical system with progressive latent conditioning, not yet a demonstrated cognitive CoT mechanism.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean engineering paper that actually moves the needle on multi-step reasoning inside diffusion models. The core move is simple and useful: keep a continuous latent thought state inside the MLLM, update it recursively, and recondition the DiT at every reasoning step, then ground only the final state with an L2 text alignment. Progressive two-stage training (supervise all steps first, then lock intermediates and polish the terminal) plus LoRA on both sides makes it trainable. They get 92% average on Maze/TSP/Sudoku/VSP under task-specific training, with the big jumps on the hard end (Maze-32 90% vs DiffThinker’s 65%, Sudoku-35 95% vs 55%). Inference-time scaling with more steps works, ablations on the semantic loss and implicit vs explicit tokens are informative, and the layer-sensitivity analysis is a nice diagnostic. Code and data are promised public.\n\nWhat is new is the closed loop of continuous endogenous thoughts that actually drive progressive DiT generation rather than a one-shot encoding or external text CoT. That combination is not just DiffThinker plus latent CoT; the joint adaptation and terminal grounding matter.\n\nThe soft spot is real but not fatal. Every intermediate step is supervised by ground-truth visual partial solutions (path segments, progressive fills). So the model could be learning multi-stage visual imitation of the provided trajectory while the latents simply track that sequence. The ablations do not fully isolate whether the h_τ encode abstract logic independent of those visual targets. Unified training drops, T is hand-chosen, and everything is synthetic. The conclusion itself flags the need for high-quality intermediate supervision. Still, the scaling curves and qualitative step-by-step outputs look better than pure pattern matching, so the claim of “genuine” CoT is a bit strong but the method is still a step forward.\n\nThis is for people working on multimodal generative reasoning and latent CoT in diffusion. It is solid enough for a serious referee; the experiments are thorough and the recipe is reproducible. I would bring it to reading group and cite the method if I am building similar systems. Send it out.","headline":"Practical iterative latent CoT for diffusion that works well on hard synthetic spatial tasks, but the GT intermediate images leave open multi-stage imitation rather than pure endogenous logic.","tokens_in":20127,"tokens_out":541,"would_cite":true,"duration_ms":13310,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Diffusion models can run genuine chain-of-thought reasoning by iteratively refining latent thought states inside the multimodal encoder, not by a single static embedding.","keywords":["endogenous chain-of-thought","diffusion transformer","multimodal large language model","latent thought states","visual reasoning","maze solving","Sudoku","flow matching"],"falsifier":"Train and evaluate an identical architecture on the same hard Maze-32 and Sudoku-35 splits but withhold all intermediate visual targets, supervising only the final image and the terminal thought alignment; if accuracy collapses to the level of static-guidance baselines, the claim that iterative latent refinement induces genuine reasoning is falsified.","tokens_in":20041,"feed_emoji":"🧩","tokens_out":870,"duration_ms":10258,"temperature":0.7,"pith_summary":"Current diffusion models that bolt multimodal language models on as text encoders fail at hard spatial and logical tasks because the encoder thinks only once and then freezes its guidance while the image is denoised. EndoCoT fixes both problems: it lets the encoder repeatedly update a continuous latent thought state, and it ties each of those states to a full denoising pass so the diffusion transformer can act on progressive plans. A final alignment loss keeps the last thought state faithful to the true answer. On maze solving, traveling-salesman tours, Sudoku, and visual spatial planning the method reaches 92 percent average accuracy and still works when maze size or font style changes. The result matters because it shows that multi-step reasoning can live inside the generation process itself rather than being pre-computed and merely rendered.","feed_headline":"Diffusion models learn to think step by step inside the latent loop","feed_subtitle":"Iterative thought states plus terminal grounding lift hard maze and Sudoku accuracy by double digits","key_machinery":"Iterative Thought Guidance: the recursive update h_τ = extract(MLLM([prefix; h_τ−1])) that produces a fresh conditioning vector for a full DiT flow-matching pass at every reasoning step, paired with Terminal Thought Grounding that L2-aligns only the final h_𝒯 to ground-truth answer embeddings.","core_discovery":"The authors show that genuine endogenous chain-of-thought reasoning arises inside a diffusion pipeline when an MLLM is allowed to iteratively refine continuous latent thought states and each state is used to condition a complete denoising trajectory of a DiT; a terminal L2 alignment to ground-truth text embeddings then keeps the trajectory from drifting. This joint iterative mechanism, not single-pass encoding or static guidance, is what lets the model solve complex multi-step visual reasoning tasks step by step and generalize to harder instances.","pith_inferences":["If the intermediate-supervision assumption holds only for algorithmically generated paths, the method may not transfer to open-ended visual planning where no clean partial solutions exist.","The continuous latent tokens could in principle support tree-structured search or backtracking inside the MLLM, turning the current linear chain into a latent planner.","The observed early commitment of vanilla diffusion models suggests that any static-conditioning pipeline, even with stronger MLLMs, will hit the same reasoning-depth wall on combinatorial tasks.","Adaptive stopping of the latent loop, rather than a fixed step budget, would remove the manual hyper-parameter the authors currently leave to the user."],"forward_implications":["Accuracy on long-horizon visual puzzles scales smoothly with the number of latent reasoning steps allocated at inference time.","A single model trained jointly on Maze, TSP, Sudoku and VSP retains competitive performance, indicating transferable latent reasoning skills.","Progressive image editing becomes controllable simply by choosing how many reasoning steps to run, without separate editing modules.","The same latent-refinement loop can be attached to other DiT backbones that already use MLLM text encoders.","High-resolution generation becomes relatively cheaper because the expensive DiT denoising is not repeated for every reasoning step."],"fun_headline_variants":["EndoCoT: latent thoughts refine then ground for DiT step-by-step solve","Diffusion DiT guided by iterative endogenous CoT states hits 92% maze/Sudoku","MLLM latent thoughts iterate then ground; DiT denoise trails them to 92%","Endogenous CoT inside diffusion: refine thought states, ground, solve multi-step","Terminal grounding keeps iterative thought-conditioned DiT from drifting on hard tasks"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That intermediate visual targets obtained by cutting ground-truth solutions into partial paths or progressive fills, plus one final text-embedding alignment, are enough to force real multi-step logic rather than multi-stage visual imitation of the supplied trajectory.","fun_headline_variants_meta":{"raw":{"variants":["EndoCoT: latent thoughts refine then ground for DiT step-by-step solve","Diffusion DiT guided by iterative endogenous CoT states hits 92% maze/Sudoku","MLLM latent thoughts iterate then ground; DiT denoise trails them to 92%","Endogenous CoT inside diffusion: refine thought states, ground, solve multi-step","Terminal grounding keeps iterative thought-conditioned DiT from drifting on hard tasks"]},"model":"grok-4.5","effort":"low","cost_usd":0.001918,"raw_usage":{"total_tokens":961,"prompt_tokens":866,"num_sources_used":0,"completion_tokens":95,"cost_in_usd_ticks":19180000,"prompt_tokens_details":{"text_tokens":866,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":866,"tokens_out":95,"duration_ms":1325,"temperature":1.0,"reasoning_tokens":0,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T22:23:51.467255+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train and evaluate an identical architecture on the same hard Maze-32 and Sudoku-35 splits but withhold all intermediate visual targets, supervising only the final image and the terminal thought alignment; if accuracy collapses to the level of static-guidance baselines, the claim that iterative latent refinement induces genuine reasoning is falsified.","supporting_citations":[],"review_version":1}