{"id":"6fc57c86-9907-4f0f-bf0a-1c44d8508d6e","arxiv_id":"2506.10741","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PosterCraft improves text-to-poster generation by cascading four stages of training (text rendering, region-weighted fine-tuning, preference optimization, and vision-language feedback), outperforming open-source baselines and approaching Gemini2.0-Flash-Gen in text accuracy and aesthetics.","lead":"PosterCraft is a four-stage training pipeline that turns the open-source FLUX image model into a poster generator with highly accurate text rendering and cohesive layouts, evaluated to approach commercial systems like Gemini2.0-Flash-Gen. The paper's significance is that it offers an end-to-end, open alternative to modular poster generation pipelines and contributes large synthetic datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OCR evaluation protocol primes Gemini with expected text before OCR, so the Table 1 headline numbers may reflect evaluator bias; independent OCR or human transcription is needed.","rationale":"The reader's Assumption A identified Gemini2.5-Flash's OCR evaluation as the weakest point, and my analysis agrees. The supplementary Prompt 11.1 reveals a specific mechanism that makes this concern concrete: the ground-truth text is inserted into the same context before the OCR step, so the evaluator is not a neutral reader of artistic typography. This is not a disagreement with the field's consensus about Gemini's OCR strength; it is an internal flaw in the measurement protocol. If the concern lands, the Table 1 gap between PosterCraft and open-source baselines, and the near-parity with Gemini2.0-Flash-Gen, could shrink or disappear. That would directly weaken the paper's central quantitative claim. The paper does have independent support: the qualitative comparisons, the ablation study, and the user study all point in the same direction, and the four-stage pipeline is well specified. But none of those provide character-level numbers with the same precision as Table 1, and the user study results are reported only as win-rate plots without numeric scores or confidence intervals. The conditional verdict is therefore appropriate: the approach is plausible and well-motivated, but the headline metric needs independent verification. My recommended verdict remains CONDITIONAL, matching the reader's assessment, so I set verdict_should_be to UNCHANGED.","tokens_in":26574,"tokens_out":3919,"duration_ms":49347,"concrete_test":"Run the same 300-image test set through an independent OCR engine (e.g., PaddleOCR-V4 or TrOCR) with a deterministic string-matching scorer, and also have three human annotators transcribe characters on a stratified 50-image subset. Recompute Table 1 recall, F-score, and accuracy under both protocols, and compare the rank order and the gap to Gemini2.0-Flash-Gen. In addition, run a two-pass Gemini evaluation where the OCR extraction call does not contain the ground-truth strings, and the metrics are computed in a separate call, to isolate the priming effect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the OCR protocol in the supplementary material, Prompt 11.1. The evaluator, Gemini2.5-Flash, is first given the design prompt and instructed to extract the exact ground-truth strings that should appear on the poster; then, in the same context, it is asked to 'Run OCR on the provided image and extract ALL rendered text exactly as it appears.' A multimodal LLM that has just been primed with the expected title, subtitle, and tagline is vulnerable to confirmation bias: it can read distorted, missing, or artistically stylized glyphs as the expected characters. Table 1 reports PosterCraft at 0.787 recall, 0.774 F-score, and 0.735 accuracy, versus Flux1.dev at 0.723/0.707/0.667 and Gemini2.0-Flash-Gen at 0.798/0.786/0.746. These numbers rest on a self-referential evaluation chain: the same commercial model family generates the test prompts, extracts the ground truth, performs the OCR, and computes the errors. No independent OCR engine, no human character-level transcription, and no confidence intervals are reported. Because the priming bias need not affect all models equally, the central claims of 'outperforming open-source baselines' and 'approaching Gemini2.0-Flash-Gen' could be artifacts of the measurement protocol rather than genuine text-rendering gains. This is addressable, and the rest of the paper—visual comparisons, ablations, and a user study—provides partial support, but the quantitative headline depends on the validity of this OCR measurement.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PosterCraft, a unified end-to-end pipeline for aesthetic poster generation built on the Flux.1-dev diffusion backbone. The method consists of four cascaded stages: (i) large-scale text-rendering fine-tuning on a new Text-Render-2M dataset, (ii) region-aware supervised fine-tuning on HQ-Poster-100K with per-pixel loss weights for large text, small text, and background, (iii) aesthetic-text reinforcement learning via best-of-n DPO on Poster-Preference-100K, and (iv) joint vision-language feedback refinement using Poster-Reflect-120K and a fine-tuned InternVL3-8B. The authors claim state-of-the-art performance among open-source systems and near-commercial quality, reporting text recall 0.787, text F-score 0.774, and text accuracy 0.735 on a 300-image Gemini-generated test set, approaching Gemini2.0-Flash-Gen (0.798/0.786/0.746). The paper also reports a user study, a Gemini-based preference evaluation, qualitative comparisons, and ablations, and states that code, models, and datasets will be released.","tokens_in":26858,"tokens_out":3967,"duration_ms":43164,"significance":"If the reported results hold, PosterCraft is a substantial contribution: it demonstrates that a standard diffusion backbone can be trained end-to-end for complex poster generation without modular layout planning, and the four-stage workflow together with the fully automated dataset-construction pipelines could serve as a practical template for design-oriented generation. The release of code, models, and datasets would be valuable to the community. The paper is also careful to include ablation studies for each stage and detailed supplementary prompts. However, the central quantitative claims rest on an evaluation protocol that is vulnerable to confirmation bias, and the evaluation loop uses the same commercial model family for prompt generation, OCR scoring, and preference evaluation. These issues undermine the current evidence for the headline numbers, even though the qualitative results and ablations provide partial support. The contribution is potentially significant, but the quantitative claims need to be verified with independent and unbiased measurement.","major_comments":[{"comment":"The Table 1 headline metrics rely on an OCR protocol in which Gemini2.5-Flash is first given the full design prompt and instructed to extract the exact ground-truth strings, and then, in the same context, is asked to run OCR on the generated image and perform character-level alignment. This primes the evaluator with the expected text and is highly susceptible to confirmation bias: the model may read distorted, missing, or stylized glyphs as the expected characters, and the bias need not affect all models equally. Since the same model family (Gemini) also generated the test prompts and is the leading closed-source comparison target, the reported differences in Table 1 may be artifacts of the measurement protocol rather than genuine text-rendering gains. The authors should repeat the evaluation with an independent OCR engine (e.g., PaddleOCR or Tesseract) and/or human character-level transcription on a random subset, and report per-model confidence intervals for recall, F-score, and accuracy.","section":"Section 4.2 and Supplementary Prompt 11.1"},{"comment":"The Gemini-based preference evaluation (Fig. 6) uses Gemini2.5-Flash as an 'authoritative evaluator' without any validation of its agreement with the human designers who participated in the user study (Fig. 5). Because Gemini2.5-Flash belongs to the same model family as Gemini2.0-Flash-Gen, the win rates in Fig. 6 could be biased toward outputs that match Gemini's aesthetic and text-rendering preferences rather than reflecting true human preferences. The authors should report human-machine agreement (e.g., Cohen's kappa on a shared subset of comparisons) and provide exact win/tie/loss counts with binomial confidence intervals for both the user study and the Gemini evaluation.","section":"Section 4.2 and Figure 6"},{"comment":"The entire quantitative comparison is based on 300 test images (100 prompts, 3 samples per model), and no confidence intervals, standard deviations, or per-prompt variability are reported. Given the small sample size and the fact that the prompts are themselves generated by Gemini2.0-Flash-Gen, the 0.01-0.02 gaps between PosterCraft and Gemini2.0-Flash-Gen in Table 1 are not statistically established. The authors should report bootstrap or other confidence intervals for the three text metrics, and ideally include a human-curated set of poster prompts from diverse real-world sources to test generalization beyond synthetic Gemini-generated prompts.","section":"Section 4.2"}],"minor_comments":[{"comment":"The region-aware calibration weights 0.6, 0.2, and 1.0 are central to the stage-2 loss, but no sensitivity analysis is provided; please report at least one ablation that varies these weights to demonstrate that the chosen values are not brittle.","section":"Section 3.2, Eq. (3)"},{"comment":"The Magic Prompter is described only as 'Qwen3-based'; please specify the exact model and version used at inference, since it is part of the generation pipeline and could affect reproducibility.","section":"Section 4.1"},{"comment":"The main text says the OCR engine is 'the SOTA VLM [30]' while the supplementary specifies Gemini2.5-Flash-Preview-05-20; please disambiguate the exact model and version used for the reported metrics.","section":"Section 4.2 and Supplementary Section 11"},{"comment":"The win-rate plots in Figs. 5 and 6 are difficult to read at print resolution; consider providing the exact numerical win/tie/loss counts in a table in the supplementary material.","section":"Figures 5 and 6"},{"comment":"The limitations section is brief and does not mention the reliance on Gemini-generated data and Gemini-based evaluation; please add a candid discussion of these dependencies and their potential impact on the reported results.","section":"Section 13 (Limitations)"}],"recommendation":"major_revision","confidential_remarks":"The core contribution -- a unified four-stage training workflow with automated dataset construction -- is interesting and potentially impactful, and the authors are to be commended for the scale of the datasets and the detailed supplementary prompts. However, the evaluation loop is troubling: Gemini is used to generate test prompts, extract ground truth, compute OCR metrics, select preference pairs, and serve as the preference evaluator, all in the same paper that compares against Gemini2.0-Flash-Gen. Even a well-intentioned protocol of this kind can produce inflated numbers, and the lack of any independent measurement makes it impossible to assess the true gap to commercial systems. I would recommend asking the authors to provide independent OCR results and human-machine agreement metrics before final acceptance. The paper is otherwise well within the scope of the venue and the methodology is largely sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: PosterCraft is a well-engineered recipe with a real chance of being useful, but the headline numbers should be treated with caution because the OCR evaluator is told what text to expect before it reads the image. The four-stage cascade — synthetic text overlay fine-tuning, region-aware SFT, DPO with HPSv2, and VLM reflection — is a coherent and genuinely new combination, and the datasets are large and documented. The ablations show each stage contributes, and the visual comparisons are striking.\n\nThe soft spot is evaluation. Prompt 11.1 in the supplementary extracts the ground-truth strings from the design prompt, then asks Gemini to OCR the image in the same context. That is a textbook confirmation-bias setup: a multimodal model primed with the expected title will often 'see' it even when glyphs are distorted or missing. Because the same commercial family generates the test prompts (Gemini2.0-Flash-Gen), serves as the top competitor, and supplies the OCR and preference evaluator (Gemini2.5-Flash), the Table 1 numbers are at least partly self-referential. The gap to Gemini (0.787 vs 0.798 recall) is small enough that evaluator bias could flip the ranking. No independent OCR, no human character-level transcription, no confidence intervals. The user study of 20 designers is mentioned but not reported with counts, so we can't weigh it properly.\n\nThat said, the paper isn't a sham. The recipe is specific enough to reproduce once they release code, models, and datasets as promised. The region-aware weighting and the HPSv2-based preference pair construction are reasonable, but a few thresholds (0.6/0.2/1.0, HPSv2 0.25, score-diff 0.025) are asserted without sensitivity analysis. Those are minor compared to the evaluation issue.\n\nBottom line: this deserves a serious referee, but the referee should demand independent OCR (or human transcription), per-pair counts from the user study, and the release of artifacts. If the numbers hold under independent measurement, it's a solid contribution for anyone building on open diffusion backbones. My recommendation: send to peer review with 'revise' — the core method is strong but the empirical foundation needs to be made independent of the vendor being compared.","headline":"A well-engineered poster-generation recipe whose headline numbers rest on a Gemini OCR protocol that primes the evaluator with the expected text — worth engaging, but the Table 1 claims need independent verification.","tokens_in":27492,"tokens_out":4654,"would_cite":true,"duration_ms":47495,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PosterCraft claims that a single diffusion model, trained through four stages, can generate finished aesthetic posters—typography, artistic content, and layout—directly from one prompt.","keywords":["poster generation","text rendering","diffusion models","direct preference optimization","vision-language feedback","region-aware calibration","flow matching","automated dataset construction"],"falsifier":"Have several human transcribers independently transcribe every text string on the 300 test posters and recompute recall, F-score, and accuracy from those transcripts; if the human scores diverge materially from the Gemini-based scores on stylized fonts, the quantitative claim is evaluator-dependent.","tokens_in":26303,"feed_emoji":"🎨","tokens_out":8362,"duration_ms":84505,"temperature":0.7,"pith_summary":"PosterCraft sets out to show that high-quality aesthetic posters do not need modular pipelines, layout planners, or predefined templates: a single diffusion model can generate finished posters, including precise typography and artistic layout, directly from a prompt. The paper builds a four-stage cascaded workflow on the Flux.1-dev backbone—text-rendering fine-tuning on two million synthetic samples, region-aware supervised fine-tuning, best-of-n preference optimization, and vision-language feedback refinement—with each stage supported by an automated dataset construction pipeline. On a 300-image Gemini-generated benchmark, PosterCraft reports text recall 0.787, text F-score 0.774, and text accuracy 0.735, above all tested open-source systems and close to Gemini2.0-Flash-Gen (0.798, 0.786, 0.746). The claim matters because it suggests that strong text rendering and aesthetic coherence can be achieved in an end-to-end generative pass rather than by composing separately generated backgrounds and text overlays.","feed_headline":"Four-step training gives open models commercial-grade poster text","feed_subtitle":"PosterCraft trains one diffusion backbone through four stages to beat modular pipelines on poster typography and layout.","key_machinery":"The load-bearing mechanism is the four-stage cascade itself, applied to the Flux.1-dev diffusion backbone without architectural changes. Stage 1 fine-tunes the model on Text-Render-2M with the flow-matching loss to make text rendering reliable. Stage 2 applies region-aware calibration, which multiplies the flow-matching loss by a per-pixel weight map $w(p)$ that is $0.6$ inside large text masks, $0.2$ inside small text masks, and $1.0$ everywhere else; this is the named identity that balances text fidelity against stylistic coherence. Stage 3 optimizes the DPO objective over best-of-5 and worst-of-5 poster pairs selected by HPSv2 and filtered by a text-accuracy check, injecting global aesthetic preferences that per-pixel losses cannot express. Stage 4 conditions generation on a joint vision-language context $c=[e_p; e_{c,s}; v_{\\text{img}}]$, concatenating the prompt embedding with T5-encoded content and style reflections and a VAE-encoded image feedback signal, and fine-tunes under the conditional flow-matching loss. The cascade is what converts a general text-to-image model into a poster-specific generator.","core_discovery":"PosterCraft's central claim is that a unified, cascaded optimization workflow can unlock the poster-generation ability of a standard diffusion backbone without architectural modification. The paper asserts that each of its four stages addresses a distinct bottleneck: flow-matching fine-tuning on Text-Render-2M supplies reliable text rendering; region-aware calibration with a per-pixel weight map (0.6 for large text, 0.2 for small text, 1.0 elsewhere) preserves text fidelity while letting non-text regions define the poster's aesthetic; aesthetic-text reinforcement learning via DPO on HPSv2-scored best-of-5 and worst-of-5 pairs injects higher-order preferences for layout balance, color harmony, and typographic cohesion; and joint vision-language conditioning, which encodes Gemini-generated content and style reflections through T5 and injects VAE-encoded image feedback, enables iterative refinement. The paper reports that this workflow yields text recall 0.787, F-score 0.774, and accuracy 0.735 on 300 test images, outperforming open-source baselines and approaching the closed-source Gemini2.0-Flash-Gen.","pith_inferences":["The paper evaluates text with Gemini's own OCR; an independent character-level transcriber on the same 300 images would show whether the reported gains are evaluator-specific, a test the paper does not run.","The hand-set region weights (0.6 for large text, 0.2 for small text, 1.0 elsewhere) are a single operating point; the paper reports no sensitivity sweep, so it is unknown how robust the text-aesthetic trade-off is to nearby weight choices.","The best-of-n preference construction selects pairs by HPSv2 score filtered by Gemini text checks; if human aesthetic judgment diverges from HPSv2 on artistic poster styles, the DPO signal could encode a biased notion of quality.","The same cascade could plausibly apply to book covers, UI mockups, or signage, where text and layout harmony matter as much as in posters; the paper does not test these adjacent tasks."],"forward_implications":["End-to-end poster generation becomes practical: a user supplies one prompt, and the model decides text placement, typographic style, and layout without an external layout planner or predefined templates.","The same automated dataset construction—synthetic text-render data, MLLM-filtered poster corpora, HPSv2-scored preference pairs, and Gemini-reflected feedback—can be rebuilt for other text-heavy visual domains, since the paper stresses the pipelines are fully automated.","The reported text metrics imply open-source poster models can approach a commercial system's typography, narrowing the gap that previously required closed APIs.","The vision-language feedback loop at inference allows a generated poster to be critiqued and regenerated, giving designers an iterative refinement mechanism rather than a one-shot output.","The workflow is explicitly transferable to stronger backbones, so future improvements to base diffusion models should compound with the four stages."],"supporting_citations":[{"why":"Supplies the Flux.1-dev diffusion backbone that PosterCraft fine-tunes through all four stages.","marker":"[9]"},{"why":"Provides the direct preference optimization objective used in the aesthetic-text reinforcement learning stage.","marker":"[23]"},{"why":"HPSv2 scores select the best-of-5 and worst-of-5 pairs that form the preference dataset.","marker":"[36]"},{"why":"Gemini supplies OCR metrics, caption and mask generation, and best-of-n selection across the data pipelines and evaluation.","marker":"[30]"},{"why":"Gives the ELBO-based log-ratio formulation that makes DPO applicable to diffusion models.","marker":"[33]"},{"why":"OmniControl's image-conditioning injection is adapted to feed VAE-encoded visual feedback into the diffusion branch.","marker":"[27]"}],"fun_headline_variants":["PosterCraft's four-stage cascade hits commercial-grade typography","Open-source diffusion model reaches near-commercial poster quality","Cascaded training turns vanilla backbone into poster power","One model, four optimizations: poster text done right","PosterCraft: unified pipeline beats modular design systems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All reported text metrics assume Gemini's OCR reads artistic poster typography accurately enough to serve as ground truth, so a systematic OCR blind spot would change the headline numbers.","fun_headline_variants_meta":{"raw":{"variants":["PosterCraft's four-stage cascade hits commercial-grade typography","Open-source diffusion model reaches near-commercial poster quality","Cascaded training turns vanilla backbone into poster power","One model, four optimizations: poster text done right","PosterCraft: unified pipeline beats modular design systems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1577,"prompt_tokens":999,"completion_tokens":578,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":497}},"tokens_in":615,"tokens_out":578,"duration_ms":7206,"temperature":1.0,"reasoning_tokens":497,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:18:25.442114+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have several human transcribers independently transcribe every text string on the 300 test posters and recompute recall, F-score, and accuracy from those transcripts; if the human scores diverge materially from the Gemini-based scores on stylized fonts, the quantitative claim is evaluator-dependent.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Flux.1-dev diffusion backbone that PosterCraft fine-tunes through all four stages."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the ELBO-based log-ratio formulation that makes DPO applicable to diffusion models."}],"review_version":1}