{"paper":{"title":"EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesis","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A two-stage SFT and DPO pipeline aligns vision-language models to cut critical errors in image editing instructions from 47% to 23%.","cross_cats":["cs.AI"],"primary_cat":"cs.CV","authors_text":"Chao Hui, Haohua Chen, Hao Shi, Honghao Cai, Tianze Zhou, Wei Zhu, Xiangyuan Wang, Xu Tang, Yao Hu, Yibo Chen, Yuling Wu, Yunhao Bai","submitted_at":"2026-04-09T13:11:33Z","abstract_excerpt":"High-quality source-target image pairs with precise editing instructions are essential for instruction-guided image editing, yet constructing such training triplets at scale remains costly. Recent pipelines often rely on vision-language models to synthesize editing instructions automatically, but we find that strong VLMs still struggle to describe visual transformations between image pairs. In particular, they exhibit three recurring failure modes: orientation inconsistency, viewpoint ambiguity, and missing fine-grained attributes. In a human evaluation on 400 image pairs, several open-source "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"fine-tuned Qwen3-VL models outperform open-source baselines; the 235B model reaches 4.712 on Eval-400 (vs. Gemini-3-Pro 4.706, GPT-4.1 4.220, Kimi-K2.5 4.111) and 4.588 on ByteMorph-Bench (vs. Gemini-3-Pro 4.522, GPT-4.1 3.412). Human evaluation shows critical errors falling from 47.75% to 23% and correctness rising from 41.75% to 66%.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the three identified failure modes (orientation inconsistency, viewpoint ambiguity, insufficient fine-grained attribute description) are the dominant sources of unusable instructions and that the human preference data collected for DPO faithfully captures them without introducing new selection biases or annotation artifacts.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"EditCaption reduces critical errors in automated image editing instructions from 47.75% to 23% via SFT and DPO, yielding fine-tuned models that match or exceed closed-source VLMs on Eval-400 and ByteMorph-Bench.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A two-stage SFT and DPO pipeline aligns vision-language models to cut critical errors in image editing instructions from 47% to 23%.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"909b0db20a362c370bf03b8ee5d6b67936c8b3fd405faabb8c7cda36d1df6e95"},"source":{"id":"2604.08213","kind":"arxiv","version":2},"verdict":{"id":"1457be13-faa7-4e28-aa20-6eb82131bf13","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-10T17:18:58.364884Z","strongest_claim":"fine-tuned Qwen3-VL models outperform open-source baselines; the 235B model reaches 4.712 on Eval-400 (vs. Gemini-3-Pro 4.706, GPT-4.1 4.220, Kimi-K2.5 4.111) and 4.588 on ByteMorph-Bench (vs. Gemini-3-Pro 4.522, GPT-4.1 3.412). Human evaluation shows critical errors falling from 47.75% to 23% and correctness rising from 41.75% to 66%.","one_line_summary":"EditCaption reduces critical errors in automated image editing instructions from 47.75% to 23% via SFT and DPO, yielding fine-tuned models that match or exceed closed-source VLMs on Eval-400 and ByteMorph-Bench.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the three identified failure modes (orientation inconsistency, viewpoint ambiguity, insufficient fine-grained attribute description) are the dominant sources of unusable instructions and that the human preference data collected for DPO faithfully captures them without introducing new selection biases or annotation artifacts.","pith_extraction_headline":"A two-stage SFT and DPO pipeline aligns vision-language models to cut critical errors in image editing instructions from 47% to 23%."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2604.08213/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"221fb1646a566e1c3b87cb96bcb394a36edf1786dc2eeef6261669106aacea27"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}