{"id":"f3e0d871-cd5a-435f-a99f-31c2bcb8b77d","arxiv_id":"2412.13916","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Raiha retrieves externally similar, illumination-consistent reference images and uses them to improve foreground-background harmonization in composite photos.","lead":"This paper introduces Raiha, a framework that retrieves an external reference image with matching content and lighting to help harmonize a composited foreground object into a background. The approach reports lower error than prior methods on standard benchmarks, but the comparison settings give Raiha access to extra information that other methods do not have.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Non-reference SOTA comparison is contaminated by target-image references used in Raiha's training, so Tables I and II do not isolate the method's contribution.","rationale":"The paper's central claim is state-of-the-art harmonization under both non-reference and retrieval-augmented settings. The retrieval pipeline's illumination proxy (Eqs. 3-5) is a secondary concern, but the decisive issue is experimental attribution. Sec. III-C(c) explicitly trains Raiha with references created from the ground-truth target image, and Sec. III-C(d) randomly mixes non-reference and retrieval-augmented iterations. Consequently, the Table I model has been trained with an auxiliary input channel containing target-derived appearance information, while HDNet, PCT-Net, and other baselines have not. Removing that input at test time does not remove the information already absorbed into the network weights. This is not evidence of dishonesty, and it may even be a legitimate training strategy, but it breaks the controlled comparison required to claim SOTA. Table II has a separate but related problem: comparing a reference-fed model to reference-free baselines measures the extra input modality rather than model quality. Table III's ablations are not sufficient to fix this, since the row labeled 'Raiha non-reference 16.34' uses the fully trained model and no ablation reports the non-reference performance of a model trained without target-augmented references. The proposed retraining test would settle whether Raiha's architecture alone, without target-reference training, still outperforms HDNet. If it does not, the paper needs revision to separate the augmentation benefit from the harmonization network benefit and to compare retrieval-augmented baselines under the same input conditions. The reader's REJECT verdict is therefore sustained.","tokens_in":906,"tokens_out":871,"duration_ms":50458,"concrete_test":"Retrain Raiha on iHarmony4 using only the non-reference branch (disable retrieval-augmented sampling and the target-augmented references from Sec. III-C(c)) and evaluate exactly as in Table I. If the test MSE rises to at or above HDNet's 16.55, the non-reference SOTA claim fails. For the retrieval-augmented claim, additionally run the same retrieved references through a reference-aware variant of a baseline such as HDNet (e.g., concatenate the reference to the background feature input) and check whether Raiha's 15.86 vs 18.72 margin persists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is the evaluation protocol for the non-reference claim, not the retrieval mechanism itself. In Sec. III-C(c), Raiha is trained using 'augmented references' created by randomly cropping, flipping, and resizing the ground-truth target image, and Sec. III-C(d) says non-reference and retrieval-augmented settings are randomly sampled each training iteration. Thus the model evaluated in Table I takes only the composite as input at test time, but its weights were trained with an auxiliary input channel derived from the target image; none of the baselines (HDNet, PCT-Net, etc.) received this privileged information. The claimed non-reference margin (Raiha 15.60 vs HDNet 16.55 MSE) may therefore reflect training-time exposure to target appearance rather than a better harmonization architecture. A similar attribution problem affects Table II: Raiha is tested with a retrieved reference while baselines are tested without any reference, so the comparison conflates an extra input modality with model quality. Table III does not resolve this because it lacks a non-reference ablation trained without target-augmented references. Unless Raiha is trained and evaluated without target-derived references for the non-reference comparison, the central SOTA claim is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Raiha is a retrieval-augmented image harmonization framework. It first retrieves a reference image from a gallery by matching foreground content via DVT dense features and background illumination via HSV histograms, then feeds the composite and reference through a U-Net equipped with a Semantic-Guided Fusion module that uses the DVT features to constrain cross-attention. For training, the authors combine retrieved references with augmented references created by randomly cropping, flipping, and resizing the ground-truth target image, and they randomly mix non-reference and retrieval-augmented iterations. Evaluations are on iHarmony4 (non-reference) and a newly constructed RAHarmony4 benchmark (retrieval-augmented), where the paper claims state-of-the-art MSE/PSNR in both settings.","tokens_in":12973,"tokens_out":5371,"duration_ms":48039,"significance":"If the comparisons were valid, the contribution would be substantive: an external-reference mechanism for harmonization, a new benchmark, and a reusable retrieval criterion. The paper also has useful ideas, notably reusing DVT features for both retrieval and attention guidance and using target-derived references to teach reference utilization. These strengths are undermined, however, by the experimental protocol: the non-reference comparison is contaminated by training-time target-derived references, the retrieval-augmented comparison is asymmetric, and the benchmark gallery is idealized. The fixes are within the scope of a revision, but the current evidence does not establish the central state-of-the-art claim.","major_comments":[{"comment":"The non-reference comparison is not a controlled test of the architecture. In Sec. III-C(c), the training protocol creates 'augmented references' by randomly cropping, flipping, and resizing the ground-truth target image, and Sec. III-C(d) says non-reference and retrieval-augmented settings are randomly sampled each training iteration. Thus the model evaluated in Table I is trained with an auxiliary reference stream that is derived from the answer image, while none of the baselines receive this privileged signal. The reported margin (Raiha 15.60 vs. HDNet 16.55 MSE) could therefore reflect training-time exposure to target appearance rather than a better harmonization architecture. Please provide a non-reference ablation trained without any target-derived references and report its iHarmony4 numbers.","section":"§III-C(c), Table I"},{"comment":"The retrieval-augmented comparison is also asymmetric: Raiha receives a retrieved reference at inference, while HDNet, PCT-Net, and the other baselines receive only the composite image. Table II therefore conflates an additional input modality with model quality and does not support the claim that Raiha is the best harmonization network. To make the comparison meaningful, either give the baselines the same retrieved reference (for example by concatenating it as in Eq. (7)) or evaluate a completely non-reference version of Raiha on the same inputs. The 'Raiha non-reference' row in Table III is not a clean non-reference model because it shares the same training protocol with target-derived references.","section":"§IV-C, Table II"},{"comment":"The RAHarmony4 gallery is constructed from iHarmony4 target images, and the testing set is a subset of those images. Because every reference is a ground-truth target image, the retrieval-augmented evaluation is performed in an idealized setting where the gallery contains perfectly harmonized images rather than arbitrary unprocessed images. Please clarify whether the gallery overlaps with the testing composites, and if so quantify the number of retrievals that return the query target or a near-duplicate; ideally, evaluate with a held-out gallery that does not contain the target images used for testing.","section":"§IV-A, RAHarmony4 construction"},{"comment":"The retrieval criterion is not validated independently. The thresholds epsilon_c=0.7 and epsilon_a=0.9 are set empirically with no sensitivity analysis, and there is no quantitative measure of retrieval precision or of whether 'illumination consistent' references actually improve harmonization beyond a random reference. Table III only varies epsilon_a in coarse bins; it does not report retrieval recall or precision. Please add a retrieval evaluation (e.g., precision@k against human-labeled content and illumination consistency) and an ablation with random or content-only references.","section":"§III-B, Eqs. (3)-(5); Table III"}],"minor_comments":[{"comment":"There is a typo 'andetc' in the abstract that should be 'and etc.'","section":"Abstract"},{"comment":"The benchmark name is inconsistent: 'RAHarmony4' appears as 'Raiharmony4' in Sec. I and as 'RAHarnomy4' in Sec. IV-B. Please standardize the spelling.","section":"§IV-B and throughout"},{"comment":"HDNet is cited as [10] in Sec. II-A but as [48] in Sec. IV-C, while reference [48] is listed as LEMART, not HDNet. Please unify the citations.","section":"§IV-C"},{"comment":"Table I reports no standard deviations for Raiha, while Table II does; please report mean and standard deviation over multiple runs for all tables.","section":"Table I"},{"comment":"The sentence 'Note that augmented references encompass the foreground region' is unclear: because the augmented references are random crops of the target image, they often contain the foreground object itself. This should be stated explicitly, since it is directly relevant to the training-contamination concern in the non-reference evaluation.","section":"§III-C(c)"}],"recommendation":"major_revision","confidential_remarks":"The evaluation-protocol problems are load-bearing but fixable: retraining a reference-free variant, making the retrieval-augmented comparison symmetric, and clarifying the gallery construction would allow the central claims to be tested. I do not see evidence of intentional misconduct, but the current manuscript should not be accepted without these corrections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — the thing to know about this paper is that the method is genuinely new and the evaluation is genuinely lopsided. Raiha is a retrieval-augmented harmonization framework: it retrieves an external reference with similar content to the foreground and similar illumination to the background, then feeds it into a U-Net with a semantic-guided fusion module that reuses DVT features to keep attention on relevant regions. That is a sensible combination, and the RAHarmony4 benchmark (reorganized from iHarmony4 targets) is a useful resource for the subfield.\n\nThe paper does a few things well. The retrieval pipeline is explicit about what a good reference is — content from the foreground, illumination from the background — and the Retinex-based justification for using HSV histograms as an illumination proxy is reasonable, even if the thresholds are empirical. The semantic-guided fusion is a neat way to inject content priors into cross-attention. The data augmentation using the target image itself as a reference is clever for training a reference-augmented model without new annotations.\n\nThe soft spot is load-bearing. In Sec. III-C(c) and (d), Raiha is trained with \"augmented references\" — random crops, flips, and resizes of the target image — and non-reference and retrieval-augmented settings are randomly sampled per iteration. So the model evaluated in Table I (non-reference) has had the target appearance as an auxiliary input during training, while HDNet, PCT-Net, etc. were not given that. The claimed margin (15.60 vs 16.55 MSE) may come from that privileged training signal, not from the architecture. Table II has a different but related problem: Raiha is tested with a retrieved reference while every baseline is tested without one, so the comparison conflates an extra input modality with model quality. Table III's \"Raiha non-reference\" row helps a bit, but it is on RAHarmony4, not iHarmony4, and there is no ablation of a model trained without target-derived references. The retrieval-augmented evaluation also assumes a gallery of ground-truth target images, which is an idealized deployment setting.\n\nMy bottom line: the paper deserves a serious referee, but the central SOTA claim is not established. The fix is clear — retrain without target-augmented references for the non-reference comparison, and give baselines references (or explicitly frame Table II as an oracle-reference study). If the authors do that, the result could be publishable; right now I would not accept it as is. I'd bring it to a reading group to discuss the evaluation trap, and I'd cite the benchmark if they release it.","headline":"Raiha has a solid retrieval-based idea but the SOTA claim is undercut by an evaluation that gives the method target-image references in training and gives baselines no references at all.","tokens_in":13476,"tokens_out":3484,"would_cite":true,"duration_ms":31185,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that retrieving a single illumination-matched reference image can make image harmonization substantially more accurate, and reports new best results in both the with-reference and no-reference settings.","keywords":["image harmonization","retrieval-augmented method","cross-attention","illumination consistency","Retinex theory","dense visual features","semantic-guided fusion","benchmark RAHarmony4"],"falsifier":"Recompute RAHarmony4 metrics with a gallery that omits the query's own ground-truth target image, using only the other 7,403 training images as candidates. If Raiha's average MSE advantage over HDNet at 15.86 versus 18.72 disappears or shrinks below the non-reference gap when the exact target is excluded, then the reported retrieval gain rests on retrieving the ground truth rather than on the illumination-consistency criteria.","tokens_in":12566,"feed_emoji":"🖼️","tokens_out":6603,"duration_ms":52472,"temperature":0.7,"pith_summary":"Image harmonization adjusts a pasted object's brightness and color so it blends into a new background. This paper argues that the usual approach, which relies on similar content already present in the background, is frequently ill-posed because such content is often absent. The proposed Raiha framework instead retrieves one reference image containing an object similar to the foreground and photographed under illumination matching the background, then feeds that reference into a U-Net that treats it as an extension of the background. On the iHarmony4 benchmark Raiha reports an average MSE of 15.60 versus 16.55 for the prior best method HDNet when no reference is used, and on a new retrieval-augmented benchmark, RAHarmony4, it reports 15.86 versus 18.72. If these numbers hold, Raiha is the current state of the art in both settings and a step toward making harmonization work when the background gives no clue.","feed_headline":"Retrieved references make image harmonization state of the art","feed_subtitle":"With one matched reference image, Raiha cuts average MSE from 16.55 to 15.60 and from 18.72 to 15.86 on two benchmarks.","key_machinery":"The central machinery is the harmonization-oriented retrieval pipeline combined with a Semantic-Guided Fusion (SGF) module. The retrieval pipeline uses a pre-trained denoising vision transformer (DVT) to extract dense patch features for content matching, and HSV histograms of image patches for illumination matching, with Retinex theory used to justify that content-plus-appearance determines illumination. The SGF module takes the DVT features ($c_f$, $c_b$, $c_r$) of foreground, background, and reference, concatenates them with the encoder features along the channel dimension, and computes the cross-attention map as $A = \\phi([e_f, c_f]_c) \\times \\phi([[e_b, e_r]_s, [c_b, c_r]_s]_c)$, so that attention is restricted to regions semantically related to the foreground. A data augmentation strategy converts the ground-truth target image into additional references by random cropping, flipping, and resizing, which lets the network learn to use references without overfitting to the exact target.","core_discovery":"The paper's central claim is that image harmonization benefits from an external reference chosen by a retrieval step tailored to the task, and that the same network, trained on a mix of reference and non-reference inputs, also improves the ordinary non-reference setting. The retrieval step has two requirements: the reference must contain an object whose dense visual features match the foreground (cosine similarity above 0.7), and the reference must share illumination with the background, judged by matching DVT content features and HSV patch histograms (threshold 0.9), justified by Retinex theory that appearance separates into reflectance and illumination. A Semantic-Guided Fusion module then concatenates the DVT features with encoder features when forming the cross-attention map, restricting attention to semantically relevant regions of the reference and background. Training alternates between retrieved references and augmented versions of the target image, and the authors report that Raiha outperforms all compared methods on iHarmony4 under the non-reference setting and on the newly constructed RAHarmony4 under the retrieval-augmented setting.","pith_inferences":["Beyond the paper: the hand-set thresholds (0.7 and 0.9) in the retrieval criteria could be replaced by a learned joint embedding of foreground content and background illumination, enabling end-to-end training of retrieval and harmonization.","Beyond the paper: the RAHarmony4 gallery consists of ground-truth target images, an idealized reference source; in a realistic deployment with unrelated gallery images, the retrieved references may be less matching, so the reported gains could shrink—a scenario the paper does not test.","Beyond the paper: the same retrieval-plus-semantic-attention recipe could transfer to other image-editing tasks that need external guidance, such as relighting, shadow removal, or object insertion, though this is speculative.","Beyond the paper: reusing the DVT features for both retrieval and fusion suggests that content features are a reusable asset across the search and generation stages; a single-stage system could be trained to jointly retrieve and harmonize."],"forward_implications":["If Raiha's reported numbers hold, harmonic harmonization no longer requires similar content to be present in the background; a single external reference suffices to reduce ill-posedness.","The same network works in both retrieval-augmented and non-reference settings because training randomly samples both, so deployment can fall back to no reference when retrieval fails.","The Semantic-Guided Fusion module shows that content priors from dense visual features can suppress attention to irrelevant regions when using an external reference.","The RAHarmony4 benchmark, built from iHarmony4 training targets, provides a testbed for retrieval-augmented harmonization that future methods can directly compare against.","Illumination consistency is an essential criterion for retrieval: the paper's ablations show that as the illumination threshold $\\epsilon_a$ is relaxed, performance degrades and can fall below the non-reference setting."],"supporting_citations":[{"why":"Supplies the iHarmony4 dataset, the train/test split, and the composite-image construction used in all experiments.","marker":"[1]"},{"why":"BargainNet is a prior background-guided harmonization method whose training setting Raiha follows.","marker":"[2]"},{"why":"RainNet provides the region-aware adaptive instance normalization baseline and the RAIN evaluation setting referenced in the experiments.","marker":"[3]"},{"why":"Prior work using regional references from the background as guidance, which motivates extending such guidance to external references.","marker":"[8]"},{"why":"SCS-Co is the prior attention-weighted background feature distribution method that Raiha's attention module improves upon.","marker":"[9]"},{"why":"HDNet is the strongest prior baseline in both settings and the source of the foreground MSE loss formulation used to train Raiha.","marker":"[10]"},{"why":"The DVT pre-trained dense visual features are load-bearing for content retrieval, illumination matching, and the Semantic-Guided Fusion attention.","marker":"[14]"}],"fun_headline_variants":["Reference images make harmonization more reliable","Retrieval-augmented harmonization beats all baselines","Harmonize better by retrieving a similar photo","Retrieved references cut harmonization errors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that matching DVT content features and HSV patch histograms above fixed thresholds reliably identifies reference images whose illumination matches the background, and that the RAHarmony4 gallery of ground-truth target images is a realistic stand-in for the references a deployment would actually retrieve; if either fails, the reported gains shrink or reverse.","fun_headline_variants_meta":{"raw":{"variants":["Reference images make harmonization more reliable","Retrieval-augmented harmonization beats all baselines","Harmonize better by retrieving a similar photo","Retrieved references cut harmonization errors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000266,"raw_usage":{"total_tokens":1653,"prompt_tokens":1027,"completion_tokens":626,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":568}},"tokens_in":643,"tokens_out":626,"duration_ms":5839,"temperature":1.0,"reasoning_tokens":568,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:39:43.608871+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute RAHarmony4 metrics with a gallery that omits the query's own ground-truth target image, using only the other 7,403 training images as candidates. If Raiha's average MSE advantage over HDNet at 15.86 versus 18.72 disappears or shrinks below the non-reference gap when the exact target is excluded, then the reported retrieval gain rests on retrieving the ground truth rather than on the illumination-consistency criteria.","supporting_citations":[{"cited_title":"Dovenet: Deep image harmonization via domain verification,","cited_arxiv_id":null,"evidence_quote":"Supplies the iHarmony4 dataset, the train/test split, and the composite-image construction used in all experiments."},{"cited_title":"Bargainnet: Background-guided domain translation for image harmonization,","cited_arxiv_id":null,"evidence_quote":"BargainNet is a prior background-guided harmonization method whose training setting Raiha follows."},{"cited_title":"Region-aware adaptive instance normalization for image harmonization,","cited_arxiv_id":null,"evidence_quote":"RainNet provides the region-aware adaptive instance normalization baseline and the RAIN evaluation setting referenced in the experiments."},{"cited_title":"Image Harmonization by Matching Regional References","cited_arxiv_id":"2204.04715","evidence_quote":"Prior work using regional references from the background as guidance, which motivates extending such guidance to external references."},{"cited_title":"Scs-co: Self-consistent style contrastive learning for image harmonization,","cited_arxiv_id":null,"evidence_quote":"SCS-Co is the prior attention-weighted background feature distribution method that Raiha's attention module improves upon."},{"cited_title":"Hierarchical Dynamic Image Harmonization","cited_arxiv_id":"2211.08639","evidence_quote":"HDNet is the strongest prior baseline in both settings and the source of the foreground MSE loss formulation used to train Raiha."}],"review_version":1}