{"id":"cde47f93-2ad2-416c-bd42-4b86a0baebcf","arxiv_id":"2501.01142","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A3MDA adaptively tunes data augmentation strength and domain alignment weights based on per-sample hardness, improving average accuracy on seven multi-source domain adaptation benchmarks.","lead":"This paper presents a method for multi-source domain adaptation that estimates how hard each image is to classify, then uses that difficulty to control how strongly the image is augmented and how much it influences the alignment between source and target domains. The method, called A3MDA, reports accuracy improvements over prior methods on seven standard image datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical support for 'outperforms other methods' is not established: hyperparameters were tuned on evaluation tasks, no variance is reported, and the paper's own R value is internally inconsistent.","rationale":"I read the paper as an engineering contribution whose central assertion is empirical. The mechanism design is coherent and the ablations suggest each component helps, but the evaluation protocol has three independent threats: selection bias from tuning on target tasks, absence of variance, and an internal contradiction in R. Any one of these would warrant conditionality; together they make the headline claim unverified. I partially agree with the reader's pseudo-label concern: Eq. (6) indeed includes the push term for different pseudo-classes, and low-confidence targets are not explicitly excluded from the cluster assignments, so early noisy pseudo-labels could in principle degrade alignment. However, the paper's results are the evidence for the claim, and the pseudo-label issue is a mechanism risk that could be settled by an ablation; the empirical validation issues are already visible in the manuscript and directly affect every number in the comparison tables. Hence the load-bearing concern is the evidential one. A focused re-run with fixed hyperparameters, multiple seeds, and reported variance would settle whether the method actually outperforms baselines. If the margins survive, the paper should be accepted with minor revisions; if not, the central claim collapses to comparable performance with extra complexity.","tokens_in":29117,"tokens_out":5789,"duration_ms":56268,"concrete_test":"Rerun all seven benchmarks with a pre-specified configuration (R=0.3, beta=0.8, tau=0.6, lambda2=0.7, B=32) using 5 distinct seeds and report mean plus/minus standard deviation; compare against the second-best baseline per benchmark. If A3MDA's margin over the second-best baseline is within one standard deviation on at least four of seven benchmarks, the claimed consistent superiority is not supported.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim is that A3MDA delivers higher classification accuracy than existing MDA methods on seven benchmarks. The most load-bearing condition is that the reported accuracy gaps reflect genuine, reproducible superiority. That condition is not currently secured. Section IV.B states that hyperparameters beta, tau, lambda2, R, and B were tuned on three tasks: '->Amazon' (Office-31), '->Artistic' (Office-Home), and '->Real' (DomainNet). These tasks appear directly in Tables I and II, so tuning on them is selection on the evaluation metric; the resulting numbers are optimistically biased for those benchmarks, and because the same hyperparameters are then used everywhere, the bias can propagate to other tables. Compounding this, no standard deviations or significance tests are reported; the paper says a fixed seed of 10 was used over 3 runs, which cannot produce meaningful variance. The gains over second-best baselines are small: 0.9, 0.8, 1.0, 1.5, 1.07, 0.6, and 0.5 points across the seven tables. Without variance estimates, these margins are indistinguishable from run-to-run noise. Finally, the paper is internally inconsistent about the selection ratio: Section IV.B sets R=0.4, while Fig. 10(a) reports that the optimal R is 0.3, and the conclusion again states R=0.4. This undermines the reproducibility of the reported configuration. The reader's concern about pseudo-label noise in Eqs. (6)-(9) is a plausible mechanism-level risk, but it is secondary: even if the mechanism is sound, the empirical evidence as presented does not support the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes A3MDA, a hardness-driven framework for multi-source domain adaptation (MDA). Three progressive Adaptive Hardness Measurements (Basic, Smooth, and Comparative AHMs) are derived from the model's own predictions: Basic AHM scores instantaneous sample difficulty, Smooth AHM temporally stabilizes these scores via an exponential moving average stored in a hardness memory, and Comparative AHM renormalizes hardness within batches or within-class clusters. These measurements are then used in three ways: Smooth AHM intensities modulate the proportion of weak versus strong augmentation per sample; Comparative AHM values serve as sample weights in a weighted-clustered variant of MMD that aligns same-class source and target clusters while repelling different classes; and hardness rankings select the hardest target samples to build a pseudo-contrastive matrix that enforces intra-domain clustering. The combined objective is evaluated on seven benchmarks (Office-31, Office-Home, DomainNet, ImageCLEF-DA, PACS, Digits-5, Office-Caltech), with the central claim that A3MDA outperforms existing MDA methods in average classification accuracy.","tokens_in":29374,"tokens_out":14123,"duration_ms":122117,"significance":"The conceptual direction is timely and reasonably well executed: applying sample-specific hardness to jointly control augmentation intensity and cluster-level alignment is a sensible extension of prior hardness- and uncertainty-based MDA work, and the paper goes beyond qualitative hard-sample selection by using hardness values quantitatively in the loss. Strengths that deserve explicit credit include the breadth of the evaluation (seven benchmarks with consistent protocols), the component-wise ablation (Table VIII) showing that each of the three utilization mechanisms contributes, the extensions to other discrepancy metrics (Table X) and backbones (Table IX), the explicit computational-cost comparison (Table VII), and the targeted robustness experiment with corrupted pseudo-labels (Fig. 11(c)). However, the empirical support for the headline claim is not yet convincing: no variance information is provided, the reported margins over the second-best method are small (0.6 to 1.5 points across the seven tables), and hyperparameters were selected using tasks that later appear in the evaluation tables.","major_comments":[{"comment":"The evaluation protocol does not currently secure the central claim that A3MDA 'outperforms other methods.' The paper states 'We maintain a fixed random seed of 10 over 3 runs and report the average results'; with a fixed seed, the three runs are not independent replicates, and no standard deviations or significance tests are reported anywhere in Tables I-VI. The margins over the second-best method are 0.9 (Office-31), 0.8 (Office-Home), 1.0 (DomainNet), 1.5 (ImageCLEF-DA), 1.07 (PACS), 0.6 (Digits-5), and 0.7 (Office-Caltech) percentage points, all of which are within the run-to-run variation typically observed for deep MDA methods. The authors should re-run the experiments with at least three different seeds, report mean plus/minus standard deviation, and verify that the reported rankings are stable across seeds.","section":"Section IV.B (Implementation and Training Details)"},{"comment":"Hyperparameters beta, tau, lambda_2, R, and B were selected using Fig. 10, whose supporting tasks are '->Amazon' (Office-31), '->Artistic' (Office-Home), and '->Real' (DomainNet). These exact tasks appear in the evaluation tables (Table I for Office-31 and Office-Home; Table II for DomainNet). Tuning on the evaluation tasks introduces selection bias that inflates the reported accuracies for those tasks, and because the chosen hyperparameters are then applied to all seven datasets, the bias can propagate to the other tables. The tuning should be moved to held-out validation tasks or to validation splits of the target data, and the final tables should be produced without having used the reported tasks for hyperparameter selection.","section":"Section IV.E (Hyperparameters tuning strategies) and Tables I-II"},{"comment":"The reported configuration is internally inconsistent regarding the selection ratio R. Section IV.B sets R = 0.4, and the concluding paragraph of Section IV.E repeats R = 0.4, but the joint tuning analysis in Section IV.E item (3) explicitly states that for both examined tasks 'the optimal performance was achieved with B = 32 and R = 0.3,' as shown in Fig. 10(a). This contradiction makes the exact configuration used to produce Tables I-VI ambiguous, and it undermines reproducibility. The authors must resolve the discrepancy and, ideally, re-run the final experiments under the configuration they actually claim to be optimal.","section":"Section IV.B, Section IV.E item (3), Fig. 10(a), and conclusion"},{"comment":"The treatment of low-confidence target samples in the cluster-level losses is unspecified. Pseudo-labels are defined in Section III as onehot(argmax(P)) multiplied by the indicator 1(z > tau), so for samples with confidence below the threshold tau the pseudo-label vector is all zeros. Nevertheless, Eq. (6) assigns each target sample to a pseudo-class cluster B_{dhat}^t and includes a repulsion term -||phi_{H,k}^{s_m} - phi_{H,dhat != k}^t||^2 that pushes the source class center away from the target center of every different pseudo-class. Early in training, when target predictions are noisy and mostly below tau = 0.6, this repulsion can separate features of the same real class across domains instead of aligning them. Relatedly, the description of the Pseudo-Label Matrix in Section III.B claims that diagonal entries of PL_B^2 are always 1, yet a thresholded zero pseudo-label would yield a zero diagonal entry under the stated definition PL_B^2 = PL_B * PL_B^T. The paper should specify how below-threshold samples enter Eqs. (6)-(9) (for example, by excluding them from the cluster centers and from the PLM), and it should provide evidence that the repulsion term is not harmful in early training; the corruption experiment in Fig. 11(c) tests only the intra-domain PCM term, not the inter-domain repulsion.","section":"Section III.B, Eqs. (6)-(9)"}],"minor_comments":[{"comment":"The lambda_1 schedule is written as '2/exp(-theta p) - 1', which is not the DANN schedule cited as reference [2]; the standard form is 2/(1+exp(-gamma p)) - 1. As printed, the expression grows without bound. Please correct the formula and define the progress variable p explicitly.","section":"Section IV.B"},{"comment":"The 'L2 + Ours' row contains a corrupted entry ('62.8 53.7 4 74.4 66.1'), and the four displayed domain values are inconsistent with the stated average of 66.1. Please regenerate the table and verify all reported numbers.","section":"Table X"},{"comment":"There are two sections numbered V, namely 'V. DISCUSSION' and 'V. CONCLUSION'; the conclusion should be renumbered as Section VI.","section":"Section headings"},{"comment":"The subscript notation is inconsistent: phi_{H,i}^{s_m} is used in the definition of the cluster mean phi_{H,k}^{s_m}, while the weighted feature on the same line is defined as phi_{S,i}^{s_m}. These subscripts should be unified.","section":"Eq. (6)"},{"comment":"The sentence 'Nguyen et al. [58] proposed a cross-domain kernel classifier and applied the max-margin principle to enhance software vulnerability detection' cites [58], which is a class-aware optimal transport paper; the described work matches reference [57]. Please correct the citation.","section":"Section II.A and reference list"},{"comment":"The Input line of Algorithm 1 lists 'labeled target batch B_t'; the target batch is unlabeled in the MDA setting and should be described as such.","section":"Algorithm 1"},{"comment":"The statement that the chosen hyperparameters 'delivering state-of-the-art performance in each' dataset is stronger than the tables support: on DomainNet, A3MDA is below the best per-task accuracy on '->Clp' (71.4 vs. 74.3 for MCC-DA) and '->Skt' (60.7 vs. 60.8), and on Digits-5 it is below STEM on '->syn' (96.2 vs. 97.5). Additionally, Section V acknowledges that matching AHMs to scenarios required manual experimentation; the Fig. 4 ablation supports the chosen assignments, but a sentence explaining why each assignment is preferred a priori would strengthen the presentation.","section":"Section IV.E (concluding paragraph) and Section V"}],"recommendation":"major_revision","confidential_remarks":"This is a journal-format submission that appears to be mid-polish: there are duplicated section numbers, a corrupted table entry, a garbled training-schedule formula, and the R inconsistency between the text and Fig. 10(a). None of these is fatal by itself, but together they suggest the manuscript would benefit from a careful proofreading pass. The decisive issue for the recommendation is empirical: the current fixed-seed, no-variance protocol, combined with hyperparameter tuning on tasks that appear in the evaluation tables, does not establish the headline claim of outperforming prior methods. If a clean multi-seed evaluation shows the 0.6-1.5 point margins collapsing, the abstract and conclusion would need substantial rewriting. The reference list contains a modest cluster of works from the authors' own group ([75] and [77] share authors with this submission); this is not improper, but the related-work discussion should not depend on those citations for positioning. I see no evidence of mathematical circularity in the hardness feedback loop; the concern is statistical grounding, not circularity."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing: this is a genuine engineering contribution. The three-level hardness scheme—Basic, Smooth, Comparative—is coherent, and the paper uses it consistently across augmentation, inter-domain alignment, and intra-domain alignment. Seven benchmark tables, component ablations, backbone robustness, metric swaps (L2, Wasserstein, CORAL), plus a complexity table and a bound analysis. That is more evidence than most MDA submissions carry. If I only read the method and ablations, I would believe the gains are real.\n\nBut the empirical claim as presented is not fully secured. The paper says hyperparameters were tuned on 'Amazon' (Office-31), 'Artistic' (Office-Home), and 'Real' (DomainNet), and those exact tasks appear in the main tables. That is selection on the evaluation metric; the numbers for those tasks are optimistically biased, and the same hyperparameters propagate to other tables. On top of that, no standard deviations or significance tests anywhere—three runs with a fixed seed of 10 does not produce meaningful variance. The margins over second-best are small: 0.9, 0.8, 1.0, 1.5, 1.07, 0.6, and 0.5 points across the seven tables. Without variance, those are within noise. Also, the paper is internally inconsistent about R: Section IV.B says R=0.4, Fig. 10(a) says optimal R=0.3, and the conclusion says 0.4 again. That needs to be fixed before anyone can reproduce the configuration.\n\nThe reader's concern about pseudo-label noise in Eqs. (6)-(9) is plausible but secondary. The weighted MMD pull/push on all target samples, including low-confidence ones, is a genuine risk early in training. The paper's own corruption experiment in Fig. 11(c) is reassuring but only tests 50% corruption over the first 20 epochs, not steady-state noisy pseudo-labels. Still, the central mechanism is standard self-training territory, not a circularity.\n\nVerdict: conditional. The method is worth a serious referee. I would send it to review, but request code, error bars, clarification of R, and either retuned hyperparameters on a validation split or a robustness study showing insensitivity. If the numbers hold up after that, this is a solid journal contribution to MDA.","headline":"Genuine method contribution with matching ablations, but the reported gains are not yet verified: tuning on test tasks, no variance, and an R inconsistency in the text.","tokens_in":30059,"tokens_out":1334,"would_cite":false,"duration_ms":12666,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A3MDA claims that hardness-adaptive augmentation and alignment, built from three progressive hardness measurements, outperforms existing multi-source domain adaptation methods on seven benchmarks.","keywords":["Multi-source domain adaptation","Hardness-aware learning","Adaptive data augmentation","Weighted maximum mean discrepancy","Pseudo-label refinement","Contrastive learning","Unsupervised domain adaptation","Transfer learning"],"falsifier":"Compare A3MDA's cluster losses (Eqs. (6)-(9)) in two versions: one using all target pseudo-labels and one using only pseudo-labels whose confidence exceeds $\\tau=0.6$. If the filtered version matches or beats the full version on the seven benchmarks, the claim that low-confidence hard samples can be safely included in the alignment losses is falsified.","tokens_in":28776,"feed_emoji":"🎯","tokens_out":10190,"duration_ms":79234,"temperature":0.7,"pith_summary":"This paper claims that multi-source domain adaptation (MDA) underperforms because it treats every sample alike, and that a hardness-driven training loop can fix three weaknesses at once: uncontrolled strong augmentation, coarse sample-level inter-domain alignment, and missing intra-domain alignment. The proposed A3MDA estimates each sample's difficulty with three progressive measurements and uses them to adapt augmentation intensity, to weight a class-aware maximum mean discrepancy (MMD) loss, and to select harder target samples for a pseudo-contrastive loss. On seven public benchmarks the method reports higher average classification accuracy than the compared MDA methods. If the claim holds, a single hardness signal can organize both data augmentation and domain alignment in MDA.","feed_headline":"Hardness-aware training wins on seven domain adaptation benchmarks","feed_subtitle":"Three adaptive hardness measures tune augmentation and alignment to beat existing MDA methods.","key_machinery":"The carrying mechanism is the three-stage Adaptive Hardness Measurement (AHM) hierarchy. Basic AHM gauges instantaneous uncertainty as the $\\ell^2$ norm of the prediction vector with the true or predicted class contribution removed. Smooth AHM applies an exponential moving average with factor $\\beta$ over the training history, stored in a hardness memory, to stabilize epoch-to-epoch fluctuation. Comparative AHM divides smoothed hardness by the batch or class-cluster sum, producing relative weights that avoid any single sample dominating a batch. These three measurements feed three utilization actions: adjusting augmentation intensity, weighting clustered MMD, and selecting hard target samples for the pseudo-contrastive matrix. Each action is guided by the rule that harder samples receive lighter perturbation but stronger alignment pressure.","core_discovery":"The central claim is that replacing uniform, sample-agnostic treatment of source and target data with hardness-adaptive treatment improves MDA classification. A3MDA defines Basic AHM $\\boldsymbol{\\Omega}_b(\\cdot)$ as the $\\ell^2$ norm of the prediction vector with the true (source) or pseudo (target) class element set to zero; Smooth AHM $\\mathbf{S}_b(\\cdot)$ is an exponential moving average of Basic values stored in a hardness memory; Comparative AHM $\\mathbf{H}_b^{[c]}(\\cdot)$ normalizes smoothed hardness within a batch or within a class cluster. These values drive three actions: Eq. (4) mixes weak and strong augmentation according to hardness; Eqs. (6)-(7) turn MMD into a weighted, cluster-level inter-domain loss; Eqs. (8)-(9) build a pseudo-contrastive matrix from harder target samples, aligning the target feature space and correcting pseudo-labels. The paper reports that the full system attains the best average accuracy among compared methods on Office-31, Office-Home, DomainNet, ImageCLEF-DA, PACS, Digits-5, and Office-Caltech.","pith_inferences":["Editorial inference: the hardness rankings could be reused as a curriculum schedule, feeding easier samples to the model before harder ones, a use the paper does not evaluate.","Editorial inference: the pseudo-contrastive matrix behaves like a contrastive objective with pseudo-label-defined positives; combining it with confidence filtering on the selected hard samples may stabilize early training further.","Editorial inference: because the framework is metric-agnostic and augmentation-based, it may extend to semantic segmentation or cross-modal adaptation, where hardness can be computed from prediction entropy at little extra cost.","Editorial inference: the claim implies a testable ordering—hardness-weighted alignment should help most on benchmarks with high intra-domain noise—so accuracy gains could be checked against target-domain prediction entropy across datasets."],"forward_implications":["If A3MDA is right, a single adaptive augmentation scheme can replace per-dataset augmentation search, since hardness smoothing automatically raises intensity as the model generalizes.","The weighted-clustered MMD should give finer inter-domain alignment than sample-level MMD, because class attributes decide which pairs are pulled together and hardness decides how strongly.","Selecting harder samples for the pseudo-contrastive matrix should suppress erroneous pseudo-labels and produce better-separated target clusters, as the pseudo-label trend analysis claims.","The hardness machinery should transfer to other discrepancy metrics such as L2, Wasserstein, and CORAL, with the reported gains on Office-Home.","Across the seven tested benchmarks, the full method reports the highest average accuracy among the compared MDA methods."],"supporting_citations":[{"why":"Supplies MFSAN, the discrepancy-based MDA baseline whose MMD formulation A3MDA upgrades to a weighted-clustered variant.","marker":"[18]"},{"why":"Supplies the traditional MMD loss and the DAN baseline that A3MDA's inter-domain alignment builds on.","marker":"[13]"},{"why":"Supplies the weak/strong augmentation categorization and the warning that strong augmentation can over-augment hard samples.","marker":"[16]"},{"why":"Supplies the M3SDA moment-matching baseline and the DomainNet and Digits-5 benchmark protocols.","marker":"[5]"},{"why":"Supplies the DANN adversarial baseline and the annealing schedule used for the inter-domain loss weight.","marker":"[2]"},{"why":"Supplies the Deep CORAL correlation-alignment baseline and the CORAL distance used in the metric comparison.","marker":"[1]"}],"fun_headline_variants":["Hardness-adaptive training excels on seven domain adaptation benchmarks","Hardness-guided augmentation and alignment boost MDA on seven benchmarks","Adaptive hardness redefines MDA: wins on seven benchmarks","A3MDA: hardness-aware augmentation and alignment for multi-source DA","Smarter MDA: hardness-aware augmentation and alignment wins on seven sets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the guessed labels assigned to every target image, including the low-confidence ones, are accurate enough to specify which image features should be pulled together and which should be pushed apart during alignment.","fun_headline_variants_meta":{"raw":{"variants":["Hardness-adaptive training excels on seven domain adaptation benchmarks","Hardness-guided augmentation and alignment boost MDA on seven benchmarks","Adaptive hardness redefines MDA: wins on seven benchmarks","A3MDA: hardness-aware augmentation and alignment for multi-source DA","Smarter MDA: hardness-aware augmentation and alignment wins on seven sets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000357,"raw_usage":{"total_tokens":2005,"prompt_tokens":1081,"completion_tokens":924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":697,"completion_tokens_details":{"reasoning_tokens":837}},"tokens_in":697,"tokens_out":924,"duration_ms":7727,"temperature":1.0,"reasoning_tokens":837,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:34:39.917707+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare A3MDA's cluster losses (Eqs. (6)-(9)) in two versions: one using all target pseudo-labels and one using only pseudo-labels whose confidence exceeds $\\tau=0.6$. If the filtered version matches or beats the full version on the seven benchmarks, the claim that low-confidence hard samples can be safely included in the alignment losses is falsified.","supporting_citations":[{"cited_title":"Aligning domain -specific distribution and classifier for cross -domain classification from multiple sources,","cited_arxiv_id":null,"evidence_quote":"Supplies MFSAN, the discrepancy-based MDA baseline whose MMD formulation A3MDA upgrades to a weighted-clustered variant."},{"cited_title":"Learning transferable features with deep adaptation networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the traditional MMD loss and the DAN baseline that A3MDA's inter-domain alignment builds on."},{"cited_title":"A simple baseline for semi-supervised semantic segmentation with strong data augmentation,","cited_arxiv_id":null,"evidence_quote":"Supplies the weak/strong augmentation categorization and the warning that strong augmentation can over-augment hard samples."},{"cited_title":"Moment matching for multi- source domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Supplies the M3SDA moment-matching baseline and the DomainNet and Digits-5 benchmark protocols."},{"cited_title":"Unsupervised domain adaptation by backpropagation,","cited_arxiv_id":null,"evidence_quote":"Supplies the DANN adversarial baseline and the annealing schedule used for the inter-domain loss weight."},{"cited_title":"Deep coral: Correlation alignment for deep domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Supplies the Deep CORAL correlation-alignment baseline and the CORAL distance used in the metric comparison."}],"review_version":1}