{"id":"b32025e7-55bd-4d11-972b-e6de8c25fc00","arxiv_id":"2505.19421","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A collaborative active domain adaptation framework that adds confident pseudo-labeled samples alongside uncertainty-based active queries, beating prior ADA methods on Office-Home and DomainNet.","lead":"This paper proposes an active domain adaptation method that labels uncertain target samples while also treating the model's most confident predictions as ground truth. The combined strategy reports large accuracy gains on Office-Home and DomainNet at a 5% annotation budget.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PLCS pseudo-label accuracy is never measured; if high-confidence predictions are systematically wrong, the method corrupts both training labels and GPAS uncertainty estimates, so the central claim is unverified.","rationale":"The paper's central claim is that incorporating confident predictions into the labeled set before active sampling improves adaptation. Both main components, PLCS and GPAS, depend on the correctness of pseudo-labels: PLCS uses them as ground truth, and GPAS groups unlabeled samples by the same pseudo-labels before computing posterior variances. The paper provides ablations showing that adding PLCS helps across several base methods, which is genuine evidence that the mechanism has value in practice. However, the accuracy of the pseudo-labels themselves is never reported, so the core assumption 'confident predictions often match ground truth' is not directly supported. The appendix acknowledges that false-positive pseudo-labels eventually hurt, but the operating point of 15% is not characterized. This is the most load-bearing gap because if pseudo-label accuracy is low, the method's benefit could disappear or reverse, and the comparison against methods using only oracle labels would be unfair due to the additional pseudo-labeled data. The reader's weakest assumption identifies the same concern, and I agree. A conditional verdict is appropriate: the method is plausible and empirically promising, but the pseudo-label accuracy measurement should be provided before the central claim is accepted. No change to the reader's conditional verdict is needed.","tokens_in":12851,"tokens_out":6426,"duration_ms":56773,"concrete_test":"On Office-Home and DomainNet, for each active-learning round, compute the precision of the PLCS-selected pseudo-labels: the fraction of samples in the PLCS-selected set whose argmax prediction equals the oracle ground-truth label. Report precision per round and cumulative at the 15% total rate. If the cumulative precision is below about 90% or not substantially above the model's overall target accuracy, the premise that confident predictions 'often match ground truth' is not established for the chosen operating point, and the central claim should be revised or re-validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is stated in the abstract: confident target predictions 'often match the ground truth,' so PLCS can safely add them to the labeled set with pseudo-labels treated as ground truth. The paper never measures the accuracy of these pseudo-labels. PLCS adds up to 15% of the target data to the labeled set without oracle verification, and GPAS uses the same pseudo-labels to group unlabeled samples into class-wise GPs (Eqs. 3-6). If a substantial fraction of the top-confidence predictions are wrong, PLCS injects incorrect supervision, and those same errors shift samples into the wrong GP, corrupting the posterior variances used for active selection. The appendix (Sec. 8.1, Fig. 4) shows that performance peaks at a 15% certain-sampling rate and then declines, attributed to increasing false-positive pseudo-labels, but the false-positive rate at the operating point is never quantified. Without this number, the reported gains over state-of-the-art could also be confounded by the larger effective labeled-set size (5% oracle plus 15% pseudo-labeled) rather than by the certainty-guided selection mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-part active domain adaptation (ADA) method. GPAS uses class-wise Gaussian processes to estimate posterior variances of unlabeled target samples and queries the top-b uncertain samples. PLCS adds the top kappa% most confident target samples per class to the labeled set with their pseudo-labels, after which training continues with a SENTRY UDA loss and uncertainty-balanced class sampling. On Office-Home and DomainNet under a 5% annotation budget, the method reports average accuracies of 76.2% and 85.1%, outperforming SDM-AG by 3.1 and 2.2 points. Ablations show each component contributes and that PLCS can be plugged into other ADA baselines. The central claim is that using confident predictions as pseudo-labels before active sampling shrinks the search space and improves adaptation.","tokens_in":13056,"tokens_out":6881,"duration_ms":57485,"significance":"If the results hold, the idea is practically valuable: it offers a simple way to exploit low-cost confident pseudo-labels within an active DA loop, and the reported gains are substantial relative to prior ADA methods. The manuscript's positive features include a clear problem formulation with explicit equations, several ablations (Tables 3 and 4) that isolate the roles of GPAS, PLCS, and UCS, and a timing comparison in Table 5. However, the main claim rests on two unverified or inconsistent pieces of evidence: the accuracy of pseudo-labels is never measured, and Table 1 contains an implausible identical row for DBAL and BADGE. Since these issues concern the core empirical claim, the paper needs revision before its contribution can be considered established.","major_comments":[{"comment":"In Table 1, the DBAL and BADGE rows are identical for all 12 Office-Home transfers (e.g., 59.2, 81.0, ..., 70.6). These are distinct algorithms and cannot be expected to produce exactly matching numbers. Please correct the DBAL row or the BADGE row and re-run the comparisons; this table is the main evidence for the claimed state-of-the-art improvement.","section":"Table 1"},{"comment":"The central assumption that confident predictions 'often match the ground truth' is never quantified. PLCS treats the top-kappa% per-class predictions as ground truth in D_lt, and Eq. (3) uses the same pseudo-labels to assign unlabeled samples to class-wise GPs. If a non-negligible fraction of the top-confidence pseudo-labels are wrong, PLCS injects incorrect labels and GPAS estimates variances on incorrectly grouped samples. Appendix 8.1 mentions that false positives increase beyond a threshold but reports no false-positive rate at the operating point kappa=15%. Please add per-round pseudo-label precision, and a control experiment in which the same 15% pseudo-labeled budget is filled by random or low-confidence samples instead of top-confidence ones, to show that the gain comes from certainty rather than from extra data volume.","section":"Sec. 3.1 and Appendix 8.1"},{"comment":"No error bars, standard deviations, or code release are provided. Given the magnitude of the reported improvements and the identical DBAL/BADGE rows, single-run accuracies are insufficient to support the claim of consistent superiority. Please report mean +/- standard deviation over at least three runs for the main tables, or make code available so results can be reproduced.","section":"Sec. 4"}],"minor_comments":[{"comment":"The first sentence of Sec. 8.1 says the active sampling rate is fixed at 15%, but the caption of Fig. 4 says it is fixed at 5%; please correct this typo.","section":"Sec. 8.1"},{"comment":"The text says the comparison includes DBAL and CoreSet, but Table 2 has no rows for either method; please add the missing rows or remove the methods from the comparison list.","section":"Sec. 4"},{"comment":"The Introduction claims that adding confident samples reduced selection time from 601s to 540s, but Table 5 reports only per-round times of 89.0s and 133.3s for the proposed method; please provide the exact protocol for the 601s/540s comparison or remove the claim.","section":"Sec. 1"},{"comment":"The EMA coefficient alpha in Eq. (11) is never specified; please report the value used and, if possible, its sensitivity.","section":"Eq. (11)"},{"comment":"The GP posterior in Eqs. (5)-(6) uses no noise term; please clarify whether the feature values F_l are treated as noiseless observations and whether the kernel inversion is intended to be a deterministic interpolation.","section":"Eqs. (5)-(6)"},{"comment":"In Algorithm 1, line 9, the set X_conf is added to D_lt, but X_conf entries carry pseudo-labels rather than oracle labels; consider renaming D_lt or adding a comment to distinguish pseudo-labeled from oracle-labeled target data.","section":"Algorithm 1, line 9"}],"recommendation":"major_revision","confidential_remarks":"I see no evidence of intentional misconduct, but the identical BADGE and DBAL rows in Table 1 should be checked against the original DBAL publication. The missing pseudo-label accuracy analysis is the main technical hurdle; the requested control experiment would directly address whether the improvement is due to certainty-based selection or simply to the larger effective labeled set."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:2505.19421 is that it has a genuinely new combination: it uses confident pseudo-labeled target samples as a free addition to the labeled set and runs GP-based uncertainty sampling on the rest. The results look strong — 76.2% vs 73.1% average on Office-Home, 85.1% vs 82.9% on DomainNet — and the ablation suggests both components contribute. It deserves a serious referee.\n\nWhat works: the idea is simple and untried in ADA. Prior uncertainty-based query functions ignore confident samples; PLCS is a reasonable way to shrink the active search space while GPAS uses class-wise GPs to estimate posterior variance. The authors include an honest appendix curve (Fig. 4) showing that performance peaks at 15% certain sampling and declines afterward, which they attribute to false-positive pseudo-labels. That is a limitation statement, and it should be counted in their favor.\n\nThe soft spots are real but not fatal. The paper never measures the accuracy of the confident pseudo-labels it adds. The abstract says they 'often match the ground truth,' but there is no number at the operating point. That matters because PLCS adds 15% pseudo-labeled data on top of the 5% oracle budget; without a false-positive rate, the gains could partly come from a larger effective labeled set rather than from the certainty-guided mechanism. The appendix hints at this by blaming 'false-positive pseudo-labels' for the decline past 15%, but it doesn't quantify them. Second, Table 1 shows DBAL and BADGE with identical accuracy for every Office-Home transfer — that is almost certainly a copy-paste error, and it erodes trust in the other tables. There are no error bars, no seeds, and no code. These are fixable in revision.\n\nThe central claim is not contradicted. The method is plausible and the evidence, while under-reported, is consistent with it. I would send this to peer review, with a request for code, variance over seeds, a corrected Table 1, and a measurement of pseudo-label accuracy at the chosen κ. For a reader in active domain adaptation, it is worth a look; it won't change the field, but it is a legitimate step forward.","headline":"New combination of pseudo-label certain sampling with GP-based uncertainty querying shows real gains in active domain adaptation, but the paper needs code, error bars, a corrected table, and a measurement of pseudo-label accuracy before the claims are fully supported.","tokens_in":13547,"tokens_out":2338,"would_cite":true,"duration_ms":19895,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Active domain adaptation improves when the model's most confident target predictions are treated as labeled data before uncertainty-based sampling.","keywords":["active domain adaptation","active learning","Gaussian process","pseudo-labeling","uncertainty sampling","domain adaptation","sample selection","confidence-based sampling"],"falsifier":"Measure the accuracy of the top-$\\kappa\\%$ confident predictions per class during the first few adaptation rounds on Office-Home and DomainNet and compare it with the gains attributed to PLCS in the ablation tables; if pseudo-label accuracy is near random while gains persist, the proposed mechanism is not what the paper describes. A cleaner experiment is to run the full method with oracle ground-truth labels for the same PLCS-selected samples and check whether the reported advantage over the pseudo-label version is large.","tokens_in":12655,"feed_emoji":"🎯","tokens_out":9423,"duration_ms":82878,"temperature":0.7,"pith_summary":"This paper proposes a new recipe for active domain adaptation, the setting where a model trained on labeled source data is adapted to an unlabeled target domain by querying labels for a few target samples. The paper's central claim is that the model's most confident target predictions are often correct and should be treated as labeled data before uncertainty-based sampling begins. It combines a Gaussian Process-based Active Sampling (GPAS) step, which queries the most uncertain target samples, with a Pseudo-Label-based Certain Sampling (PLCS) step, which adds the top-confidence predictions per class to the labeled set without human annotation. The paper reports that this collaborative procedure beats prior ADA methods on Office-Home (76.2% average accuracy vs 73.1% for the strongest baseline) and DomainNet (85.1% vs 82.9%), while also cutting query time. If correct, it means that confidence, not just uncertainty, is a usable signal in active adaptation.","feed_headline":"Confident target predictions accelerate active domain adaptation","feed_subtitle":"A mix of uncertain and confident sampling beats prior methods on Office-Home and DomainNet.","key_machinery":"The central object is a class-wise Gaussian Process built over the feature extractor's outputs. For each predicted class $c$, the labeled features $F^l_{[c]}$ and unlabeled features $F^u_{[c]}$ define a GP with a linear kernel $K(P,Q)_{j,k} = P_j Q_k^T / (\\|P_j\\| \\|Q_k\\|)$, and the posterior covariance $\\Sigma^{u,c}$ yields per-sample variances; the $b$ samples with largest $\\mathrm{diag}(\\Sigma^{u,c})$ are queried. The complementary object is PLCS, which takes the top $\\kappa\\%$ highest-probability predictions per class as pseudo-labels and moves them into the labeled set. The mechanism works as a loop: PLCS removes confident samples from the search space, GPAS selects uncertain samples that reduce the domain gap, and the growing share of confident samples feeds back into both components.","core_discovery":"The discovery the paper advances is that the search space for active domain adaptation shrinks if the model's most confident target predictions are incorporated into the labeled set before any active query is made. In each sampling round, PLCS ranks target samples by the maximum classifier probability within each predicted class and moves the top $\\kappa\\%$ into the labeled target set; GPAS then fits a class-wise Gaussian Process to the extracted features of labeled and unlabeled samples and selects the $b$ unlabeled samples with the highest posterior variance for human annotation. These two phases are interleaved with an Uncertainty-balanced Class Sampling step that weights target samples by their class-level GP variance and with an entropy-consistency objective on augmented target views. The paper shows empirically that each ingredient contributes to the final accuracy, and that adding PLCS to established ADA baselines improves their results as well. The headline evidence is that the full method reaches 76.2% average accuracy on Office-Home and 85.1% on DomainNet under a 5% annotation budget, surpassing previous ADA methods.","pith_inferences":["A direct test of the underlying premise would be to measure pseudo-label accuracy on the top-$\\kappa\\%$ confident samples; if accuracy falls sharply on a harder domain shift, the 15% certain sampling rate would need to be lowered or screened by a confidence threshold.","The confidence-first idea could transfer to other label-efficient settings, such as fine-tuning on a target distribution where a model's confident predictions are pre-committed before selecting uncertain examples.","Class-wise GP variance depends on the pseudo-label grouping; on imbalanced target classes, the per-class uncertainty estimates may need a per-class budget adjustment rather than a global top-$\\kappa\\%$.","Because the benefit of PLCS relies on confidence calibration, test-time calibration methods could change the optimal $\\kappa$; one could test whether a calibrated confidence score shifts the certain-sampling schedule."],"forward_implications":["Under the reported protocol (1% query per round, 5% total budget), the method outperforms entropy, CLUE, SDM-AG, and other baselines on both Office-Home and DomainNet.","Ablations in the paper indicate that replacing GPAS with random sampling, entropy, CLUE, or SDM-AG while keeping PLCS and the consistency objective yields lower accuracy, locating the benefit in the GP variance query.","Adding PLCS to existing ADA methods (random, entropy, CLUE, SDM-AG) improves each of them on Office-Home, implying the confident-sampling idea is a transferable augmentation rather than a quirk of one query function.","The certain sampling rate has an optimum: raising $\\kappa$ beyond 15% eventually hurts accuracy as false pseudo-labels appear, so the benefit is bounded by pseudo-label correctness.","Query time stays competitive (e.g., 133.3s vs 151.4s for entropy on DomainNet R→P), which matters for large-scale deployment."],"supporting_citations":[{"why":"Defines the SDM-AG ADA method whose reported averages (73.1% Office-Home, 82.9% DomainNet) serve as the main baselines to beat.","marker":"[6]"},{"why":"Defines TQS, a transferable-query baseline compared across all domain splits.","marker":"[7]"},{"why":"Defines CLUE, an uncertainty-weighted clustering ADA baseline compared and used in the PLCS integration study.","marker":"[9]"},{"why":"Supplies the Gaussian Process posterior mathematics used to compute class-wise variance in GPAS.","marker":"[10]"},{"why":"Supplies the SENTRY entropy-consistency objective applied to unlabeled target samples.","marker":"[14]"},{"why":"Defines the CoreSet active learning baseline that measures diversity-based selection.","marker":"[16]"},{"why":"Is the DomainNet dataset used for the 85.1% average accuracy result.","marker":"[19]"},{"why":"Is the Office-Home dataset used for the 76.2% average accuracy result.","marker":"[20]"}],"fun_headline_variants":["Confident and uncertain sampling refine domain adaptation","Certainty-aware sampling speeds active domain adaptation","Mix confident pseudo-labels with active queries for ADA","Active domain adaptation with certainty and uncertainty","Fusing confident and uncertain labels boosts adaptation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the model's most confident target predictions are usually correct, because PLCS trains on them as ground truth; if a large share of those pseudo-labels are wrong, the method injects errors into the labeled set and the class-wise GP variance estimates that drive sampling become unreliable.","fun_headline_variants_meta":{"raw":{"variants":["Confident and uncertain sampling refine domain adaptation","Certainty-aware sampling speeds active domain adaptation","Mix confident pseudo-labels with active queries for ADA","Active domain adaptation with certainty and uncertainty","Fusing confident and uncertain labels boosts adaptation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000158,"raw_usage":{"total_tokens":1189,"prompt_tokens":875,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":247}},"tokens_in":491,"tokens_out":314,"duration_ms":3709,"temperature":1.0,"reasoning_tokens":247,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:13:23.021103+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the accuracy of the top-$\\kappa\\%$ confident predictions per class during the first few adaptation rounds on Office-Home and DomainNet and compare it with the gains attributed to PLCS in the ablation tables; if pseudo-label accuracy is near random while gains persist, the proposed mechanism is not what the paper describes. A cleaner experiment is to run the full method with oracle ground-truth labels for the same PLCS-selected samples and check whether the reported advantage over the pseudo-label version is large.","supporting_citations":[{"cited_title":"We conduct extensive abla- tion studies to assess the effectiveness of each component in our method, reporting results in Table 3 on the Office-Home dataset","cited_arxiv_id":null,"evidence_quote":"Defines the SDM-AG ADA method whose reported averages (73.1% Office-Home, 82.9% DomainNet) serve as the main baselines to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines TQS, a transferable-query baseline compared across all domain splits."},{"cited_title":"Adversarial discriminative domain adapta- tion,","cited_arxiv_id":null,"evidence_quote":"Defines CLUE, an uncertainty-weighted clustering ADA baseline compared and used in the PLCS integration study."},{"cited_title":"Cycada: Cycle-consistent adversarial domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian Process posterior mathematics used to compute class-wise variance in GPAS."},{"cited_title":"Transferable query selection for active domain adaptation,","cited_arxiv_id":null,"evidence_quote":"Supplies the SENTRY entropy-consistency objective applied to unlabeled target samples."},{"cited_title":"Active domain adaptation via clustering uncertainty-weighted embeddings,","cited_arxiv_id":null,"evidence_quote":"Defines the CoreSet active learning baseline that measures diversity-based selection."},{"cited_title":"Active learning with gaussian processes for object categorization,","cited_arxiv_id":null,"evidence_quote":"Is the DomainNet dataset used for the 85.1% average accuracy result."},{"cited_title":"Gaussian processes in machine learning,","cited_arxiv_id":null,"evidence_quote":"Is the Office-Home dataset used for the 76.2% average accuracy result."}],"review_version":1}