{"id":"735e075f-4fa1-4de2-bf0b-467107bf61dd","arxiv_id":"2411.12334","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new bag-level alignment loss (BagCSI) for covariate-shifted hybrid LLP, with a generalization error bound and consistent gains on large bags.","lead":"This paper studies a hybrid privacy setting where some training data has individual labels and other data only has group-average labels, and proposes losses that use both to predict better on the target group. The method comes with a generalization bound and shows gains over existing LLP and domain adaptation baselines on real and synthetic data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.2's proof as written only yields ε(B,h) ≥ ν/(16k) for h with ε(DT,h) ≥ ν, which does not imply the stated ε(DT,h) ≤ 16k ε(B,h) for h whose loss is much larger than ν.","rationale":"The reader's weakest assumption, preservation of p(Y|X) across domains, is a real scope condition and is not validated on the real datasets; the Wine and IPUMS domain splits may well violate it. However, that concern is an explicit modeling assumption in the problem definition. The more load-bearing internal issue is that the proof of the headline theoretical result, Theorem 3.2, does not justify the multiplicative bound as written: the final step only lower-bounds ε(B,h) by ν/(16k), not by ε(DT,h)/(16k). Since the paper's central theoretical narrative is that BagCSI loss upper-bounds target instance-level error via this theorem, an unproven statement of this form is a genuine soft spot. I am not recommending rejection: the empirical comparisons are extensive, the code is released, and the synthetic experiments support the practical value of the method. The theorem may be repairable with a relative uniform-convergence or peeling argument, or the claim could be weakened to the threshold form that the proof actually establishes; either way the paper needs a fix before the theoretical guarantee can be taken as stated.","tokens_in":27368,"tokens_out":21671,"duration_ms":230514,"concrete_test":"Independently re-derive the implication from Eq. (11) to Theorem 3.2, replacing the final step ε̂(Z,h) ≥ 3ν/4 with a target-dependent bound ε̂(Z,h) ≥ c·ε(DT,h). Check whether the cited Theorem 17.1 or any argument in Section 5 or Appendix A.3 supplies such a bound for all h with ε(DT,h) ≥ ν under the stated sample complexity m ≥ O(...max{1/(kν²), k²/ν}). If it does not, Theorem 3.2 is not established as stated. A minimal computational probe is to instantiate a two-function class {h1, h2} with εDT(h1)=ν and εDT(h2)=100ν, set m at the stated threshold, and verify whether the proof's events force ε(B,h2) ≥ 100ν/(16k); if the only derivable lower bound is ν/(16k), the theorem statement fails for that class.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"In Section 5 the proof defines F̂err = {h : ε̂(Z,h) ≥ 3ν/4}. After Eq. (11) it obtains ε(B,h) ≥ ε̂(Z,h)/(12k), and then uses ε̂(Z,h) ≥ 3ν/4 to conclude ε(B,h) ≥ ν/(16k). Theorem 3.2, however, requires ε(B,h) ≥ ε(DT,h)/(16k) for every h with ε(DT,h) ≥ ν. The lower bound ν/(16k) is independent of h's actual loss, so for a regressor with ε(DT,h) = 10ν the proof as written permits ε(B,h) = ν/(16k), which violates the claimed inequality. What is missing is a relative uniform-convergence statement such as ε̂(Z,h) ≥ c·ε(DT,h) for all h ∈ Ferr, or a chaining/peeling argument over loss levels ν, 2ν, 4ν, ...; the one-sided threshold bound h ∈ Ferr ⇒ ε̂(Z,h) ≥ 3ν/4 cannot supply the missing factor. This matters because the BagCSI motivation in Section 3.1 uses the multiplicative form to convert small bag loss into a controlled target loss. If only the threshold form is proved, the claimed upper bound is vacuous for losses well above ν.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper defines covariate-shifted hybrid LLP: the learner has instance-labeled source data from a covariate-shifted distribution and bag-labeled target data, and wants an instance-level predictor for the target. It proposes the BagCSI loss (Eq. 3), which combines source instance MSE, target bag MSE, and a label-weighted feature-alignment term ξ(S,B), together with a pseudo-label variant PL-WFA. The theoretical contribution is Lemma 3.1, bounding the bag-loss/source-loss gap, and Theorem 3.2, a bag-to-instance generalization bound for regression with pseudo-dimension-based sample complexity. Experiments on synthetic data, Wine, IPUMS, and Criteo compare BL-WFA and PL-WFA against LLP and domain-adaptation baselines across random, correlated, and mixed-size bags.","tokens_in":27759,"tokens_out":17171,"duration_ms":193495,"significance":"If Theorem 3.2 is established, the paper provides a useful bag-to-instance regression guarantee and a principled motivation for using source instance labels together with target bag labels in domain adaptation. The empirical study is broad and reproducible, with code released, multiple bag-generation protocols, and consistent gains at large bag sizes. However, the written proof of Theorem 3.2 stops short of the stated multiplicative bound, and the claim that BagCSI is an upper bound on target error is an overstatement; these issues need to be repaired before the theoretical motivation is sound. The empirical claims also rely on assumptions that are not validated on the real datasets, so the paper's scope should be stated more carefully.","major_comments":[{"comment":"The proof as written concludes ε(B,h) ≥ ν/(16k) for all h with ε(DT,h) ≥ ν. This is only a threshold lower bound on the bag loss and does not by itself imply the theorem's multiplicative claim ε(DT,h) ≤ 16k ε̄(B,h) for every such h. For example, a regressor with ε(DT,h)=10ν would need ε̄(B,h) ≥ 10ν/(16k), whereas the written chain permits ε̄(B,h)=ν/(16k). The gap is repairable: Theorem 17.1 of Anthony and Bartlett gives the stronger uniform-convergence consequence ε̂(Z,h) ≥ ε(DT,h) − ν/4, which, combined with (11), yields ε̄(B,h) ≥ (ε(DT,h) − ν/4)/(12k) ≥ ε(DT,h)/(16k) for all ε(DT,h) ≥ ν. This step should be added explicitly; as printed, the theorem is not proved.","section":"Section 5"},{"comment":"The sentence \"We have shown above that BagCSI loss is an upper bound over ε(DT,h)\" is not supported by the preceding derivation. Lemma 3.1 bounds a difference of empirical losses, not ε(DT,h); the passage from the lemma to Eq. (3) replaces ξ(S,B) by ξ²(S,B), drops R(h,S,T) and λ′(S,T), and treats ∥r_h∥₂ as an unknown bounded constant. Thus BagCSI is a heuristic surrogate motivated by the bound, not a certified upper bound. Please revise this claim and the surrounding motivation in Section 3.1 accordingly.","section":"Section 6"},{"comment":"The covariate-shift assumption that p(Y|X) is identical on source and target is stated in Section 1 but is not validated on any real dataset. For the Wine split (France versus the rest), the label is price and the features include review-text indicators, so the label mechanism may well differ across domains; in that case source instance labels could actively mislead the target predictor. The synthetic experiments control this assumption, but the real-data conclusions should be qualified, or a diagnostic should be added to check whether p(Y|X) appears stable across the chosen domains.","section":"Section 7"},{"comment":"Theorem 3.2 assumes labels and regressor outputs in [0,1], and its proof uses |z_i| ≤ 1 in Eq. (6) and in the covering arguments. The real-world experiments use unbounded labels (Wine price, Criteo sales amount) and a final linear layer without output clamping, so the theoretical guarantee does not directly apply to those settings. The paper should either normalize labels to the theorem's range or explicitly state that the experiments are outside the theoretical scope.","section":"Section 7"}],"minor_comments":[{"comment":"The references for Quadrianto et al. (2009a) and (2009b) are identical, as are Li and Culotta (2023a) and (2023b); these duplicates should be merged.","section":"References"},{"comment":"Inference 3 says proposed methods are better than using instance-level labeled target data, but Table 3 reports target instance loss 0.19 ± 0.02 while PL-WFA and BL-WFA at bag size 256 are 18.40 and 13.76, respectively. This statement is contradicted by the table; presumably \"source instance loss\" was intended. Please correct it.","section":"Section 8"},{"comment":"The notation B is overloaded: it denotes both an individual bag B ⊆ R^d and the sample of bags B(m,k) used in ε̄(B,h). Please use a separate symbol, e.g., script B for the bag sample.","section":"Section 3"},{"comment":"There are several typos and small formatting issues: \"consistenty\" in the Conclusion, \"± 10.\" in Table 4, and \"discarding features with labels in the top 5 percentile\" in Appendix E.2 should be \"discarding instances\". These should be cleaned up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The proof gap in Theorem 3.2 appears fillable using the uniform-convergence consequence already implied by the cited Theorem 17.1; I recommend requiring the authors to add the missing relative step rather than weakening the theorem to a threshold bound, since the multiplicative form is used in Section 3.1 and Section 6. The empirical evaluation is solid and the release of code is a plus, but the real-data conclusions need to be tempered by the unvalidated p(Y|X) assumption and the boundedness mismatch between theory and experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the setup is genuinely new: hybrid LLP with covariate-shifted source instance labels and target bag labels, and the BagCSI/PL-WFA methods are the first to use target bag labels in the domain-alignment term itself. The experiments are broad and the gains at large bag sizes are consistent across synthetic, Wine, IPUMS, and Criteo. Second, the headline theory as stated is not proven. Theorem 3.2 claims ε(DT,h) ≤ 16k ε(B,h) for all h with ε(DT,h) ≥ ν. The proof only delivers a threshold lower bound: with high probability, every such h has ε(B,h) ≥ ν/(16k). That does not control the ratio. For a regressor with true loss 10ν, the derivation permits ε(B,h) = ν/(16k), which violates the claimed inequality by a factor of 10. A peeling argument over loss levels might fix it, but it is not in the paper, and the current proof cannot be patched by adjusting constants. This matters because Section 3.1 uses the multiplicative form to justify the BagCSI loss.\n\nWhat the paper does well: Lemma 3.1's decomposition is clean; the Appendix C construction correctly shows the k-factor degradation is unavoidable for constant predictors; the experiments include random and correlated bags, mixed bag sizes, and a synthetic dataset where the covariate-shift assumption p(Y|X) equal holds by construction. The code and data are public. The ablation showing the dropped regularizer R is small empirically is honest.\n\nSoft spots beyond the theorem. The BagCSI loss is called an upper bound on ε(DT,h), but Lemma 3.1 bounds a difference with λ'(S,T) and R(h,S,T) present; dropping R is justified empirically but the 'upper bound' wording is stronger than proven. The theory assumes labels in [0,1]; Wine prices and Criteo sales amounts are not in that range. The covariate-shift assumption is standard but not validated on the real datasets; Wine, in particular, has price as a label and country as a domain, so p(Y|X) plausibly changes. The hyperparameter protocol, especially the adaptive κ scaling, is under-specified.\n\nFor whom: people working on LLP, privacy-sensitive aggregated labels, or domain adaptation. The empirical baseline is now a useful reference. The proof flaw is real but local; the paper deserves a serious referee. I would send it out, with the expectation that the authors either fix Theorem 3.2 with a proper peeling argument or restate it as a threshold result and adjust the motivation accordingly.","headline":"New hybrid LLP problem with solid empirics, but Theorem 3.2 as stated is not proven—the proof only yields a threshold lower bound on bag loss, not the claimed multiplicative control.","tokens_in":28230,"tokens_out":4516,"would_cite":true,"duration_ms":46677,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68Q32"],"pacs":[],"model":"deepseek-v4-flash","headline":"When target labels are available only as bag averages and source data is fully labeled under covariate shift, the paper's BagCSI loss bounds target error and improves prediction.","keywords":["learning from label proportions","covariate shift","domain adaptation","bag-level supervision","regression","generalization bound","pseudo-dimension","feature alignment"],"falsifier":"A concrete check: take any real dataset, create a source/target split where $p(Y|X)$ is known to differ (for example, the same product features with prices set by different markets), and compare BagCSI against a target-bags-only baseline; if the alignment term increases target MSE, the paper's reliance on shared $p(Y|X)$ is the failure point. Alternatively, with target instance labels available for evaluation, compute $\\bar{\\varepsilon}(B,h)$ and $16k \\bar{\\varepsilon}(B,h)$ for a trained $h$ and check whether $\\varepsilon(D_T,h)$ violates the bound under the theorem's sampling scheme.","tokens_in":27214,"feed_emoji":"🎯","tokens_out":6229,"duration_ms":62268,"temperature":0.7,"pith_summary":"The paper introduces covariate-shifted hybrid LLP: a setting where training data consists of fully labeled examples from a source distribution plus target-domain examples grouped into bags with only their average label. It argues that target bag labels should not be treated as supervision only for prediction; they also carry signal for domain adaptation. The central claim is that a regression loss with three terms—instance loss on source, bag loss on target, and a label-weighted alignment of source and target feature means—yields better instance-level predictors on the target than either LLP-only or domain-adaptation-only baselines. A supporting theorem shows that, for random bags of size $k$, the target instance error of any regressor with error at least $\\nu$ is bounded by $16k$ times its bag-level error, with high probability. If correct, this means bag-level supervision can directly certify target instance accuracy up to a bag-size factor, and the empirical results on four datasets support the practical gain.","feed_headline":"Bag labels plus shifted source labels beat baselines","feed_subtitle":"A three-term loss proves bag-level error controls target error and wins on four datasets.","key_machinery":"The central object is the BagCSI loss (equation 3), a weighted sum of the target bag-level mean-squared error $\\bar{\\varepsilon}(B,h)$, the source instance-level mean-squared error $\\hat{\\varepsilon}(S,h)$, and the squared label-weighted feature-alignment gap $\\xi^2(S,B)$, where $\\xi(S,B)$ is the norm of the difference between the bag-label-weighted average of target features and the instance-label-weighted average of source features. This last term is what converts bag labels into a domain-adaptation signal. The argument is carried by Lemma 3.1 and Theorem 3.2: the lemma bounds the gap between target bag loss and source instance loss in terms of $\\xi$, and the theorem shows that bag loss controls target instance error up to the factor $16k$, with the sample-size requirement set by the pseudo-dimension of the regressor class. The pseudo-label variant $\\psi(S,B)$ replaces true target instance labels with pseudo-labels that shift each bag's predictions to match its bag average, so the same alignment idea works without target instance labels.","core_discovery":"The discovery is a loss and training procedure that uses target bag labels inside the domain-alignment term, not just in the prediction loss. The BagCSI loss is $\\lambda_1$ times the target bag mean-squared error, plus $\\lambda_2$ times the source instance mean-squared error, plus $\\lambda_3 \\xi^2(S,B)$, where $\\xi$ is the norm of the difference between the average of bag-label-weighted target feature means and the average of source label-feature products. The paper proves Lemma 3.1, that the target bag loss minus source instance loss is bounded by $\\xi$ times the regressor norm plus label-independent terms, and Theorem 3.2, the first bag-to-instance generalization bound for regression in LLP using pseudo-dimension: for random bags of size $k$, with high probability every regressor with target error at least $\\nu$ satisfies $\\varepsilon(D_T,h) \\leq 16k \\bar{\\varepsilon}(B,h)$. The PL-WFA variant replaces bag labels with per-instance pseudo-labels that respect the bag average, giving the same alignment effect when instance-level target predictions are available. The paper's experiments show BL-WFA and PL-WFA matching or beating LLP and domain-adaptation baselines, with larger gains at larger bag sizes.","pith_inferences":["Inference: The BagCSI alignment term is essentially first-moment matching of label-weighted features; a natural extension is to second-order or class-conditional alignment, which the paper does not explore.","Inference: The method's practical value hinges on the shared-$p(Y|X)$ assumption; when the label mechanism differs across domains, the alignment term could pull the predictor toward the source label mechanism, so a diagnostic comparing source and target residual structure would be useful.","Inference: In privacy-driven applications, bag sizes are often large, and the theorem implies the source term becomes increasingly important exactly in that regime, so the practical payoff is largest where bag information is weakest.","Inference: A testable extension is to measure the slack in Theorem 3.2 directly by computing $16k \\bar{\\varepsilon}(B,h)$ for a trained $h$ and comparing it to the target instance error on held-out labels."],"forward_implications":["Training with BagCSI is an upper-bound-motivated surrogate for target instance error, so minimizing the three terms together should give a predictor whose target error is controlled by the quantities in the bound.","For a fixed bag size, the target bag-level loss alone certifies instance-level error up to factor $16k$, so smaller bags give tighter guarantees and larger bags need the source term to compensate.","The PL-WFA variant extends the alignment idea to settings where a trained hypothesis can distribute each bag's average among its instances, making the method applicable in iterative or self-training pipelines.","The baselines adapted from prior work (AF, LR, DANN variants, DMFA) are natural comparisons, and the paper's tables show PL-WFA and BL-WFA improving on them at large bag sizes, including on correlated and mixed-size bags.","The linear degradation of the bound with bag size is not an artifact: the appendix example shows the $k$ blowup is unavoidable for random bags."],"supporting_citations":[{"why":"Supplies the pseudo-dimension, covering-number bounds, and generalization tools that the proof of Theorem 3.2 builds on.","marker":"(Anthony and Bartlett, 2009)"},{"why":"Provides the LLP domain-adaptation methods (AF, LR, AF-DANN, LR-DANN) that the paper adapts to regression and uses as baselines.","marker":"(Li and Culotta, 2023a)"},{"why":"Establishes MMD-based deep adaptation networks, motivating the DMFA baseline and the feature-alignment idea.","marker":"(Long et al., 2015)"},{"why":"Extends domain alignment to joint distributions, providing additional baseline context for the paper's comparison.","marker":"(Long et al., 2017)"},{"why":"Gives prior bag-to-instance generalization results for random bags in classification, which Theorem 3.2 extends to regression.","marker":"(Busa-Fekete et al., 2023)"},{"why":"Contrasts curated versus random bags and informs the paper's choice of random bag generation for the target training set.","marker":"(Chen et al., 2023)"},{"why":"Establishes a PAC framework for label-proportion regression that the paper positions its generalization bound against.","marker":"(Yu et al., 2014)"},{"why":"Provides the IPUMS USA census dataset used in the empirical evaluation.","marker":"(Ruggles et al., 2024)"},{"why":"Provides the Criteo Sponsored Search Conversion Logs dataset used in the empirical evaluation.","marker":"(Tallis and Yadav, 2018)"}],"fun_headline_variants":["Three-term loss for covariate-shifted hybrid LLP","Bag labels plus shifted source data improve target predictions","Bag-level error controls target error: new LLP bound","Hybrid LLP with covariate shift beats baselines","Covariate-shifted hybrid LLP: better predictors from bag labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that only the feature distribution changes between source and target, while the mapping from features to labels $p(Y|X)$ stays the same; if the label mechanism also changes, source instance labels can mislead the target predictor.","fun_headline_variants_meta":{"raw":{"variants":["Three-term loss for covariate-shifted hybrid LLP","Bag labels plus shifted source data improve target predictions","Bag-level error controls target error: new LLP bound","Hybrid LLP with covariate shift beats baselines","Covariate-shifted hybrid LLP: better predictors from bag labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001192,"raw_usage":{"total_tokens":4955,"prompt_tokens":1019,"completion_tokens":3936,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":3858}},"tokens_in":635,"tokens_out":3936,"duration_ms":31404,"temperature":1.0,"reasoning_tokens":3858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:39:39.074313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: take any real dataset, create a source/target split where $p(Y|X)$ is known to differ (for example, the same product features with prices set by different markets), and compare BagCSI against a target-bags-only baseline; if the alignment term increases target MSE, the paper's reliance on shared $p(Y|X)$ is the failure point. Alternatively, with target instance labels available for evaluation, compute $\\bar{\\varepsilon}(B,h)$ and $16k \\bar{\\varepsilon}(B,h)$ for a trained $h$ and check whether $\\varepsilon(D_T,h)$ violates the bound under the theorem's sampling scheme.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes MMD-based deep adaptation networks, motivating the DMFA baseline and the feature-alignment idea."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends domain alignment to joint distributions, providing additional baseline context for the paper's comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contrasts curated versus random bags and informs the paper's choice of random bag generation for the target training set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the IPUMS USA census dataset used in the empirical evaluation."}],"review_version":1}