{"id":"ba38402f-7063-4e4f-ab43-8d3ae8c62cb4","arxiv_id":"2412.12327","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A group-classification plus multi-expert regression framework with symmetric descending soft labels improves deep imbalanced regression on age and text-similarity benchmarks.","lead":"This paper proposes a deep learning method for imbalanced regression that first sorts target values into groups, classifies each input into a group, and then uses a group-specific expert to predict the exact value. The authors report improved accuracy on three standard benchmarks and argue that grouping plus a soft-label training signal helps regression when the training labels are heavily skewed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 1's upper-bound inequality and the actual loss in Eq. (3) are disconnected, so the Bayesian motivation for the method is not established.","rationale":"The reader's weakest_assumption identifies exactly the gap I find most load-bearing: Lemma 1's derivation does not connect to the actual loss. The conditioning identity itself is fine, but the upper-bound inequality treats densities as probabilities, and the practical loss in Eq. (3) regresses only with the ground-truth group, not with the sum over all groups. For hard group assignments, the claimed bound is infinite, so it cannot motivate the algorithm. This matters because the paper explicitly advertises the Bayesian decomposition as its starting point and as a contribution; if that motivation is formally broken, the paper's novelty rests on the empirical recipe alone. I do not see this as fatal: the method is well-ablated and achieves a real improvement on AgeDB-DIR, and the theory could be repaired by stating the exact conditional identity for the ground-truth group and dropping the invalid upper bound. The IMDB-WIKI-DIR claim is also overstated (Ours MAE 7.22 vs VIR 7.19 in Table 2), and the absence of standard deviations makes the SOTA comparisons fragile, but those are additive concerns rather than the deepest structural flaw. The correct verdict remains CONDITIONAL: the empirical method is promising, but the central theoretical motivation needs correction and the empirical claims need tightening.","tokens_in":17901,"tokens_out":4540,"duration_ms":46615,"concrete_test":"Independently re-derive the justification of Eq. (1) from Lemma 1 for a continuous label y and a Gaussian p(y|x,g) whose variance is small enough that the density exceeds 1 on a set of positive measure, using a two-group partition. If the asserted inequality fails in this regime, the upper-bound derivation collapses. Separately, verify whether minimizing Eq. (3) is equivalent to minimizing the claimed upper bound: for hard groups, the bound contains infinite terms for non-ground-truth groups, so the derivation and algorithm cannot be connected. A minimal fix is to state the conditional identity without summation and derive -log p(y|x) exactly for the ground-truth group; if the method is re-run with this corrected motivation and the AgeDB-DIR results are unchanged, the empirical claims survive but the paper's theoretical novelty must be reframed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claimed contribution is a Bayesian bridge between DIR and classification, but the bridge does not survive contact with the actual loss. Lemma 1 states p_tr(y|x) = sum_g p(g|x) p(y|x,g), which is true as a conditioning identity. The next step is invalid as written: -log p(y|x) <= sum_g [-log p(g|x) - log p(y|x,g)]. The footnote justifies this by treating p(y|x,g) as a probability less than 1, but y is continuous and p(y|x,g) is a density that can exceed 1, so the inequality log(a+b) >= log a + log b can fail and the individual -log terms need not be positive. More importantly, the derived upper bound is a sum over all groups, whereas the implemented regression loss in Eq. (3) is sum over groups g of (y_phi_g - yhat_phi_g)^2 for samples whose ground-truth label lies in g: each sample is regressed only by its own group expert. With a hard group partition, p(y|x,g)=0 for every group that does not contain y, so the claimed bound contains infinite terms and cannot justify the finite algorithm. The correct exact statement would be -log p(y|x) = -log p(g0|x) - log p(y|x,g0) for the ground-truth group g0, which matches the empirical loss but removes the sum-over-groups decomposition and the upper-bound motivation. Thus the theoretical contribution is either formally wrong or redundant as stated, though the empirical recipe (group contrastive learning, multi-expert regression, soft labeling) could still be useful.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Group-DIR, a divide-and-conquer framework for deep imbalanced regression. Labels are partitioned into ordinal groups; an ordinal group-aware contrastive loss is trained on features; a classifier predicts groups with a symmetric descending soft-label cross-entropy; and a multi-expert regressor regresses each sample using the expert of its group (ground-truth group at training, predicted group at test). The method is motivated by a Bayesian decomposition in Lemma 1 and evaluated on AgeDB-DIR, IMDB-WIKI-DIR, and STS-B-DIR against a wide range of baselines.","tokens_in":18245,"tokens_out":8558,"duration_ms":74640,"significance":"If the empirical results are reproducible, the method is a useful addition to the DIR toolbox: it combines several known ingredients (contrastive learning, multi-expert regression, label smoothing) in a simple recipe, releases code, and is evaluated on three standard benchmarks with many baselines. The main weakness is the theoretical motivation: Lemma 1's upper-bound inequality is invalid for continuous densities and hard group partitions, and as written it does not connect to the actual loss in Eq. (3). The exact decomposition -log p(y|x) = -log p(g*|x) - log p(y|x,g*) would support the method, but the paper needs to be revised accordingly. Several empirical SOTA claims are also overstated relative to the tables.","major_comments":[{"comment":"The stated upper bound is not valid as written. The footnote justifies -log p(y|x) <= sum_g [-log p(g|x) - log p(y|x,g)] by log(a+b) >= log a + log b for a,b in (0,1), but in a continuous regression problem p(y|x,g) is a conditional density and may exceed 1; in addition, since the groups form a hard partition, for any y exactly one group has p(y|x,g)>0 and the remaining terms are zero, so the sum on the right contains terms equal to +infinity and the inequality is vacuous. The exact identity -log p(y|x) = -log p(g*|x) - log p(y|x,g*) for the group g* containing y is sufficient to motivate the classification-plus-regression objective and matches Eq. (3); please replace the upper-bound argument with this exact decomposition or a correctly regularized bound.","section":"Motivation, Lemma 1 and Eq. (1)"},{"comment":"The text overstates the empirical results. On IMDB-WIKI-DIR, Table 2 lists Ours with All MAE 7.22 and All GM 3.88, whereas VIR is 7.19 and 3.85; only the Med and Few rows improve. On STS-B-DIR, Table 3 lists Ours with All Pearson 77.4 and Many 74.9, below VIR's 77.6 and 75.2. Since the reported margins are small and experiments are averaged over five seeds without standard deviations, please correct these SOTA claims and report variance or significance.","section":"Result Analysis, Tables 2 and 3"},{"comment":"Table 5 compares group-classification accuracy for soft labels against three imbalance-classification losses, but the paper's claim is that soft labeling improves regression. Please report end-to-end MAE (and ideally GM) for CE, LA, and soft-label variants under the same Group-DIR pipeline; otherwise the link between the better group accuracy and the final regression performance is only indirect.","section":"Ablation Study and analysis on Classification Criterion, Table 5"}],"minor_comments":[{"comment":"There are many typos, including 'Sceince' in the affiliations, 'constrastive' in the methodology heading, 'classificaiton' in the abstract, 'porblem' in the appendix, 'oroder' in the model architecture description, 'TSN-E' in Figure 8, and 'V ANILLA' in Table 1; please proofread the manuscript.","section":"Throughout"},{"comment":"The summation in Eq. (3) is ambiguous: the condition y in [g] should be made explicit by indexing samples (e.g., y_i in [g]), and the notation \\hat{y}^{\\phi_g} should indicate the sample index.","section":"Methodology, Eq. (3)"},{"comment":"The columns MAE-GT and G-Mean-GT are not defined in the caption; state explicitly that GT means routing each sample to its ground-truth expert at test time.","section":"Experiments, Table 4"},{"comment":"The reference list contains duplicate entries for Zha et al. 2023a and 2023b, and Table 5 cites 'NC (Yang 2022)', 'DO (Cong 2024)', and 'ReBAT (Wang 2024)' with incomplete author lists in the corresponding reference entries; please complete these citations.","section":"References and Table 5"},{"comment":"The sentence 'Therefore, we do not need balance the three losses in the final loss in the fine tuning' is unclear; specify how lambda1 and lambda2 are set (e.g., grid search) and whether they are tuned per dataset or group count.","section":"Appendix Implementation Details"},{"comment":"The captions for Figures 2 and 4 should state which color corresponds to the ground-truth distribution and which to the model's estimates; the current captions only say 'GT CE' and 'GT LA'.","section":"Figures 2 and 4"}],"recommendation":"major_revision","confidential_remarks":"I have no competing interests. The code link and reproducibility are strengths; the theoretical section should be revised and the result claims aligned with the tables before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful part is the recipe. The paper combines ordinal group-aware contrastive learning, a multi-expert regressor, and a symmetric descending soft labeling scheme for group classification in deep imbalanced regression. On AgeDB-DIR it gets MAE 6.87, beating the previous best 6.99; on STS-B-DIR it's competitive with VIR; on IMDB-WIKI-DIR it improves median and few-shot but is slightly worse overall than VIR (7.22 vs 7.19), so the paper's SOTA claim there is overstated. The ablations are the strength: group number, classifier criterion, and hyperparameters are swept, comparisons are broad, and the code is public.\n\nThe soft spot is the theory. Lemma 1 starts with a valid conditioning identity, but the next step - the inequality - is wrong as stated. p(y|x,g) is a conditional density, not a probability bounded by 1, and the log terms can be negative. More importantly, the actual loss in Eq. (3) regresses each sample only through its ground-truth group expert; it does not sum over groups. With hard group assignment the sum-over-groups bound contains infinite terms, so the bound can't justify the implemented loss. The correct statement is the exact identity -log p(y|x) = -log p(g0|x) - log p(y|x,g0) for the true group g0, which matches the algorithm but makes the decomposition trivial. So the theory is either wrong or redundant as written. The authors should reframe it as an exact per-group conditioning identity and drop the upper-bound story. That is fixable, and the empirical method doesn't depend on it.\n\nAlso, the appendix says experiments were run with five seeds, but no variance is reported anywhere. Given that the headline gains are around 0.1-0.2 MAE, the paper needs error bars before I'd trust the AgeDB edge. This is a minor but necessary addition.\n\nWho is this for? People working on deep imbalanced regression who want a concrete, well-ablated recipe. The framework is sensible and the soft-label encoding for ordinal group classification is the most distinctive piece. The theoretical contribution, as written, is not sound. That doesn't sink the paper, but it needs a revision before acceptance.\n\nI'd send this to peer review. It deserves a serious referee who will ask for the theory-practice gap to be closed and variance to be reported.","headline":"A well-ablated empirical recipe for deep imbalanced regression whose theoretical motivation doesn't survive contact with the loss; referee it for the recipe, not the lemma.","tokens_in":18765,"tokens_out":2549,"would_cite":true,"duration_ms":22853,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Group-first decomposition cuts deep imbalanced regression error below prior methods.","keywords":["deep imbalanced regression","group classification","soft labeling","ordinal contrastive learning","multiple expert regressors","divide-and-conquer","age estimation","label similarity"],"falsifier":"On AgeDB-DIR, retrain the full model with the same ordinal contrastive loss and multi-experts regressor but replace the symmetric descending soft label with a one-hot cross-entropy loss. The paper's thesis predicts a clear MAE degradation; if the MAE stays within noise of 6.87, the soft-label mechanism is not the driver of the reported gains.","tokens_in":17699,"feed_emoji":"🎯","tokens_out":6865,"duration_ms":58639,"temperature":0.7,"pith_summary":"The paper sets out to establish that deep imbalanced regression—learning from labels that are continuous and heavily skewed—should be solved as a two-stage problem: classify each input into an ordered group of nearby target values, then regress the exact value within that group. It derives this split from a Bayesian decomposition of the training density $p_{tr}(y|x)$ into a sum over groups of a classification term $p(g|x)$ and a per-group regression term $p(y|x,g)$. To make the split work, the paper adds an ordinal group-aware contrastive loss for the feature representation, one expert regressor per group, and a symmetric descending soft-labeling scheme that trains the group classifier to respect similarity among nearby labels. The combined system reports the best mean absolute error on AgeDB-DIR (6.87), and on IMDB-WIKI-DIR and STS-B-DIR it posts the under-represented median- and few-shot improvements the paper highlights. The framework matters because it gives a concrete, reusable recipe for a common problem where rare and mid-range target values are systematically underestimated.","feed_headline":"Group-first regression cuts imbalanced age error to 6.87 MAE","feed_subtitle":"Splitting skewed labels into ordered groups and regressing each group separately beats one-shot regressors.","key_machinery":"The load-bearing object is the Bayesian decomposition in Lemma 1: $p_{tr}(y|x)=\\sum_{g} p_{tr}(g|x)p_{tr}(y|x,g)$, which turns DIR into a sum of a group classification loss and a within-group regression loss. The method operationalizes this with three components: an ordinal group-aware contrastive loss that pulls same-group features together and pushes apart groups in proportion to their label distance; a multi-experts regressor that trains one regression head per group and selects the head by the predicted group at test time; and a symmetric descending soft labeling scheme that replaces the one-hot group label with a vector peaking at the true group and descending symmetrically toward both ends, so the classifier is trained to respect label similarity rather than only discriminative boundaries.","core_discovery":"On its own terms, the paper claims that the DIR objective is a sum of a group-classification objective and a per-group regression objective, and that a model built to reflect that structure outperforms prior DIR methods. The concrete evidence is a set of results across three benchmarks: on AgeDB-DIR the method reaches MAE 6.87, below the previous best VIR at 6.99; on IMDB-WIKI-DIR it improves the median MAE (11.42 versus 11.81) and few-shot MAE (20.25 versus 20.96) relative to VIR; and on STS-B-DIR it improves few-shot Pearson correlation (85.8 versus 84.5). The paper interprets these gains as showing that explicit group classification with similarity-aware soft labels helps regression more than classification-based regularizers used only as auxiliary losses.","pith_inferences":["The paper's formal motivation, Lemma 1, is presented as the reason for the design, but the implemented training loss regresses only with the ground-truth group, not the sum over all groups; a reader can treat the Bayesian story as motivation and the soft-label plus contrastive recipe as the empirical contribution.","On IMDB-WIKI-DIR the overall MAE is 7.22 versus VIR's 7.19, so the reported superiority is carried by the median and few-shot slices; the method's practical value may be strongest precisely where imbalance is hardest.","The symmetric soft label assumes that similarity between label groups is symmetric and falls off linearly; for target spaces where similarity is one-sided, such as medical severity scores, an asymmetric variant would be a natural extension the paper does not explore.","A stress test with overlapping or soft group boundaries would separate the contribution of the hard divide-and-conquer assignment from the soft-label classifier, since the current design conflates them."],"forward_implications":["If the decomposition holds, any imbalanced regression dataset can be handled by choosing a group width, training a classifier over groups, and assigning each group its own regressor, without changing the backbone.","The ordinal contrastive loss encourages the feature space to preserve the ordering of labels, which should transfer to other ordinal regression problems such as depth estimation or medical scoring.","The symmetric descending soft label offers an alternative to logit adjustment for imbalanced ordinal classification: it encodes label similarity instead of only class priors.","On the reported benchmarks the largest gains appear in the median and few-shot slices, suggesting the method's benefit is concentrated in under-represented but not extreme tail regions.","The group count is a tunable hyper-parameter; the ablations show MAE stays competitive across 15 to 40 groups, so practitioners can trade off classification accuracy against regression granularity."],"supporting_citations":[{"why":"Supplies the DIR datasets (IMDB-WIKI-DIR, AgeDB-DIR, STS-B-DIR), the imbalanced training settings, and the LDS/FDS baselines that every comparison must beat.","marker":"(Yang et al. 2021)"},{"why":"Provides the balanced-MSE family (BMC, GAI) and the b-MAE evaluation used on IMDB-WIKI-DIR.","marker":"(Ren et al. 2022)"},{"why":"Supplies the Rank-N-Contrast representation learning idea and the training settings the ordinal group-aware contrastive loss builds on.","marker":"(Zha et al. 2023a)"},{"why":"Defines logit adjustment, the strongest imbalance-classification baseline the soft labeling is compared against.","marker":"(Menon et al. 2021)"},{"why":"Contributes RankSim, a direct feature-ranking baseline for DIR that the method must outperform.","marker":"(Gong, Mori, and Tung 2022)"},{"why":"Contributes VIR, the previous best baseline on AgeDB-DIR with MAE 6.99, which the paper's headline numbers are measured against.","marker":"(Wang and Wang 2023)"},{"why":"Motivates soft labels for ordinal regression, which the symmetric descending scheme extends.","marker":"(Díaz and Marathe 2019)"},{"why":"Cited for the empirical finding that classification can help regression, which the paper's divide-and-conquer design relies on.","marker":"(Pintea et al. 2023)"}],"fun_headline_variants":["Grouping labels by ordinal proximity lifts imbalanced regression","Descending soft labels turn regression into easier group tasks","Divide-and-conquer imbalanced regression: classify then regress","Ordinal groups and soft labels cut age MAE to 6.87"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is Lemma 1: the training density $p_{tr}(y|x)$ equals the sum over groups of $p_{tr}(g|x)p_{tr}(y|x,g)$, and the inequality $\\log(a+b)\\ge\\log a+\\log b$ applies to those terms; with continuous labels the per-group term is a density, so neither the equality's interpretability nor the inequality is guaranteed as stated.","fun_headline_variants_meta":{"raw":{"variants":["Grouping labels by ordinal proximity lifts imbalanced regression","Descending soft labels turn regression into easier group tasks","Divide-and-conquer imbalanced regression: classify then regress","Ordinal groups and soft labels cut age MAE to 6.87"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1314,"prompt_tokens":952,"completion_tokens":362,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":292}},"tokens_in":568,"tokens_out":362,"duration_ms":4034,"temperature":1.0,"reasoning_tokens":292,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:12:16.582306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On AgeDB-DIR, retrain the full model with the same ordinal contrastive loss and multi-experts regressor but replace the symmetric descending soft label with a one-hot cross-entropy loss. The paper's thesis predicts a clear MAE degradation; if the MAE stays within noise of 6.87, the soft-label mechanism is not the driver of the reported gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DIR datasets (IMDB-WIKI-DIR, AgeDB-DIR, STS-B-DIR), the imbalanced training settings, and the LDS/FDS baselines that every comparison must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes RankSim, a direct feature-ranking baseline for DIR that the method must outperform."},{"cited_title":"L.; Lin, Y.; Dijkstra, J.; and van Gemert, J","cited_arxiv_id":null,"evidence_quote":"Cited for the empirical finding that classification can help regression, which the paper's divide-and-conquer design relies on."}],"review_version":1}