{"id":"535cf0e7-71f6-4462-8bc3-4f4d8bd055c9","arxiv_id":"2411.15216","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Dist Loss regularizes regression models by matching the sorted distribution of predictions to the label distribution, improving few-shot region accuracy on age and ECG-potassium benchmarks.","lead":"This paper introduces Dist Loss, a training objective that punishes a model when its predictions have a different overall spread than the true labels, on top of the usual per-sample error. The aim is to make regression models accurate on rare, important cases such as abnormal potassium levels, not just on common cases.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control for linear-layer-only fine-tuning: +Dist Loss gains may come from protocol change, not the distribution term.","rationale":"The reader's weakest_assumption is exactly the load-bearing concern: every Dist Loss configuration changes the training protocol to linear-layer-only fine-tuning, and no standard-loss linear-probe control is provided. The paper itself confirms this in Appendix A.2 and Table 3, so the concern is grounded in the manuscript, not invented. The claim that Dist Loss 'complements existing methods' (Table 2) has the same confound, since each '+Dist Loss' row also adopts linear-only fine-tuning. The comparison to Balanced MSE is helpful because both methods share the linear-probe protocol and Dist Loss still wins on few-shot MAE, but the margin is small (0.992, 0.491, 0.088) and no error bars or repeated-seed variance are reported, so statistical significance is unclear. The inverse-probability weighting inside L(·) further means the sample-wise loss is not the vanilla L1/L2 loss; a control using inverse-weighted MSE alone is needed to attribute gains to the distribution-distance term. These issues do not disprove the method, but they make the central claim conditional on an additional control experiment. Since the reader's verdict is already CONDITIONAL with this reasoning, no verdict change is needed; the stress-test pass confirms the same concern and specifies a concrete test that would settle it.","tokens_in":18822,"tokens_out":2489,"duration_ms":30355,"concrete_test":"On IMDB-WIKI-DIR, take the pretrained vanilla ResNet-50 checkpoint, freeze all layers except the final linear layer, and train it for 90 epochs with plain inverse-weighted MSE (no distribution term) using the Appendix A.2.1 schedule (Adam, lr 1e-3, decay at epochs 60/80, batch 512). Compare the few-shot MAE/GM against the reported Dist Loss values of 22.516 and 13.752. If this control reaches within ~0.5 MAE of Dist Loss, the protocol change—not the distribution-distance term—explains most of the gain. Repeat once for the +LDS +Dist Loss setting by fine-tuning a linear layer with LDS-only loss to check Table 2's attribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim attributes the few-shot MAE gains (22.550, 9.122, 1.329) to the Dist Loss objective. But Appendix A.2 states every Dist Loss run uses 'the same strategy as Balanced MSE, fine-tuning the linear layer based on pre-trained model (vanilla model) parameters,' and Table 3 confirms that Balanced MSE and Dist Loss are fastest precisely because they only fine-tune linear layers. Baselines such as LDS, FDS, Ranksim, and ConR are trained end-to-end (Appendix A.2), so the comparison mixes two changes: the proposed distribution-distance term and the linear-probing protocol. There is no control run that fine-tunes a linear layer with only the standard loss (plain or inverse-weighted MSE). Table 2's '+Baseline +Dist Loss' rows similarly switch to linear-only fine-tuning, so their gains over the end-to-end baseline could reflect the well-known strength of linear probing on frozen features rather than the distribution constraint. Additionally, the sequence-difference function used for Dist Loss is inverse-probability-weighted MSE, so the sample-wise term already incorporates reweighting; without ablating this term alone, the contribution of the distribution-distance component is not isolated. This is a load-bearing experimental gap: if a linear-probe control with standard loss reproduces the few-shot improvements, the paper's central attribution is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Dist Loss, a loss term for imbalanced regression that augments a sample-wise regression loss with a differentiable distribution-distance term. The label distribution is estimated with KDE, pseudo-labels are sampled to match that distribution, predictions are sorted with a differentiable sorting operator, and the distance between the pseudo-labels and sorted predictions is minimized. Experiments on IMDB-WIKI-DIR, AgeDB-DIR, and ECG-K-DIR report few-shot MAE improvements over several baselines and show that Dist Loss can be added to existing methods, with SOTA few-shot MAE values of 22.550, 9.122, and 1.329 on the three datasets.","tokens_in":19065,"tokens_out":6854,"duration_ms":60922,"significance":"If the reported gains are attributable to the loss term, Dist Loss is a valuable, simple, and cheap contribution: it requires no architectural change, adds modest computational overhead, and is compatible with existing imbalanced-regression methods. The paper ships code, includes extensive ablations (loss-function choice, batch size, imbalance ratio), and studies a clinically relevant potassium-regression task. The main caveat is that the experimental protocol for Dist Loss differs from most baselines, and the specific contribution of the distribution-distance term is not isolated; these issues are fixable with additional control experiments.","major_comments":[{"comment":"The comparison in Table 1 conflates the proposed loss term with a change in training protocol. Appendix A.2 states that Dist Loss runs 'used the same strategy as Balanced MSE, fine-tuning the linear layer based on pre-trained model (vanilla model) parameters,' while LDS, FDS, Ranksim, and ConR are trained end-to-end. Table 3 confirms that Balanced MSE and Dist Loss are much faster precisely because they only fine-tune linear layers. There is no control run that fine-tunes a linear layer using only the standard (or inverse-weighted) loss without the distribution-distance term. Consequently, the reported gains over the end-to-end baselines in Table 1 could be due to linear probing on frozen features rather than to the Dist Loss objective. This is load-bearing because the paper's central claim attributes the few-shot improvements to Dist Loss. Please add a linear-probe control with the vanilla loss, and report the comparison to each baseline under a matched protocol.","section":"Appendix A.2, Table 1"},{"comment":"The contribution of the distribution-distance component is not isolated from the sample-wise reweighting. The sequence-difference function used for Dist Loss is inverse-probability-weighted MSE (INV-L2), as stated in Appendix A.2, so the sample-level term already upweights rare labels. Table 4 only varies the underlying metric (INV-L1 vs INV-L2); it does not ablate the distribution-distance mechanism (sorted pseudo-labels and pseudo-predictions) while keeping the inverse weighting. A control using plain per-sample inverse-weighted MSE without sorting would identify how much of the gain comes from distribution matching rather than from the reweighting that is already embedded in the loss. Without this ablation, the mechanism behind the few-shot improvements is ambiguous.","section":"Section 3.2.1, Table 4"},{"comment":"The results are reported from single runs without error bars or multiple seeds. Several of the headline margins are very small (e.g., 0.203 MAE over LDS on IMDB-WIKI-DIR, 0.088 MAE over Balanced MSE on ECG-K-DIR, and 0.491 MAE over Balanced MSE on AgeDB-DIR). Given the stochasticity of deep learning training, these differences may lie within run-to-run variation. The 'state-of-the-art' claims in Section 4.3 require mean and standard deviation over at least three seeds, or at least a statement of variance. This is especially important because the protocol is not fully matched for the non-linear-probe baselines.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"In Table 8, the ConR + Dist Loss entry for ECG-K-DIR Few MAE is reported as 11.336; this appears to be a typo for 1.336, since Tables 2 and 12 report 1.336 for the same configuration.","section":"Table 8"},{"comment":"Figure 1c is presented as demonstrating the effectiveness of Dist Loss, but since the loss explicitly minimizes the distribution distance between predictions and labels, the reduced discrepancy shown in the figure is expected by construction. The actual evidence for the method should rest on the accuracy metrics in the tables rather than on this distribution-alignment visualization.","section":"Figure 1c"},{"comment":"There is an inconsistency between the batch sizes reported in Table 3 (64, 64, and 256 for IMDB-WIKI-DIR, AgeDB-DIR, and ECG-K-DIR) and those stated in Appendix A.2 (512 for Dist Loss and Balanced MSE on all datasets, and 512 for all methods on ECG-K-DIR). Please clarify which batch sizes were used for the timing experiments and for training.","section":"Table 3, Appendix A.2"},{"comment":"The numerical example in Appendix A.5 contains an apparent formatting error: the inequality '40 × 1.015 > 42 × 105' should presumably compare products such as 40·(10.1)^5 and 42·(10)^5. Please correct the notation.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":"The core experimental gap (matched-protocol control and isolation of the distribution term) is significant and justifies major revision. The paper is otherwise well-organized, and the method is plausible; with the suggested controls and uncertainty reporting, it could become a strong contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Nie et al. propose Dist Loss, a distribution-distance term for imbalanced regression that sorts predictions with a differentiable approximation and compares them against KDE-derived pseudo-labels from the label distribution. The idea is clean, easy to drop into existing pipelines, and the paper ships code. The comparison that matters most—against Balanced MSE, which uses the same linear-only fine-tuning protocol—shows a consistent few-shot MAE improvement on all three datasets (22.550 vs 23.542, 9.122 vs 9.613, 1.329 vs 1.417). That is a real result and it is not confounded.\n\nThe soft spot is exactly what the stress-test flags. The headline SOTA claim mixes two changes: the loss and the training protocol. Appendix A.2 says Dist Loss and Balanced MSE fine-tune only the linear layer of a pretrained model, while LDS, FDS, Ranksim, and ConR are trained end-to-end. Table 3 makes the same point. There is no control run that fine-tunes a linear layer with the plain loss, so you cannot attribute the gains over the end-to-end baselines to the distribution term rather than to linear probing on frozen features. The paper should have included that control and it is a serious omission. It is not fatal to the central idea, because the Balanced MSE comparison already controls for protocol, but it does mean the broader SOTA claim is unsupported.\n\nA second, smaller gap: the sample-wise term in Dist Loss is inverse-probability-weighted MSE, so the loss already contains reweighting. The paper never ablates that reweighting term alone against the full distribution-distance objective, so the marginal contribution of the sorting/pseudo-label component is not isolated.\n\nAlso no error bars or multiple seeds anywhere, and Figure 1c is a construction, not evidence. The batch-size and loss-function ablations are fine and show the method is not brittle. The GM-metric discussion in Appendix A.5 is honest about where the method loses ground.\n\nBottom line: the paper is worth reading for the loss design and for the controlled Balanced MSE comparison. It deserves a serious referee; the request should be for a linear-probe control and variance reporting. I would not cite it in its current form, but I would be glad to see a revised version.","headline":"Neat loss, one clean comparison, but the SOTA claim mixes protocol and objective.","tokens_in":19562,"tokens_out":2300,"would_cite":false,"duration_ms":22929,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dist Loss adds a differentiable constraint that aligns the distribution of a regressor's predictions with the label distribution, and the paper reports that this term improves accuracy on rare target values in imbalanced regression…","keywords":["imbalanced regression","few-shot regions","distribution distance","loss function","kernel density estimation","differentiable sorting","age estimation","ECG potassium prediction"],"falsifier":"Run the identical linear-layer fine-tuning from the same pretrained checkpoints on IMDB-WIKI-DIR, AgeDB-DIR, and ECG-K-DIR using only the inverse-probability-weighted MSE loss, with no distribution term. If the few-shot MAE lands near 22.55, 9.12, and 1.33 respectively, the paper's attribution of the gains to the distribution term is not supported.","tokens_in":18627,"feed_emoji":"🎯","tokens_out":6481,"duration_ms":60480,"temperature":0.7,"pith_summary":"This paper argues that regression models trained on imbalanced data fail on rare target values because their output distribution collapses toward the densely populated label regions, and that this failure can be repaired at the loss level. The proposed Dist Loss adds a differentiable term that measures the distance between the distribution of the model's predictions and the distribution of the training labels, and minimizes it together with sample-wise prediction error. On three imbalanced benchmarks—two age-estimation datasets and one ECG-to-potassium dataset—the authors report that Dist Loss achieves the best few-shot mean absolute error on all three, with values of 22.550, 9.122, and 1.329, and that adding it to existing methods improves them in 26 of 30 metrics. The practical upshot is that a cheap, easy-to-add loss term can make deep regression models better on the sparse cases that are often the clinically important ones.","feed_headline":"Dist Loss cuts rare-value error across three regression benchmarks","feed_subtitle":"Adding a differentiable distribution constraint helps regression models focus on sparse, clinically important labels.","key_machinery":"The central object is a differentiable surrogate for the distance between two distributions in one dimension. The label side is summarized by pseudo-labels: the kernel-density-estimated label density is discretized into bins, expected frequencies per bin are computed for a batch-sized sample, and the bin values are repeated to form an ordered sequence. The prediction side is summarized by pseudo-predictions, the sorted values of the model's outputs within the batch. Sorting is made differentiable through the projection-based fast differentiable sorting algorithm of Blondel et al. (2020), so the whole pipeline can be trained end-to-end. The distance between the two ordered sequences—computed with an inverse-probability-weighted L2 loss in the reported experiments—is what carries the distribution-aligning signal.","core_discovery":"The central claim is that aligning the model's output distribution with the label distribution during training reduces error in sparse-data regions without sacrificing overall accuracy. Dist Loss does this by building two one-dimensional sequences: pseudo-labels, sampled from the kernel-density-estimated label distribution at a size matching the batch, and pseudo-predictions, obtained by sorting the batch's predictions. The distance between these sorted sequences is treated as a differentiable surrogate for the distribution distance, using fast differentiable sorting to keep the operation trainable. Optimizing that distance together with inverse-probability-weighted MSE is reported to yield state-of-the-art few-shot MAE of 22.550 on IMDB-WIKI-DIR, 9.122 on AgeDB-DIR, and 1.329 on ECG-K-DIR, and to improve baselines in 26 of 30 metrics.","pith_inferences":["Because the pseudo-labels are generated from the training label distribution, Dist Loss is tied to the training marginal; if the test distribution shifts away from it, the alignment term may actively hurt, a regime the paper does not test.","The sorted-prediction construction effectively performs a rank-based matching between predictions and labels, so Dist Loss can be seen as a differentiable one-dimensional Wasserstein-style distance; replacing the KDE sampling with exact quantile matching is a natural further simplification.","The same mechanism should transfer to other long-tailed regression problems, such as forecasting rare weather extremes or estimating drug doses, where the same distribution-collapse phenomenon appears."],"forward_implications":["Adding Dist Loss to a vanilla regressor lowers few-shot MAE by about 4.4 years on IMDB-WIKI-DIR, 3.8 years on AgeDB-DIR, and 0.44 mmol/L on ECG-K-DIR.","Dist Loss can be stacked on existing imbalance methods such as LDS, FDS, RankSim, ConR, and Balanced MSE, improving their few-shot metrics in 26 of 30 reported cases.","The gains extend beyond the sparse tail: supplementary results show Dist Loss also improves median-shot MAE on the two age datasets and median-shot GM on the ECG dataset.","The method adds little training cost, with per-epoch time comparable to Balanced MSE and far below feature-level methods like FDS, RankSim, and ConR.","Batch size has little effect on the result between 256 and 768, indicating the distribution estimate does not require a large batch."],"supporting_citations":[{"why":"Defines the DIR benchmarks, the few/median/many shot regions, and the FDS and LDS baselines that Dist Loss is compared against.","marker":"Yang et al. (2021)"},{"why":"Supplies the fast differentiable sorting algorithm used to make the pseudo-prediction sequence trainable.","marker":"Blondel et al. (2020)"},{"why":"Provides kernel density estimation, the nonparametric method used to model the label distribution for pseudo-label generation.","marker":"Parzen (1962)"},{"why":"Introduces Balanced MSE, the closest baseline and the source of the linear-layer fine-tuning protocol that Dist Loss adopts.","marker":"Ren et al. (2022)"},{"why":"Provides the MIMIC-IV data source used to construct the ECG-K-DIR potassium regression dataset.","marker":"Johnson et al. (2020)"},{"why":"Supplies the Net1D architecture used for the ECG-K-DIR experiments.","marker":"Hong et al. (2020)"}],"fun_headline_variants":["Dist Loss: distribution-aware training boosts few-shot regression","Aligning output distribution with labels reduces rare-region error","Differentiable distribution loss targets sparse regression labels","Dist Loss: distribution alignment cuts error in scarce data zones"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every result attributed to Dist Loss also changes the training protocol to linear-layer-only fine-tuning from a pretrained model, and the paper never runs the same fine-tuning with only the standard loss; if that protocol change alone produces the few-shot gains, the distribution term is not responsible.","fun_headline_variants_meta":{"raw":{"variants":["Dist Loss: distribution-aware training boosts few-shot regression","Aligning output distribution with labels reduces rare-region error","Differentiable distribution loss targets sparse regression labels","Dist Loss: distribution alignment cuts error in scarce data zones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000652,"raw_usage":{"total_tokens":2990,"prompt_tokens":943,"completion_tokens":2047,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":1984}},"tokens_in":559,"tokens_out":2047,"duration_ms":14833,"temperature":1.0,"reasoning_tokens":1984,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:25:11.480365+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical linear-layer fine-tuning from the same pretrained checkpoints on IMDB-WIKI-DIR, AgeDB-DIR, and ECG-K-DIR using only the inverse-probability-weighted MSE loss, with no distribution term. If the few-shot MAE lands near 22.55, 9.12, and 1.33 respectively, the paper's attribution of the gains to the distribution term is not supported.","supporting_citations":[{"cited_title":"Delving into deep imbalanced regression","cited_arxiv_id":null,"evidence_quote":"Defines the DIR benchmarks, the few/median/many shot regions, and the FDS and LDS baselines that Dist Loss is compared against."},{"cited_title":"Fast differentiable sorting and ranking","cited_arxiv_id":null,"evidence_quote":"Supplies the fast differentiable sorting algorithm used to make the pseudo-prediction sequence trainable."},{"cited_title":"Balanced mse for imbalanced visual regression","cited_arxiv_id":null,"evidence_quote":"Introduces Balanced MSE, the closest baseline and the source of the linear-layer fine-tuning protocol that Dist Loss adopts."}],"review_version":1}