{"id":"3765ac8e-aa94-4cd7-bff3-16a7ac3e5d4d","arxiv_id":"2501.14090","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"RF-DLC trains a particle ensemble with a utility-weighted, class-rebalanced Bayesian objective so long-tailed classifiers can optimize task-specific decision costs rather than plain accuracy.","lead":"This paper proposes RF-DLC, a training and decision framework for long-tailed classification that lets users assign different costs to different mistakes through a utility matrix. It combines Bayesian decision theory, class reweighting, and an ensemble of models, and reports lower rates of dangerous tail-to-head misclassifications with comparable or better accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq.16 maximizes E_q log g, not log E_q g, so the test-time decision rule is not the Bayes-optimal rule claimed; exact-rule experiments are needed to support the optimality claim.","rationale":"The reader's weakest assumption (Section 4.2 distributional) is real but is a scope limitation the authors explicitly acknowledge, and the benchmark evaluations use uniform test splits, so it does not threaten the empirical claims. The most load-bearing issue is the test-time decision rule: Eq.16 is presented as the Bayesian decision rule but is a Jensen lower bound, not an approximation to the posterior expected gain. This directly undermines the 'optimal decision-making' contribution, independent of dataset-shift assumptions. It is testable by re-running the decision stage on existing trained particles. Because the empirical results may still hold and the fix may be a one-line change to Eq.16, the appropriate disposition is unchanged from the reader's CONDITIONAL verdict: require the authors to justify or replace Eq.16 and compare against the exact Bayes rule.","tokens_in":30874,"tokens_out":12058,"duration_ms":112701,"concrete_test":"Using the already-trained RF-DLC particles, recompute decisions on CIFAR100-LT and DermaMNIST with the exact Bayes rule d* = argmax_d log(Σ_j exp(Σ_{y'} U_{y',d} log p(y'|x,θ_j))) instead of Eq.16, and report ACC, ECE, and FHR for Tables 1, 5, and 6. If any metric moves by more than one standard deviation, Eq.16 is not equivalent to the claimed optimal rule and the paper must either adopt the exact rule or revise the optimality claim. If all metrics are unchanged, the approximation is benign on these benchmarks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Under the paper's own definitions, the optimal decision at test time should maximize log G(d|x*,D) = log E_{θ~p(θ|D)} g(d|x*,θ). With the particle approximation q(θ) ≈ p(θ|D) of Eq.13, and g(d|x,θ)=∏_{y'} p(y'|x,θ)^{U_{y',d}} from Eq.5, this is log((1/M) Σ_j exp(Σ_{y'} U_{y',d} log p(y'|x,θ_j))). Eq.16 instead maximizes Σ_j Σ_{y'} U_{y',d} log p(y'|x,θ_j), which is E_q log g, not log E_q g. Jensen's inequality gives log E_q g ≥ E_q log g, so Eq.16 maximizes a lower bound on the log expected gain; the inequality is strict whenever particles disagree. Replacing log E_q g by E_q log g is not a posterior-approximation step (it persists even if q is the exact posterior), and the two objectives can have different argmax d. Consequently, the central assertion that the method makes Bayes-optimal decisions is not established by the derivation. This concern is internal to the framework and does not rely on the uniform-test or intra-class-consistency assumptions; it affects the headline claim even when all distributional assumptions hold.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RF-DLC, a Bayesian decision-theoretic framework for long-tailed classification. It defines an integrated gain that combines a task utility matrix with the training-data class distribution, derives a lower bound (Eq. 10) using importance sampling under Assumption 1, approximates the posterior with particles (Eq. 13) plus repulsive regularization (Eq. 15), and makes test decisions via Eq. (16). Experiments on CIFAR10/100-LT, ImageNet-LT, iNaturalist, and DermaMNIST report gains in False Head Rate, accuracy, and calibration, with ablations over utility values, f(ny), particle number, and repulsive force.","tokens_in":31241,"tokens_out":6681,"duration_ms":66038,"significance":"If the theoretical claims are established, the framework is a valuable contribution: it offers a principled way to inject asymmetric error costs into long-tailed training and provides an uncertainty estimate through the particle posterior. The paper is strong empirically in breadth: it includes multiple benchmarks, a new cost-oriented metric, an ensemble comparison in Table 9, a robustness study of utility values in Table 15, and comparisons with cost-sensitive baselines in Table 14. The two main concerns below concern the optimality claim of the test-time rule and the relation between the derived and implemented objectives; both are fixable in revision.","major_comments":[{"comment":"Equation (16) is presented as maximizing log G(d|x*,D), but it computes sum_j sum_{y'} U_{y',d} log p(y'|x*,theta_j), which is E_{q(theta)} log g(d|x*,theta). With the particle approximation of Eq. (13), log G(d|x*,D) is approximately log((1/M) sum_j exp(sum_{y'} U_{y',d} log p(y'|x*,theta_j))). Jensen's inequality gives log E_q g >= E_q log g, with strict inequality whenever the particles disagree, and the two objectives can have different argmax decisions. This is not a posterior-approximation artifact: it persists even if q equals the true posterior supported on the M particles. The paper's central assertion that RF-DLC makes optimal decisions according to Bayesian Decision Theory is therefore not established by the derivation. Please either add experiments using the correct log-sum-exp rule, or explicitly reframe Eq. (16) as an approximate decision rule and measure the decision gap relative to the exact rule.","section":"Section 4.4, Eq. (16)"},{"comment":"The implemented training objective in Eq. (24) is not the derived objective in Eq. (10). Eq. (10) weights the entire per-sample term by 1/f(n_{y_i}) and has no alpha; Eq. (24) displays neither the 1/f(n_y) weighting nor the KL term, and introduces a tuned rescaling 1/alpha on the utility term, with alpha set differently per dataset in Table 10. If the class-count weighting is applied in the actual code around Eq. (24), the manuscript needs to show that explicitly; as written, the empirical method is not the objective whose derivation forms the paper's theoretical contribution. Since the paper claims that the utility matrix and variational strategy are directly derived from Bayesian Decision Theory (Section 1), this discrepancy must be resolved, and the sensitivity of results to alpha should be reported.","section":"Appendix E.2, Eq. (24)"}],"minor_comments":[{"comment":"Because RF-DLC is an ensemble of M particles while most rows are single models, please state this in the main text and reference Table 9 at the first presentation of Table 3; as presented, the comparison is not apples-to-apples.","section":"Section 5.3, Table 3"},{"comment":"The growth-rate column in Table 7 appears to be the ratio of the last-class weight to the first-class weight, but the text calls it growth rate without defining it; please clarify the definition to avoid confusion.","section":"Section 5.6, Table 7"},{"comment":"The abstract and Section 1 use 'quantity' where 'quantify' is intended ('to quantity the mispredictions'); please fix the typo.","section":"Abstract and Section 1"},{"comment":"The limitations section mentions non-uniform testing distributions but not the failure of Assumption 1 (p_train(x|y)=p_test(x|y)); since the importance ratio in Eq. (9) depends on that assumption, please add a sentence noting this limitation and its potential effect on the objective.","section":"Section 4.2, Assumption 1"},{"comment":"The statement that code will be released after acceptance conflicts with the reproducibility emphasis of the empirical section; providing the code with the revision would substantially strengthen the paper.","section":"Appendix E.2"}],"recommendation":"major_revision","confidential_remarks":"The two main issues are substantive but fixable: the test-time rule can be corrected or explicitly reframed as an approximation, and the implementation discrepancy can be documented and reconciled with the derivation. The empirical work is extensive and the ensemble comparison in Table 9 already addresses the most obvious fairness concern, so this does not require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper gives you a workable recipe for injecting asymmetric misprediction costs into long-tailed classifiers, and it ships a new metric (False Head Rate) that practitioners will likely borrow. The experiments are consistent and reasonably thorough: three datasets, ablations on the utility matrix, the reweighting form, particle count, repulsive force, and an ensemble comparison that partially addresses the unfair main-table setup. If you work on decision-aware or cost-sensitive classification, this is worth a read.\n\nWhat is genuinely new is the specific combination: product-form decision gain, class-count reweighting, particle variational inference, and test-time utility weighting. The derivation in Eq. 10 is a legitimate ELBO-style bound, not a curve fit to the metric. The authors also state their distributional assumptions (Assumption 1) and list limitations honestly, including the uniform-test restriction.\n\nThe soft spots are real but mostly addressable. The load-bearing one is internal and was not in the reader's take. The test-time rule in Eq. 16 is not the Bayes-optimal rule claimed. The true posterior expected gain under the particle approximation is log E_q g = log (1/M) sum_j exp(sum_{y'} U_{y',d} log p(y'|x, theta_j)), but Eq. 16 maximizes E_q log g = (1/M) sum_j sum_{y'} U_{y',d} log p(y'|x, theta_j). Jensen's inequality says these differ whenever particles disagree, and the gap persists even if q equals the exact posterior. So the central optimality claim is not established. Fixing this requires either justifying the swap as an approximation or running experiments with the exact rule, which is a concrete revision. Second, the implemented objective (Eq. 24) includes an alpha rescaling and an annealed repulsive force (Eq. 23) that are not in the derived Eq. 10; these extra knobs are practical but should be acknowledged as modifications. Third, the closest recent cost-aware long-tailed baseline (He et al. 2024) is cited but not compared, though Table 14 does include Elkan-style baselines.\n\nWho is this for? Researchers or engineers who want a flexible, empirically solid way to reduce tail-to-head errors without trashing accuracy. The theory section overclaims, but the method and metric are useful. I would send this to peer review and ask for a revision that fixes Eq. 16 or weakens the claim, reports exact-rule results, and adds the missing baseline. With those changes, it becomes a solid paper.","headline":"A practical cost-sensitive long-tailed method with strong empirical results, but the optimality claim rests on swapping log E g for E log g; that gap is fixable.","tokens_in":103,"tokens_out":2468,"would_cite":true,"duration_ms":52433,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single decision-theoretic objective lets long-tailed classifiers be trained for the errors that actually matter, with no loss in accuracy or calibration.","keywords":["long-tailed classification","Bayesian decision theory","cost-sensitive learning","utility matrix","false head rate","variational inference","particle-based ensembles","uncertainty quantification"],"falsifier":"Train RF-DLC on a long-tailed benchmark, then evaluate it on a held-out test set with a known non-uniform class distribution and compare against a version trained with the correct $p_{\\mathrm{test}}(y)/p_{\\mathrm{train}}(y)$ ratio; if the decision gain or accuracy is worse, the uniform-test assumption is the cause. A second check is to compare per-class feature statistics between training and test images for the same label, which would directly test the intra-class consistency assumption.","tokens_in":30661,"feed_emoji":"🎯","tokens_out":10343,"duration_ms":80141,"temperature":0.7,"pith_summary":"The paper tries to show that long-tailed classifiers can be trained to make decisions that are optimal for the user's actual error costs, not just to maximize accuracy. It builds a single objective from Bayesian decision theory—the integrated-gain lower bound—in which a utility matrix assigns penalties to specific mispredictions and an importance weight $1/f(n_y)$ corrects for class imbalance. The paper argues that this objective is a principled alternative to re-weighting and logit-adjustment losses, and that the accompanying test-time rule is the right way to use the learned ensemble. If the claim holds, practitioners can directly suppress high-risk errors such as calling a tail-class patient healthy while preserving or improving standard accuracy and calibration.","feed_headline":"One objective tunes long-tailed models to task-specific error costs","feed_subtitle":"Class reweighting plus utility-weighted Bayesian decisions cuts false-head errors while keeping accuracy and calibration.","key_machinery":"The load-bearing object is the integrated gain $G(d) = \\mathbb{E}_{(x_1,y_1),\\dots,(x_N,y_N)\\sim p_{\\mathrm{test}}(x,y)}\\mathbb{E}_{\\theta\\sim p(\\theta|D)}\\prod_i g(d_i|x_i,\\theta)$, where the per-input decision gain is $g(d|x,\\theta)=\\prod_{y'} p(y'|x,\\theta)^{U_{y',d}}$. This fuses the task-specific utility matrix $U$, the class-imbalance correction $p_{\\mathrm{test}}(y)/p_{\\mathrm{train}}(y)\\propto 1/f(n_y)$, and a posterior over models into one quantity. The paper converts this quantity into the tractable lower bound $L(q,d=Y)$ of Eq. 10 using Jensen's inequality, importance sampling, and a particle approximation $q(\\theta)=\\sum_j w_j\\delta(\\theta-\\theta_j)$, with a repulsive-force term from the KL divergence that keeps particles diverse. The same machinery supplies the test-time decision rule in Eq. 16.","core_discovery":"The central claim is that optimal decision-making on long-tailed data can be written as one objective: maximize a variational lower bound on the integrated gain, the expected utility-weighted predictive likelihood under the test distribution. The training loss is $L(q,d=Y) = \\sum_i \\mathbb{E}_{\\theta\\sim q(\\theta)}[\\frac{1}{f(n_{y_i})}(\\sum_{y'} U_{y',y_i}\\log p(y'|x_i,\\theta) + \\log p(y_i|x_i,\\theta))] - \\mathrm{KL}(q(\\theta)\\|p(\\theta)) + C$, with $q(\\theta)$ represented by an ensemble of particles. At test time, the optimal decision for input $x^*$ is $d^* = \\arg\\max_d \\sum_j \\sum_{y'} U_{y',d} \\log p(y'|x^*,\\theta_j)$. The experiments report that a tail-sensitive utility lowers false-head errors on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT, that a one-hot utility preserves or improves overall and tail accuracy, and that the resulting Bayesian predictive distribution improves calibration.","pith_inferences":["If the test set's class distribution is known and non-uniform, the same derivation suggests replacing $1/f(n_y)$ with $p_{\\mathrm{test}}(y)/p_{\\mathrm{train}}(y)$; the paper notes this as a limitation but does not test it.","The inference-time rule in Eq. 16 improves FHR when applied to other baselines, so part of the reported gain may come from decision-time re-scoring rather than from the new training objective alone.","The objective is a lower bound on the integrated gain, and the gap from the one-sample Monte Carlo approximation and Jensen's inequality is not measured; a tighter or multi-sample estimate could be tested.","A regression analogue could be built by replacing the categorical decision gain with a continuous utility function, a direction the paper lists as future work."],"forward_implications":["A user can encode any asymmetric penalty in the utility matrix, and the same training and inference equations apply without changing the model family.","With a tail-sensitive utility, false-head errors drop on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT across tail regions of 25%, 50%, and 75%.","With a one-hot utility, the framework reduces to a class-weighted Bayesian ensemble, and the reported overall and tail accuracies are at or above the compared long-tailed baselines.","The Bayesian predictive distribution from the particle ensemble gives better AUC and lower ECE than maximum softmax and evidential uncertainty baselines.","The ablation recommends the linear form $f(n_y)=n_y$, since larger growth rates between head and tail weights improve tail and overall accuracy."],"supporting_citations":[{"why":"Supplies the Bayesian decision-theoretic foundations for utility functions and the posterior expected gain.","marker":"Robert et al. 2007"},{"why":"Provides the statistical decision theory background for integrated gain and optimal decisions.","marker":"Berger 2013"},{"why":"Contributes the loss-calibrated variational inference derivation that Eq. 10 extends to long-tailed data.","marker":"Lacoste-Julien et al. 2011"},{"why":"Defines the alternative decision gain the paper modifies for stability and flexibility.","marker":"Cobb et al. 2018"},{"why":"Source of the intra-class consistency assumption that reduces the importance ratio to $p_{\\mathrm{test}}(y)/p_{\\mathrm{train}}(y)$.","marker":"Hong et al. 2021"},{"why":"Supplies the importance-sampling identity used to move the expectation from the test to the training distribution.","marker":"Kloek & Van Dijk 1978"},{"why":"Provides the class-balanced loss and effective-number form of $f(n_y)$ used as a comparison weighting.","marker":"Cui et al. 2019"},{"why":"Introduces particle-based variational inference and the repulsive-force idea behind the variational distribution.","marker":"Liu & Wang 2016"},{"why":"Supports the repulsive deep-ensemble interpretation used to justify the particle approximation.","marker":"D'Angelo & Fortuin 2021"},{"why":"Foundational cost-sensitive learning baseline that the paper compares against to show the need for a unified decision-aware objective.","marker":"Elkan 2001"}],"fun_headline_variants":["Bayesian decisions cut false-head errors in long-tailed tasks","One loss integrates class imbalance and error costs","Long-tailed model tunes itself to task-specific risks","Utility-aware variational framework lowers False Head Rate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole weighting scheme reduces to $1/f(n_y)$ only because the paper assumes the test set is class-uniform and that, within each class, test inputs match training inputs; if either fails, the objective reweights the wrong quantity.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian decisions cut false-head errors in long-tailed tasks","One loss integrates class imbalance and error costs","Long-tailed model tunes itself to task-specific risks","Utility-aware variational framework lowers False Head Rate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1287,"prompt_tokens":980,"completion_tokens":307,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":248}},"tokens_in":596,"tokens_out":307,"duration_ms":3122,"temperature":1.0,"reasoning_tokens":248,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:22:42.238241+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RF-DLC on a long-tailed benchmark, then evaluate it on a held-out test set with a known non-uniform class distribution and compare against a version trained with the correct $p_{\\mathrm{test}}(y)/p_{\\mathrm{train}}(y)$ ratio; if the decision gain or accuracy is worse, the uniform-test assumption is the cause. A second check is to compare per-class feature statistics between training and test images for the same label, which would directly test the intra-class consistency assumption.","supporting_citations":[],"review_version":1}