{"id":"069db567-caa5-49e3-b3d3-c7226893e2ac","arxiv_id":"2412.16540","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Long-tailed classifiers can be rebalanced after training by dividing class probabilities by the model's own average predicted prior, yielding small accuracy gains over using class frequencies.","lead":"This paper proposes a post-hoc fix for long-tailed image classifiers: instead of subtracting class-frequency priors from the logits, it estimates the prior the model actually learned from its own average predictions, then rebalances using that estimate. The fix is simple and gives small accuracy gains in benchmark tables, but the paper's proof that the fix is optimal does not hold up.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 proves a marginal property of an unnormalized score; after softmax normalization the class-independent factor drops out and the marginal guarantee does not survive.","rationale":"Reading in good faith, the paper identifies a genuinely useful observation—the effective model prior differs from the class-frequency prior—and the proposed post-hoc correction gives consistent gains in Tables 1-3. Those results support the method as an empirical heuristic, and the toy experiment is suggestive. The problem is the central formal claim. Definition (7) defines optimality as matching the test marginal, and Theorem 1 proves exactly that property by direct substitution; as an optimality theorem this is definitional rather than substantive. More seriously, Eq. (8) is not normalized over classes. If the adjusted scores are passed through softmax, the class-independent factor P(x)/P^t(x) cancels and the normalized posterior has a different marginal, so the theorem's conclusion does not apply to the logit-adjusted softmax classifier actually evaluated. The two-point counterexample makes the failure concrete and independent of dataset noise. The reader's weakest-assumption statement focuses on marginal matching being insufficient for accuracy-optimality; my check targets the prior gap between the unnormalized proof and the normalized deployment, so agreement is partial rather than full. Because the paper's headline claim is a proof of optimality and that proof is internally inconsistent, rejection is appropriate. A revised version that presents the method strictly as a heuristic, with honest comparison and without the optimality theorem, could be reconsidered.","tokens_in":15446,"tokens_out":6796,"duration_ms":60514,"concrete_test":"Analytical check: take C=2, P^t(x=1)=P^t(x=2)=0.5, P(x=1)=0.9, P^m(A|x=1)=0.9, P^m(A|x=2)=0.1, and P^t(A)=0.5. Compute the softmax-normalized corrected posterior Q(y|x) ∝ P^m(y|x) P^t(y)/P^m(y) and then its P^t-marginal Q(A)=Σ_x Q(A|x)P^t(x). The calculation gives Q(A)≈0.344, not 0.5, showing that Theorem 1's guarantee is lost once the classifier is normalized. If instead the paper intentionally uses an unnormalized score for decisions, it is not a probability model and Eq. (7) is not an optimality condition for accuracy.","verdict_should_be":"REJECT","load_bearing_attack":"The central theoretical claim is Theorem 1 built on Definition (7): the adjusted distribution P^a(y|x)=P^m(y|x) P^t(y)/P^m(y) P(x)/P^t(x) is called optimal because its P^t-marginal equals P^t(y). As a statement about an unnormalized score this is just a one-line integral, but the deployed classifier uses the softmax-normalized posterior. Under normalization, P(x)/P^t(x) cancels, giving Q(y|x)=P^m(y|x) P^t(y)/P^m(y) / Σ_z P^m(z|x) P^t(z)/P^m(z). The proof in Eqs. (10)-(13) never normalizes over classes, and the actual normalized marginal Q(y)=∫ Q(y|x)P^t(x)dx is not generally P^t(y). Thus the promised property does not transfer to the prediction rule used in experiments. Moreover, the Definition makes 'optimality' precisely the property the theorem then proves, so Theorem 1 does not establish accuracy optimality either. A simple two-point counterexample (P^t(x) uniform, P(x=1)=0.9, P^m(A|x=1)=0.9, P^m(A|x=2)=0.1) yields Q(A)≈0.344 instead of 0.5. The empirical gains are real evidence for the correction as a heuristic, but they do not repair the formal claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a post-hoc logit-adjustment method for long-tailed recognition, called Prior2Posterior (P2P). Rather than using class frequencies to model the training class prior, the authors estimate an 'effective model prior' P^m(y) from the model's own average posterior predictions over the training (or validation) data and then adjust predictions by multiplying by P^t(y)/P^m(y), with an additional P(x)/P^t(x) factor in the plain-cross-entropy case. The paper claims to prove optimality of this adjustment for both plain cross-entropy training (Theorem 1) and logit-adjusted training (Theorem 2). Experiments cover toy data, CIFAR10/100-LT, ImageNet-LT, and iNaturalist-18, showing consistent gains over several baselines and small improvements when the correction is applied post-hoc to existing methods.","tokens_in":15682,"tokens_out":5800,"duration_ms":48995,"significance":"The empirical observation that the model's own marginal predictions can differ from class frequencies, and using that to correct residual bias, is a plausible and practically useful idea. The experimental breadth is a strength: the method is tested on several large-scale benchmarks, test-time shifted distributions, and applied to a variety of existing methods, with gains that are consistently positive. However, the central theoretical claim is not what it appears to be. The paper's 'optimality' is defined as matching the test class marginal, not as accuracy or Bayes-risk optimality, and the marginal-matching property is proven only for an unnormalized density that does not correspond to the softmax-normalized classifier actually deployed. The empirical results, while suggestive, are mostly small and do not by themselves rescue the broken theory.","major_comments":[{"comment":"The optimality claim is definitional. The Definition states that P^a(y|x) is 'optimal' if its test class marginal satisfies P^a(y)=P^t(y); Theorem 1 then proves exactly this marginal property for the unnormalized quantity in Eq. (8). No accuracy or decision-theoretic optimality is established. The phrasing 'optimal adjustment' in the Theorem statement and Section 4.1 therefore overstates what is proven; at most, this is a marginal-matching adjustment.","section":"Section 4, Definition (Eq. 7) and Theorem 1 (Eqs. 8-13)"},{"comment":"The marginal-matching property does not survive softmax normalization, which is the normalization actually used in the deployed classifier. The adjusted unnormalized score is P^a(y|x)=P^m(y|x) P^t(y)/P^m(y) * P(x)/P^t(x), and the deployed rule is Q(y|x)=P^a(y|x)/\\sum_z P^a(z|x). The class-independent factor P(x)/P^t(x) cancels in this ratio, so Q(y|x)=P^m(y|x) P^t(y)/P^m(y) / \\sum_z P^m(z|x) P^t(z)/P^m(z), and the integral \\int Q(y|x) P^t(x) dx is not in general equal to P^t(y). The proof in Eqs. (10)-(13) integrates the unnormalized P^a and never normalizes over classes. For a concrete failure, take two classes A,B, P^t(x) uniform on two points, P(x=1)=0.9, P^m(A|x=1)=0.9, P^m(A|x=2)=0.1, and P^t(A)=P^t(B)=0.5; then the normalized marginal is Q(A)≈0.344, not 0.5. Thus Theorem 1's guarantee does not hold for the prediction rule used in the experiments.","section":"Section 4.1, Eq. (8) and proof Eqs. (10)-(13)"},{"comment":"The 'effective model prior' is estimated from the model's own predicted posterior, and the correction then divides by that same estimated quantity. This makes the correction essentially a self-referential rescaling: the 'prior' is derived from the posterior it is meant to correct, and the theorem's conclusion is enforced by construction on the unnormalized scores. The claim that the estimated P^m(y) captures a true underlying bias requires the additional assumption that the model's conditional probabilities are calibrated estimates of P(y|x), an assumption that is neither established nor plausibly satisfied given the paper's own discussion of overfitting to head classes. The empirical gains should therefore be interpreted as evidence for a heuristic, not as confirmation of the theoretical optimality claim.","section":"Section 4.3, Eq. (27)-(29)"}],"minor_comments":[{"comment":"The notation for the two distributions in the logit-adjusted case is inconsistent: 'P m(y|x)' is used both for the inference-time model and for the training-time adjusted distribution. Please introduce distinct symbols (e.g., P^m and \\tilde P^m) and use them consistently throughout the proof of Theorem 2.","section":"Section 4.2, before Theorem 2"},{"comment":"The reported improvements are all positive, but no error bars or significance tests are given. Given the small margins on ImageNet-LT (e.g., +0.22 to +0.35 for several methods), it would be useful to state whether these differences exceed run-to-run variation.","section":"Table 3"},{"comment":"The text says 'As shown in Table 8' but the table with the full test-time-shift results appears in the supplementary material; please provide a cross-reference that distinguishes the main-text table from the supplementary table.","section":"Section 6.4"},{"comment":"The normalization in Eq. (28) uses the same denominator \\sum_k n_k as Eq. (27), but the average is over validation samples from P^t(x); the correct normalization constant should be the number of validation samples, not the training-set size. This should be stated explicitly.","section":"Eq. (28)"}],"recommendation":"reject","confidential_remarks":"The manuscript's empirical contribution is real but the central theoretical claim is broken: the 'optimality' theorem proves a marginal property of an unnormalized score that does not transfer to the softmax-normalized classifier. This is not a local gap but a fundamental mismatch between the stated result and the deployed method. If the authors were to remove the optimality claim and reframe the contribution as a heuristic, the paper could potentially be viable as an empirical study; as it stands, the main advertised contribution is unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The useful core of this paper is small and empirical: estimate the model's effective prior by averaging its predicted posteriors over the training set, then use that in place of class frequencies for logit adjustment. That's a clean, transferable idea, and the results show consistent, if modest, gains on CIFAR-LT, ImageNet-LT, and iNaturalist18. It also plugs into existing methods (cRT, MisLAS, RIDE, etc.) for a further point or so. The observation that a model's effective prior differs from the empirical label frequency is real and worth knowing.\n\nThe theory, though, does not say what the authors claim. Theorem 1 proves that the unnormalized score in Eq. 8 integrates to P^t(y) under P^t(x). That is a marginal-matching property, not accuracy optimality, and the 'optimality' in Definition 7 is just that property restated. Worse, the deployed classifier uses the softmax-normalized version, and the class-independent factor P(x)/P^t(x) cancels under normalization. For the normalized rule Q(y|x) ∝ P^m(y|x) P^t(y)/P^m(y), the marginal ∫ Q(y|x) P^t(x) dx is not generally P^t(y). A simple two-point example gives Q(A)≈0.344 instead of 0.5. So the promised marginal guarantee does not survive in the actual prediction rule. The method still works as a heuristic, and the empirical tables stand on their own, but the formal claim is overreach.\n\nThe empirical evaluation is typical for the area: no error bars, alpha tuned on the validation set each time, and the SOTA claim is within a narrow set of logit-adjustment baselines. These are weaknesses, but not disqualifying for a post-hoc method paper. The gains are small but consistent across settings; the biggest boosts appear when the method is combined with feature retraining (FT+P2P), so the pure post-hoc advantage is more modest than the headline numbers suggest.\n\nOverall: a useful empirical note with an overstated theoretical wrapper. I would send it to a serious referee, because the method is simple, reproducible, and likely to be used, and a referee can push the authors to either fix the theory or present the approach honestly as a heuristic. Rejecting it outright would discard a genuinely useful recipe. For your own work, worth a skim and possibly a citation with a caveat.","headline":"Useful empirical recipe wrapped in an overstated theory: the marginal-matching theorem does not survive softmax normalization.","tokens_in":16251,"tokens_out":3348,"would_cite":true,"duration_ms":56691,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the right prior for long-tail correction is the one the model itself learned from its average predictions, and that reweighting by that prior provably balances the test class proportions and outperforms…","keywords":["long-tailed recognition","logit adjustment","effective prior","class imbalance","post-hoc correction","Bayesian posterior","distribution alignment"],"falsifier":"On a two-class Gaussian toy problem with imbalance 100, train a linear model, compute the Bayes-optimal decision boundary, and then apply the P2P correction; if the corrected boundary and balanced accuracy differ from the Bayes classifier even though the adjusted class proportions are balanced, then marginal matching is not the same as optimality.","tokens_in":15181,"feed_emoji":"⚖️","tokens_out":9568,"duration_ms":74337,"temperature":0.7,"pith_summary":"The paper sets out to show that the class bias of a long-tailed classifier is better captured by the model's own effective prior — the average of its predicted probabilities over the data — than by the empirical class frequencies used in standard logit adjustment. It proposes Prior2Posterior (P2P), a post-hoc reweighting of the predicted posterior by the ratio of the balanced test prior to this learned prior, and proves that the adjusted posterior has exactly the balanced test marginals for both plain cross-entropy models and models trained with logit-adjusted loss. The authors argue that this makes the correction optimal for the balanced test set, and their experiments show consistent accuracy gains over class-frequency adjustment on CIFAR-LT, ImageNet-LT, and iNaturalist18. The practical claim is that P2P can be dropped onto already-trained models of many existing methods and improve them without retraining.","feed_headline":"Model bias, not class counts, sets the long-tail correction","feed_subtitle":"Post-hoc reweighting by the model's own prior lifts accuracy on long-tailed benchmarks without retraining.","key_machinery":"The load-bearing object is the effective prior $P^m(y)$, defined as the integral of the model's softmax posterior over the data distribution and estimated in practice by averaging $P^m(y|x)$ over training samples (Eq. 27) and, for logit-adjusted models, over a balanced validation set or a frequency-scaled training estimate (Eqs. 28–29). This prior carries the entire correction: each predicted posterior is multiplied by $P^t(y)/P^m(y)$, which down-weights classes the model over-predicts and up-weights classes it under-predicts, so the aggregate predictions are forced to match the balanced test prior. The same estimated prior is what lets the method inspect any trained model for residual bias and remove it post-hoc.","core_discovery":"The central discovery is that the prior implicit in a trained model, $P^m(y)=\\int P^m(y|x)P(x)\\,dx$, differs systematically from the class-frequency prior, so frequency-based corrections remove the wrong amount of bias. The paper derives the adjusted posterior $P^a(y|x)=P^m(y|x)\\frac{P^t(y)}{P^m(y)}\\frac{P(x)}{P^t(x)}$ for plain cross-entropy training (Theorem 1) and $P^a(y|x)=P^m(y|x)\\frac{P^t(y)}{P^m(y)}$ for logit-adjusted training (Theorem 2), where $P^m(y)$ is the effective prior estimated from the model's own outputs. In both cases the adjustment forces the aggregate predicted class proportions on the test distribution to equal the balanced prior $P^t(y)$, which the paper identifies as the defining optimality condition. Empirically this corrected posterior improves accuracy on long-tailed benchmarks and, applied post-hoc, boosts many existing methods beyond their original results.","pith_inferences":["Beyond the paper: the proof establishes marginal matching, $P^a(y)=P^t(y)$, not per-sample Bayes optimality; two posteriors with identical marginals can have different error rates, so the \"optimality\" claim is stronger than what the theorem shows.","Beyond the paper: because the correction targets $P^t(y)$, the same recipe can point at any desired test prior, so it could be extended to deliberately imbalanced or shifted test distributions beyond the ones tested.","Beyond the paper: the consistent gains on pretrained models suggest P2P may partly act as a calibration step; comparing it against temperature scaling or per-class calibration on the same models would separate prior correction from general calibration effects."],"forward_implications":["P2P applied to a plain cross-entropy model beats class-frequency logit adjustment on CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist18.","Applied to logit-adjusted models (classifier retraining or feature tuning), P2P removes residual head-class bias and gives further gains, with the best results from feature tuning plus P2P.","Many existing published methods improve when P2P is applied to their pretrained outputs, with reported gains up to 1.48 points on ImageNet-LT and 1.67 points on iNaturalist18 without retraining.","P2P also tracks test-time prior shifts better than distribution-alignment baselines across forward and backward imbalance ratios.","Averaging the training-set and validation-set estimates of the effective prior yields the best correction, indicating that the estimate improves with more samples."],"supporting_citations":[{"why":"supplies the logit-adjustment formula and the class-frequency baseline that P2P replaces","marker":"[23]"},{"why":"provides the balanced softmax loss and training protocol used for the Stage 2 models","marker":"[29]"},{"why":"establishes the two-stage decoupled training and classifier-retraining baselines that P2P builds on","marker":"[12]"},{"why":"offers the distribution-alignment baseline and the test-time-shift evaluation setting","marker":"[10]"},{"why":"supplies the margin-loss baseline and the CIFAR-LT ResNet32 experimental setup","marker":"[4]"},{"why":"justifies treating network softmax outputs as estimates of Bayesian posterior probabilities, from which the effective prior is derived","marker":"[30]"},{"why":"supports the premise that deep-network posteriors are imperfectly calibrated, motivating a learned prior rather than the frequency prior","marker":"[9]"}],"fun_headline_variants":["The effective prior, not the count, corrects long-tail bias","Post-hoc prior correction beats retraining on long-tail","Model's own prior shows the real bias to fix","Prior2Posterior: accurate prior, better long-tail","Class counts mislead; model prior corrects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument collapses if matching the predicted class marginal to the balanced test prior is not sufficient for accuracy optimality, since the correction never changes the per-sample ordering of classes that actually decides each prediction.","fun_headline_variants_meta":{"raw":{"variants":["The effective prior, not the count, corrects long-tail bias","Post-hoc prior correction beats retraining on long-tail","Model's own prior shows the real bias to fix","Prior2Posterior: accurate prior, better long-tail","Class counts mislead; model prior corrects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000616,"raw_usage":{"total_tokens":2918,"prompt_tokens":1062,"completion_tokens":1856,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":1777}},"tokens_in":678,"tokens_out":1856,"duration_ms":13449,"temperature":1.0,"reasoning_tokens":1777,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:29:10.959947+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a two-class Gaussian toy problem with imbalance 100, train a linear model, compute the Bayes-optimal decision boundary, and then apply the P2P correction; if the corrected boundary and balanced accuracy differ from the Bayes classifier even though the adjusted class proportions are balanced, then marginal matching is not the same as optimality.","supporting_citations":[{"cited_title":"Learning imbalanced datasets with label- distribution-aware margin loss","cited_arxiv_id":null,"evidence_quote":"supplies the margin-loss baseline and the CIFAR-LT ResNet32 experimental setup"},{"cited_title":"Disentangling label dis- tribution for long-tailed visual recognition","cited_arxiv_id":null,"evidence_quote":"offers the distribution-alignment baseline and the test-time-shift evaluation setting"},{"cited_title":"Balanced meta-softmax for long-tailed visual recogni- tion","cited_arxiv_id":null,"evidence_quote":"provides the balanced softmax loss and training protocol used for the Stage 2 models"},{"cited_title":"Neural net- work classifiers estimate bayesian a posteriori probabilities","cited_arxiv_id":null,"evidence_quote":"justifies treating network softmax outputs as estimates of Bayesian posterior probabilities, from which the effective prior is derived"},{"cited_title":"On calibration of modern neural networks","cited_arxiv_id":null,"evidence_quote":"supports the premise that deep-network posteriors are imperfectly calibrated, motivating a learned prior rather than the frequency prior"}],"review_version":1}