{"id":"b945c8d0-5377-4b25-bcd8-f73a436598e2","arxiv_id":"2507.10574","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The lineARN adaptive cross entropy loss, -(1-p) log p, is mathematically identical to focal loss with gamma=1 and shows a minor top-5 error improvement on CIFAR-100 over cross entropy.","lead":"This paper proposes a loss function that multiplies cross-entropy by the model's uncertainty and reports small accuracy gains on CIFAR-100. The proposed loss turns out to be a known special case of focal loss, which undermines the central novelty claim.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (8) sets a nonnegative KL divergence equal to a negative quantity by dropping terms that diverge for one-hot labels; without Eq. (8), the claimed derivation of Adp(P,Q) from Jeffreys divergence collapses.","rationale":"The reader's weakest assumption is exactly the weak point. I agree with the rejection, but I would put primary weight on the mathematical invalidity of Eq. (8) rather than on the focal-loss equivalence, because the derivation is what the paper advertises as its contribution. If Eq. (8) fails, the loss has no information-theoretic foundation; it is a heuristic modification. The empirical results (top-5 error 6.2±0.15 vs 6.7±0.1 over five runs on CIFAR-100 with one ResNet-18 variant) are too thin to independently establish 'consistently outperforms,' and the failure to compare with the known focal-loss baseline (γ=1) makes the novelty claim unsupported. Still, those weaknesses are secondary: the invalid derivation is sufficient. The paper should be rejected unless the authors can supply a valid derivation or reframe the contribution as an empirical heuristic without the information-theoretic claims.","tokens_in":6236,"tokens_out":7078,"duration_ms":73990,"concrete_test":"Take any softmax output from a checkpoint (or the analytic example Q=(0.5,0.5,0,...), P=(1,0,...)). Set ε=10^-6 for the false entries of P and compute D(Q,P)=Q_c log Q_c + Σ_{i≠c} Q_i log(Q_i/ε). Compare with Eq. (8)'s Q_c log Q_c. If the full value is large and positive while Q_c log Q_c is negative, Eq. (8) is refuted. The sign argument alone is decisive, so the check can be done by hand: re-derive Eq. (8) from the definition of D(Q,P) and inspect the false-class terms.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's advertised contribution is that Eq. (4), Adp(P,Q) = -[1 - Q(x_c)] log Q(x_c), is 'derived' from Jeffreys divergence. The derivation's only bridge to D(Q,P) is Eq. (8), which claims D(Q,P) ≈ Q_c log Q_c by 'dropping the false classes terms.' This claim is mathematically untenable. For any valid probability distributions, D(Q,P) = Σ_i Q_i log(Q_i/P_i) ≥ 0, with equality only if Q=P. But Q_c log Q_c ≤ 0 for 0 < Q_c ≤ 1, so Eq. (8) has the wrong sign and cannot represent a KL divergence term. Concretely, for one-hot P (with P_i=ε for i≠c), the dropped terms are Σ_{i≠c} Q_i log(Q_i/ε), which tend to +∞ as ε→0 whenever any false-class probability is nonzero. At a softmax output with Q_c=0.5 and Q_j=0.5, the full D(Q,P) is ≈ 0.5 log(0.5/ε)+0.5 log(0.5) ≈ 7.6 for ε=10^-6, not the negative value Q_c log Q_c≈-0.35. Thus Eq. (8) is not an approximation; it is a contradiction. The remainder of the derivation inherits this error. The proposed loss may still be a usable objective, but the paper's central claim that it is derived from information theory rests on an invalid step.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a loss function Adp(P,Q) = -(1 - Q(x_c)) log Q(x_c) as a drop-in replacement for cross entropy in classification. It claims this loss is derived from Jeffreys divergence, using the approximations D(P,Q) ≈ -log Q(x_c) and D(Q,P) ≈ Q(x_c) log Q(x_c) for one-hot labels. The authors evaluate the loss on CIFAR-100 with an 18-layer ResNet over 200 epochs, reporting lower top-5 error compared to plain cross entropy (6.2% vs 6.7%) and only two extra arithmetic operations per sample.","tokens_in":6552,"tokens_out":4922,"duration_ms":50124,"significance":"If the derivation were valid and the empirical result held broadly, this would be a simple, almost cost-free modification to cross entropy that improves accuracy. The manuscript has some strengths: the loss is simple and clearly specified; the experimental section reports five independent trials with standard deviations, which is a reasonable first step; and the computational overhead is negligible. However, the central theoretical claim is not sound, and the empirical evidence is limited to a single dataset and a single baseline. The paper provides no code, and the data availability statement is vague. The proposed loss might still be useful as an empirical trick, but the paper does not establish that in its current form.","major_comments":[{"comment":"The approximation D(Q,P) ≈ Q(x_c) log Q(x_c) is mathematically untenable. For any distributions, D(Q,P) = Σ_i Q_i log(Q_i/P_i) ≥ 0. For one-hot P with softened false-class probabilities ε, the dropped terms Σ_{i≠c} Q_i log(Q_i/ε) diverge to +∞ as ε→0 whenever any Q_i (i≠c) is nonzero, while Q_c log Q_c ≤ 0. The approximation thus has the wrong sign and cannot be obtained by dropping terms. For example, with Q_c=0.5 and one false class with probability 0.5 and ε=10^-6, the true D(Q,P) is approximately 7.6, whereas Q_c log Q_c ≈ -0.35. Since Eq. (4) is constructed from Eqs. (7) and (8), the claimed derivation from Jeffreys divergence collapses.","section":"Linearly Adaptive Cross Entropy Loss, Eq. (8)"},{"comment":"The empirical claim that the proposed loss 'consistently outperforms' cross entropy is not adequately supported. The evaluation uses only CIFAR-100, a single architecture (ResNet-18), and compares only against plain cross entropy. No significance test is reported; although the means differ (6.2 vs 6.7), the reader cannot assess whether the difference is statistically robust given only mean and standard deviation over five trials. Moreover, no comparison with standard modifications such as label smoothing or focal loss is provided, so the specific benefit of the extra term is not isolated. At minimum, a paired test across multiple datasets and a state-of-the-art baseline are needed before making a general claim.","section":"Simulation results, Table 1 and Figs. 1-2"},{"comment":"The paper justifies Eq. (7) by 'using a very small value instead of zero to avoid singularities' but never specifies the value or demonstrates that the results are insensitive to it. This is not a minor issue because Eq. (8) is infinite in the one-hot limit, and the entire derivation depends on a limit that is not defined. The authors should either provide a rigorous limiting argument or abandon the claim of a derivation.","section":"Linearly Adaptive Cross Entropy Loss, text after Eq. (7)"}],"minor_comments":[{"comment":"The manuscript contains awkward phrasing, e.g., 'One of loss functions using in machine learning' and 'the proposed one has been evaluated'. The paper would benefit from thorough copyediting.","section":"Abstract and Introduction"},{"comment":"The caption refers to 'orange (upper) line' and 'blue (lower) line', but colors are not distinguishable in grayscale print; use distinct line styles or symbols.","section":"Fig. 1 caption"},{"comment":"The table labels a column 'top-5 error (%)' while the text discusses 'test accuracy'; clarify whether the numbers are error rates or accuracies. The row 'mean and std.' is also confusingly placed.","section":"Table 1"},{"comment":"The section is misspelled as 'Data availiability' and the statement 'available from the corresponding author on reasonable request' is not a standard data availability statement for a public dataset such as CIFAR-100.","section":"Data availability"},{"comment":"The claim of 'practically the same efficiency' is not supported by any measured runtime; reporting wall-clock time or FLOPs per step would strengthen this claim.","section":"Computation cost"}],"recommendation":"reject","confidential_remarks":"The manuscript's central derivation is invalid, and the empirical evidence is too narrow to rescue it. I would not recommend further review unless the paper is fundamentally reframed as an empirical study with rigorous baselines and multiple datasets. There is also no code release, which hampers reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's two central claims — novelty and information-theoretic derivation — both fail. The proposed loss, Adp(P,Q) = -[1-Q(xc)] log Q(xc), is exactly focal loss with gamma=1 (Lin et al., 2017), which the paper does not cite. The derivation from Jeffreys divergence is invalid: Eq. (8) sets D(Q,P) ≈ Q_c log Q_c, but the right side is nonpositive while D(Q,P) is nonnegative, and the dropped false-class terms are positive and diverge as the one-hot smoothing epsilon goes to zero. So the bridge between Jeffreys divergence and the loss collapses. The stress-test note is correct.\n\nThat said, the paper has some virtues. It is clearly written, the loss is embarrassingly simple to implement, and the authors report a consistent small improvement (6.2% vs 6.7% top-5 error) over five runs, which is more than many short papers do. The computational cost claim (one subtraction, one multiplication) is fair.\n\nThe problems are serious. Beyond the missing focal-loss citation, the empirical section compares only against plain cross entropy on a single dataset (CIFAR-100) with a single architecture (ResNet-18). No focal loss baseline, no significance test, no test-time variation, no code. The claimed 'consistently outperforms' is supported only by a small mean difference that could be noise. The paper would need at least a comparison to focal loss with gamma=1 on multiple datasets and an analysis of variance.\n\nThe citation pattern is also off: it cites many cross-entropy variants but misses the one that already proposed this exact weighting. That is not a minor oversight; it defeats the novelty claim.\n\nWho is this for? Maybe a practitioner looking for a one-line loss change to try. But the current manuscript doesn't provide enough evidence to trust the benefit beyond cross entropy, and the theoretical story is misleading.\n\nRecommendation: desk reject. The paper would need a full rewrite — dropping the derivation, adding focal loss baselines, more datasets, significance tests — to become a minor empirical note. As is, it doesn't deserve referee time.","headline":"The loss is focal loss with gamma=1, the derivation is mathematically invalid, and the experiments are too thin to support the novelty claims.","tokens_in":7078,"tokens_out":2897,"would_cite":false,"duration_ms":35090,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a one-line modification of cross entropy, $-[1-Q(x_c)]\\log Q(x_c)$, improves classification accuracy on CIFAR-100.","keywords":["linearly adaptive cross entropy loss","cross entropy","Jeffreys divergence","Kullback-Leibler divergence","one-hot encoding","CIFAR-100","ResNet","loss function design"],"falsifier":"A direct check is to compute $\\sum_{i\\neq c} q_i \\log(q_i/\\varepsilon)$ with a small positive $\\varepsilon$ replacing the zero probabilities of the one-hot label during CIFAR-100 training and see whether it is negligible compared with $q_c \\log q_c$; alternatively, train the same ResNet using the exact Jeffreys divergence instead of the proposed loss and see whether the accuracy difference persists. If the omitted sum is substantial, or the exact divergence does not reproduce the claimed gains, Eq. (8) is the point where the derivation fails.","tokens_in":5989,"feed_emoji":"🎯","tokens_out":6245,"duration_ms":63187,"temperature":0.7,"pith_summary":"This paper proposes a drop-in replacement for the cross entropy loss in classification: the Linearly Adaptive Cross Entropy Loss, $Adp(P,Q)=-[1-Q(x_c)]\\log Q(x_c)$, where $Q(x_c)$ is the predicted probability of the true class. The paper claims that this loss follows from Jeffreys divergence when labels are one-hot, and that it consistently outperforms standard cross entropy throughout training. The supporting experiment trains an 18-layer ResNet on CIFAR-100 for 200 epochs, five trials per loss, and reports a mean top-5 error of $6.2\\%$ for the new loss versus $6.7\\%$ for cross entropy. If this holds, a one-line change to the loss function would improve classifier accuracy at essentially no extra computational cost.","feed_headline":"Linearly adaptive loss beats cross entropy in CIFAR-100 trials","feed_subtitle":"A one-line change to the loss lowers top-5 error from 6.7% to 6.2% in ResNet experiments.","key_machinery":"Jeffreys divergence, $J(P,Q)=D(P,Q)+D(Q,P)$, is the symmetric sum of the two Kullback-Leibler divergences, and it is the object from which the proposed loss is supposedly obtained. For one-hot labels the paper keeps only the true-class term in each direction, $D(P,Q)\\approx -\\log Q(x_c)$ and $D(Q,P)\\approx Q(x_c)\\log Q(x_c)$, whose sum is $-[1-Q(x_c)]\\log Q(x_c)$. The extra factor $1-Q(x_c)$ is the operative novelty: it makes the loss depend on the predicted probability of the true class and vanishes as that probability approaches one.","core_discovery":"The paper's central claim is that the usual one-hot cross entropy loss, $-\\log Q(x_c)$, is improved by multiplying it by $1-Q(x_c)$, giving $Adp(P,Q)=-[1-Q(x_c)]\\log Q(x_c)$. The author presents this formula as the simplification of Jeffreys divergence $J(P,Q)=D(P,Q)+D(Q,P)$ under one-hot labels, with $D(P,Q)$ reducing to $-\\log Q(x_c)$ and $D(Q,P)$ reducing to $Q(x_c)\\log Q(x_c)$. The empirical claim is that on a ResNet-based model trained on CIFAR-100, this loss reaches a mean top-5 error rate of $6.2\\%$ compared with $6.7\\%$ for cross entropy, averaged over five trials over the final ten epochs.","pith_inferences":["Editorial inference: the derivation's key approximation, dropping all false-class terms from $D(Q,P)$, is not checked numerically; a direct evaluation during training would show whether the claimed information-theoretic basis holds or whether the loss works for a different reason.","Editorial inference: because the loss is only compared with plain cross entropy on CIFAR-100, a natural next test is to compare it with other simple reweightings such as focal loss and label smoothing on several datasets, including CIFAR-10 and ImageNet.","Editorial inference: the factor $1-Q(x_c)$ shrinks the loss contribution of confident correct predictions, so the loss may interact with noisy labels differently than cross entropy; the paper does not test this."],"forward_implications":["The proposed loss can replace cross entropy in any softmax classifier without changing the network architecture.","Training with the new loss costs only one subtraction and one multiplication per sample beyond ordinary cross entropy.","If the reported advantage generalizes, models trained with the new loss on CIFAR-100 should show lower top-5 error than cross entropy-trained models across repeated runs.","The loss is compatible with standard training pipelines such as SGD with momentum, stepwise learning-rate decay, and data augmentation."],"supporting_citations":[{"why":"Supplies the Kullback-Leibler divergence, the asymmetric building block of Jeffreys divergence.","marker":"[3]"},{"why":"Introduces the symmetric Jeffreys divergence that the proposed loss is claimed to simplify.","marker":"[4]"},{"why":"Provides the standard cross entropy formulation used as the baseline and context.","marker":"[5]"},{"why":"Provides the ResNet architecture used in the CIFAR-100 experiments.","marker":"[20]"}],"fun_headline_variants":["Adaptive loss shaves top-5 error to 6.2% on CIFAR-100","One-line loss tweak cuts CIFAR-100 top-5 error","Cross entropy times (1-p) beats plain CE on CIFAR-100","Linearly adaptive CE improves CIFAR-100 accuracy","Simple loss change lowers CIFAR-100 top-5 to 6.2%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, for one-hot labels, the reverse divergence can be approximated by its true-class term alone, with all false-class terms dropped; if that approximation is not valid, the proposed loss is a heuristic rather than a derived information-theoretic measure.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive loss shaves top-5 error to 6.2% on CIFAR-100","One-line loss tweak cuts CIFAR-100 top-5 error","Cross entropy times (1-p) beats plain CE on CIFAR-100","Linearly adaptive CE improves CIFAR-100 accuracy","Simple loss change lowers CIFAR-100 top-5 to 6.2%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1388,"prompt_tokens":848,"completion_tokens":540,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":433}},"tokens_in":464,"tokens_out":540,"duration_ms":5905,"temperature":1.0,"reasoning_tokens":433,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:29:16.728766+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check is to compute $\\sum_{i\\neq c} q_i \\log(q_i/\\varepsilon)$ with a small positive $\\varepsilon$ replacing the zero probabilities of the one-hot label during CIFAR-100 training and see whether it is negligible compared with $q_c \\log q_c$; alternatively, train the same ResNet using the exact Jeffreys divergence instead of the proposed loss and see whether the accuracy difference persists. If the omitted sum is substantial, or the exact divergence does not reproduce the claimed gains, Eq. (8) is the point where the derivation fails.","supporting_citations":[{"cited_title":"Kullback and R","cited_arxiv_id":null,"evidence_quote":"Supplies the Kullback-Leibler divergence, the asymmetric building block of Jeffreys divergence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the symmetric Jeffreys divergence that the proposed loss is claimed to simplify."}],"review_version":1}