{"id":"32c4a118-eb17-4347-95a3-1fda6dcdcfb3","arxiv_id":"2412.07219","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Truncating the Taylor expansion of the outlier exposure loss suppresses in-distribution samples in noisy auxiliary OOD data and improves OOD detection.","lead":"The paper proposes Taylor Outlier Exposure, a change to the Outlier Exposure training loss that lets OOD detection work when the auxiliary OOD dataset is contaminated with in-distribution images. It weakens the loss for highly confident samples by truncating a Taylor expansion of the -log probability term, and reports gains over OE and WOODS.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 4/5 print the Taylor coefficients as 1/n!, but the true expansion of -log p uses 1/n. This invalidates the claimed mechanism (Ltoe converging to Loe as t→∞, and t controlling regularization strength), so verify which loss the code actually implements.","rationale":"The reader identified the empirical separation of ID vs OOD Loe values as the weakest assumption. That is a genuine limitation, but the more immediate load-bearing problem is the mathematical correctness of the loss itself, because the paper's stated mechanism—truncating the Taylor expansion of -log p—is not what Eq. 5 describes. If the implemented code uses the printed 1/n! coefficients, then Ltoe is not a Taylor approximation, does not converge to Loe, and t does not meaningfully control suppression; the reported hyperparameter sweeps in Figs. 8-9 would then be hard to explain and the central claim as articulated would fail. If the code uses correct 1/n coefficients, the printed equations are wrong and the experimental results need to be re-evaluated under the correct loss. In either case, the current version needs correction before the empirical comparisons can be taken as support for TaylorOE. I still see the empirical comparisons as potentially salvageable, which is why the appropriate disposition is CONDITIONAL rather than outright rejection: one concrete verification of the implemented coefficients and a re-derivation of the loss would settle the issue. I partially agree with the reader in that the separation assumption is also untested beyond one WRN/CIFAR-10 configuration, but I do not think it is the single most load-bearing concern; the coefficient error sits upstream of the loss definition and the entire t-sensitivity story.","tokens_in":12797,"tokens_out":6113,"duration_ms":72486,"concrete_test":"Inspect the released code at https://github.com/fukuchan41/TaylorOE and check the coefficient in Ltoe: is it 1/n! as printed, or 1/n as in the true expansion? Then recompute Table 1 (CIFAR-10, π=0.1) with t=30 and t=300 using the printed Eq. 5. If the coefficient is 1/n!, the two settings differ by less than 1e-30 and the reported t-sensitivity in Fig. 8/9 cannot be reproduced; if the coefficient is 1/n, Eq. 4/5 must be corrected and all tables re-verified, since the actual loss differs from the one presented.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central derivation is internally inconsistent. Eq. 4 states -log p = Σ_{n=1}^∞ (1-p)^n / n!, but the correct Taylor expansion around p=1 is -log p = Σ_{n=1}^∞ (1-p)^n / n. This is not a cosmetic typo: with 1/n! the infinite sum equals e^{1-p}-1, which is bounded by e-1 ≈ 1.718 at p=0 instead of diverging as -log p does. Consequently the truncated sum in Eq. 5 does not converge to -log p as t→∞, contradicting the text that TaylorOE 'gradually converges to Loe' (Sec. 4.2). It also makes the claimed control of regularization strength by t nearly vacuous: for t≥10, factorial terms beyond t are negligible, so choosing t from {30,...,300} (Figs. 8-9, Sec. 5.1) has essentially no effect on Ltoe. The paper nevertheless reports t-dependent optima in exactly that range, which suggests either the printed coefficients do not match the implemented loss, or the reported sensitivity is not attributable to the Taylor truncation. Because the paper's scientific claim is that truncating the Taylor expansion of -log p suppresses ID samples in noisy OOD data, the method as written is not a Taylor expansion of -log p and its explanatory mechanism (Fig. 6, Fig. 7) is not tied to the actual loss. The empirical superiority over OE may still hold for whatever loss was implemented, but the current manuscript cannot support the stated mechanism without correcting Eq. 4/5 and re-examining the experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Taylor Outlier Exposure (TaylorOE), a modification of Outlier Exposure (OE) for training with auxiliary OOD datasets contaminated by in-distribution (ID) samples. OE regularizes by minimizing the average negative log-probability (Eq. 3); TaylorOE replaces this penalty with a truncated Taylor expansion of -log p around p=1, with order t controlling how much the loss is suppressed for high-confidence, ID-like samples. The authors motivate the design by measuring that a pretrained model assigns larger Loe values to ID than to OOD samples (Sec. 4.1), and they evaluate on CIFAR-10/CIFAR-100 with six OOD test sets over noise ratios pi in {0.05,...,1.0}, comparing with OE and WOODS, and applying the idea to two OE extensions. They report that TaylorOE outperforms baselines especially at low pi, and they release code.","tokens_in":13172,"tokens_out":5975,"duration_ms":59800,"significance":"The practical problem (learning from noisy auxiliary OOD data without cleaning) is relevant, and the empirical study is broad: six test OOD sets, multiple contamination ratios, five seeds, and comparisons to OE, WOODS, and energy baselines. The paper also tests applicability to OOD resampling and synthesis, and it ships code. If the derivation and experiments are corrected, the idea of a polynomial regularizer with order-controlled suppression is a plausible and simple contribution. However, the current manuscript's central mechanism rests on an incorrect Taylor expansion, and until that is fixed the quantitative claims cannot be interpreted as evidence for the proposed explanation.","major_comments":[{"comment":"The Taylor expansion of -log p about p=1 is -log p = sum_{n=1}^infty (1-p)^n / n, not sum (1-p)^n / n!. As printed, the infinite sum in Eq. (4) equals e^{1-p} - 1, which is bounded by e-1 approx 1.718 at p=0 and does not diverge like -log p. Consequently Ltoe in Eq. (5) does not converge to Loe as t tends to infinity, contradicting the sentence in Sec. 4.2 that the regularization \"gradually converges to Loe\". This is not a cosmetic typo: the polynomial coefficients determine the saturation behavior that Fig. 6 and the motivation in Sec. 4.1 rely on. The authors must correct Eqs. (4)-(5) and reconcile Fig. 5 and the convergence claim; if the experiments used coefficients 1/n!, then the reported results do not test the Taylor mechanism that the paper claims.","section":"Sec. 4.2, Eqs. (4)-(5)"},{"comment":"Under Eq. (5) with 1/n! coefficients, the terms with n>10 contribute less than about 10^{-6} for p in [0,1], so choosing t from {30,...,300} or {250,...,300} would have essentially no effect on Ltoe. Yet Figs. 8 and 9 report clear FPR95 changes across exactly these ranges, and Sec. 5.1 describes selecting t from {30,...,300}. This inconsistency suggests either the implementation uses different coefficients from those printed, or the reported t-sensitivity is due to something other than the Taylor truncation. Either way, the experiments as presented do not support the stated mechanism of order-based regularization.","section":"Secs. 5.1 and 6, Figs. 8-9"},{"comment":"The paper reports five-seed averages without error bars or significance tests. In the low-noise regime (pi=0.9,1.0), many differences between OE and TaylorOE are extremely small (e.g., Table 1 at pi=1.0 gives AUROC 99.39 vs 99.41 for SVHN and 99.30 vs 99.31 for LSUN-R), so the abstract's claim of \"consistently outperforms conventional methods\" is not supported for those regimes. Adding standard deviations or a paired significance test is necessary, especially because Table 3 averages over six datasets and may hide variance.","section":"Tables 1 and 3, Sec. 5.2"},{"comment":"The empirical premise that ID samples in the noisy auxiliary dataset have larger Loe (and hence need suppression) is demonstrated with a single pretrained WRN on CIFAR-10 against 300K Random Images. The method is then applied to CIFAR-100 and to different noise ratios and test OOD sets without showing that the separation persists in those settings. If the ordering of Loe between ID and OOD is weaker for another ID/OOD pair, the polynomial truncation may suppress useful OOD gradients rather than only noisy ID gradients. The authors should verify the separation on CIFAR-100, or at least discuss the risk, since Fig. 9 indicates that the optimal t changes with the ID dataset and class count.","section":"Sec. 4.1, Fig. 3"}],"minor_comments":[{"comment":"The section heading \"Propeties of OE\" should be \"Properties of OE\".","section":"Sec. 4.1"},{"comment":"The captions and text refer to \"information content\" and to \"Loe values\" for TaylorOE; these should be clarified to distinguish the regularization value during training from the loss of the trained model.","section":"Sec. 4.2 and Figs. 4, 7"},{"comment":"The text should specify that the Taylor expansion is about p=1 for p in (0,1] and that the series diverges at p=0; this is important for the reader to understand the behavior near p=0.","section":"Eq. (4)"},{"comment":"The pi column is repeated inside the table body, which is a formatting artifact that makes the table harder to read.","section":"Table 3"},{"comment":"The captions should state explicitly that lower FPR95 is better, as this is only implicit from the text.","section":"Figs. 8-9"}],"recommendation":"major_revision","confidential_remarks":"The main risk for the editor is that the mismatch between Eq. (5) and the reported t-sensitivity suggests that the code may implement a different loss. The manuscript includes a GitHub link; an editor or reviewer with access to the code should check whether the coefficients are 1/n or 1/n!. If 1/n, only the equations and figures need correction; if 1/n!, the experiments do not validate the Taylor-expansion story and a significant revision is needed. This is a correctness issue, not a novelty dispute."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version: this is a genuinely useful idea — truncate the OE regularization polynomial so confident ID samples in noisy auxiliary data get less gradient — and the experiments are broad and consistent at the noise levels that matter. But there is a load-bearing error in the printed math: Eqs. 4 and 5 state the expansion of -log p with coefficients 1/n!, which is not the Taylor expansion. The correct coefficients are 1/n. With 1/n!, the infinite sum equals e^{1-p}-1, which stays bounded near 1.7, so the truncated sum never approximates -log p, and varying t from 30 to 300 changes almost nothing. The paper nevertheless reports clear t-dependent optima in exactly that range. So either the equations are a typo and the code uses the correct 1/n coefficients, or the reported sensitivity is not attributable to the truncation. This is the first thing a referee must check.\n\nWhat is actually new: prior work (Feng et al., PolyLoss) applies polynomial truncation to cross-entropy for label noise; this paper is the first to apply it to the OE regularization term, and that move is well-motivated by their analysis that ID samples in the noisy OOD set produce large -log p values. The experiments are solid in scope: CIFAR-10 and CIFAR-100 as ID, six OOD test sets, noise ratios from 0.05 to 1.0, five seeds. The gains at low pi are large (FPR95 improvements of 10-20+ points) and consistent across datasets. The application to Resampling and DivOE is a nice bonus that shows it drops into existing pipelines.\n\nSoft spots beyond the math error: no error bars or significance tests, so the differences at high pi might be noise. The abstract's 'consistently outperforms' overstates things; at pi=0.9 and 1.0, TaylorOE and OE are essentially tied, and some AUROC values are slightly lower. The motivation in Sec. 4.1 rests on a single pretrained WRN and one OOD source; that's thin but acceptable as a starting point.\n\nMy take: the idea is worth publishing if the coefficients are fixed and the implementation is verified. As written, the paper can't support its own mechanism. I'd send it to review with a request to correct Eqs. 4-5, re-run the t-sensitivity analysis with the actual loss, and add error bars. The empirical trend at low pi is strong enough that I'd expect the method to survive the fix.","headline":"A useful extension of OE with a real mathematical error in the printed Taylor expansion — the empirical results may hold, but the mechanism as written does not.","tokens_in":13646,"tokens_out":3086,"would_cite":false,"duration_ms":29802,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TaylorOE replaces the OE regularization term with a truncated Taylor polynomial of $-\\log p$, so that in-distribution contamination in a noisy auxiliary OOD dataset is suppressed and OOD detection training works without data cleaning.","keywords":["out-of-distribution detection","outlier exposure","Taylor expansion","polynomial loss","noisy auxiliary data","robust regularization","CIFAR-10","CIFAR-100"],"falsifier":"Run the Sec. 4.1 histogram analysis on an ID/OOD pair with different statistical texture, for example ImageNet as ID and a visually similar OOD set, using a pretrained ResNet; if the $L_{\\mathrm{oe}}$ distribution for ID samples does not sit noticeably to the right of the OOD distribution, then a single truncation order $t$ cannot separate harmful from useful samples, and the reported FPR95 gains over OE should shrink or invert at low $\\pi$.","tokens_in":1907,"feed_emoji":"🎯","tokens_out":2485,"duration_ms":156686,"temperature":0.7,"pith_summary":"This paper aims to make Outlier Exposure (OE) work when the auxiliary OOD dataset is contaminated with in-distribution (ID) samples, so that no data cleaning or additional OOD sampling is needed. The proposed TaylorOE replaces the OE regularization term with a truncated Taylor polynomial of the per-class negative log-probability, so that confident ID samples contribute far less gradient than clean OOD samples. The paper reports consistent improvements over OE and the WOODS baseline across noise ratios from 5% to 100% on CIFAR-10 and CIFAR-100, with the largest gains at the highest contamination levels. If the claim holds, practitioners can use raw, uncurated web-scale auxiliary data for OOD detection.","feed_headline":"TaylorOE trains OOD detectors on dirty auxiliary data","feed_subtitle":"Truncating the OE loss's Taylor expansion suppresses in-distribution contamination, beating OE and WOODS.","key_machinery":"The central object is the truncated Taylor polynomial $L_{\\mathrm{toe}}(f(x)) = \\frac{1}{K}\\sum_{i=1}^K \\sum_{n=1}^{t} \\frac{(1-p_i)^n}{n!}$, which is the paper's replacement for the OE term $L_{\\mathrm{oe}} = \\frac{1}{K}\\sum_i -\\log p_i$. Taylor-expanding $-\\log p$ around $p=1$ and keeping only the first $t$ terms yields a loss whose gradient is bounded and vanishes for samples with extremely confident predictions, so samples with very large logits (the ID contamination in the noisy auxiliary set) stop influencing training. The order $t$ is the control knob: low $t$ suppresses more of the high-confidence tail, high $t$ approximates OE. The argument rests on the empirical separation, established in Sec. 4.1 with a pretrained Wide Residual Network on CIFAR-10 versus 300K Random Images, that ID samples have larger $L_{\\mathrm{oe}}$ values than clean OOD samples.","core_discovery":"TaylorOE asserts that the failure mode of OE on noisy auxiliary data is the tail of the OE loss $L_{\\mathrm{oe}} = \\frac{1}{K}\\sum_i -\\log p_i$, which is dominated by classes whose predicted probability is near zero. Because in-distribution samples concentrate probability on one class, their $-\\log p$ values are large across many classes, making their $L_{\\mathrm{oe}}$ systematically larger than that of clean OOD samples (Sec. 4.1). Truncating the Taylor expansion $-\\log p = \\sum_{n=1}^{\\infty} (1-p)^n/n!$ at a finite order $t$ gives $L_{\\mathrm{toe}} = \\frac{1}{K}\\sum_i\\sum_{n=1}^{t} (1-p_i)^n/n!$, a polynomial that closely matches $-\\log p$ near $p=1$ but lies below it near $p=0$. With a suitable $t$, the gradient contributed by confident ID samples is suppressed or becomes zero, while clean OOD samples keep a regularization nearly identical to OE. The paper claims that, across noise ratios $\\pi \\in \\{0.05, \\ldots, 1.0\\}$, TaylorOE consistently outperforms OE and WOODS (Tables 1–3), and that the regularizer can be dropped into other OE variants such as OOD resampling and OOD synthesis.","pith_inferences":["A natural extension would be an adaptive, per-sample truncation order (or a soft gate on logits) instead of a global $t$, which could remove the reported sensitivity to $\\pi$ and dataset.","The same truncation idea could be applied to other per-sample regularizers whose magnitude is monotone in a confidence score (e.g., energy loss), turning any noisy auxiliary dataset into a usable training signal.","The paper's key hypothesis is distributional rather than architectural; a cheap test is to measure the overlap of $L_{\\mathrm{oe}}$ histograms on a new dataset pair before committing to training.","Because TaylorOE and PolyLoss both arise from polynomial expansions of classification losses, the two strands could be unified into a single robust-OOD framework, though the paper does not make that connection."],"forward_implications":["OE-style training becomes feasible directly on raw, uncurated auxiliary data, eliminating the cleaning step that OE currently requires.","At high contamination ratios ($\\pi=0.05$), TaylorOE improves average FPR95 over OE by about 16.7 points on CIFAR-10 and by comparable margins on CIFAR-100.","Because $L_{\\mathrm{toe}}$ is a drop-in replacement for $L_{\\mathrm{oe}}$, it can be combined with other OE extensions, such as OOD resampling and OOD synthesis, without retraining or redesign.","As the noise ratio rises (cleaner auxiliary data), the optimal truncation order $t$ increases, so TaylorOE automatically approaches plain OE performance at $\\pi=1$.","The relationship between $\\pi$, the number of ID classes, and the optimal $t$ (Sec. 6) gives a practical tuning rule: use smaller $t$ for more contaminated or lower-class-count ID data, larger $t$ otherwise."],"supporting_citations":[{"why":"Defines the Outlier Exposure method that TaylorOE extends and serves as the primary baseline throughout the experiments.","marker":"[9]"},{"why":"Supplies the noisy OOD data construction via the Huber contamination model and the WOODS baseline that TaylorOE competes against.","marker":"[14]"},{"why":"Introduces the idea of a robust loss derived from a Taylor expansion of cross-entropy, which motivates the polynomial regularizer in TaylorOE.","marker":"[7]"},{"why":"Provides the energy score used as the OOD scoring function in all experiments and the no-fine-tuning baseline.","marker":"[21]"},{"why":"Generalizes polynomial expansions of classification loss functions, a concept TaylorOE adapts to OOD regularization.","marker":"[17]"},{"why":"One of the OE extensions (background data resampling) on which TaylorOE is demonstrated as a drop-in replacement.","marker":"[18]"},{"why":"One of the OE extensions (diversified outlier exposure via OOD synthesis) on which TaylorOE is demonstrated as a drop-in replacement.","marker":"[37]"}],"fun_headline_variants":["TaylorOE trains OOD detectors with contaminated data","Truncated Taylor expansion fixes Outlier Exposure","Noisy OOD data? TaylorOE suppresses ID contamination","TaylorOE: robust OOD detection on dirty auxiliary sets","Polynomial OE loss handles polluted OOD datasets"],"cache_read_input_tokens":15744,"weakest_assumption_plain":"The method's usefulness depends on the empirical claim, measured on one pretrained CIFAR-10 model, that in-distribution samples in the noisy auxiliary dataset have systematically larger negative-log-probability values than clean OOD samples; if that ordering is weak or reversed for another ID/OOD pair, the truncation suppresses useful OOD gradients as much as the harmful ID ones.","fun_headline_variants_meta":{"raw":{"variants":["TaylorOE trains OOD detectors with contaminated data","Truncated Taylor expansion fixes Outlier Exposure","Noisy OOD data? TaylorOE suppresses ID contamination","TaylorOE: robust OOD detection on dirty auxiliary sets","Polynomial OE loss handles polluted OOD datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000521,"raw_usage":{"total_tokens":2587,"prompt_tokens":1074,"completion_tokens":1513,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":690,"completion_tokens_details":{"reasoning_tokens":1438}},"tokens_in":690,"tokens_out":1513,"duration_ms":12249,"temperature":1.0,"reasoning_tokens":1438,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:55:47.102726+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Sec. 4.1 histogram analysis on an ID/OOD pair with different statistical texture, for example ImageNet as ID and a visually similar OOD set, using a pretrained ResNet; if the $L_{\\mathrm{oe}}$ distribution for ID samples does not sit noticeably to the right of the OOD distribution, then a single truncation order $t$ cannot separate harmful from useful samples, and the reported FPR95 gains over OE should shrink or invert at low $\\pi$.","supporting_citations":[{"cited_title":"Deep anomaly detection with outlier exposure","cited_arxiv_id":null,"evidence_quote":"Defines the Outlier Exposure method that TaylorOE extends and serves as the primary baseline throughout the experiments."},{"cited_title":"Training ood detectors in their natural habitats","cited_arxiv_id":null,"evidence_quote":"Supplies the noisy OOD data construction via the Huber contamination model and the WOODS baseline that TaylorOE competes against."},{"cited_title":"Can cross entropy loss be robust to label noise? In IJCAI, pages 2206–2212, 2021","cited_arxiv_id":null,"evidence_quote":"Introduces the idea of a robust loss derived from a Taylor expansion of cross-entropy, which motivates the polynomial regularizer in TaylorOE."},{"cited_title":"Energy-based out-of-distribution detection","cited_arxiv_id":null,"evidence_quote":"Provides the energy score used as the OOD scoring function in all experiments and the no-fine-tuning baseline."},{"cited_title":"Polyloss: A polynomial expansion perspective of classifica- tion loss functions","cited_arxiv_id":null,"evidence_quote":"Generalizes polynomial expansions of classification loss functions, a concept TaylorOE adapts to OOD regularization."},{"cited_title":"Background data resampling for outlier-aware classification","cited_arxiv_id":null,"evidence_quote":"One of the OE extensions (background data resampling) on which TaylorOE is demonstrated as a drop-in replacement."},{"cited_title":"Diversified outlier ex- posure for out-of-distribution detection via informative ex- trapolation","cited_arxiv_id":null,"evidence_quote":"One of the OE extensions (diversified outlier exposure via OOD synthesis) on which TaylorOE is demonstrated as a drop-in replacement."}],"review_version":1}