{"id":"89096f70-12a6-4005-ac3d-c83dda93b901","arxiv_id":"1908.02650","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Adding a regression constraint to a cross-entropy classifier improves cervical cell severity classification on Herlev from 70.1% to 74.5% accuracy and produces nucleus-focused attributions.","lead":"This paper trains a ResNet-101 on the Herlev cervical cell dataset and finds that adding a mean-squared-error regression penalty to the usual classification loss raises severity classification accuracy from 70.1% to 74.5%. It also uses integrated-gradient attribution to suggest the model relies on cell nuclei, matching clinical expectations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-split accuracy numbers, not ordinal spacing, are the weak link: the claimed 4.4-point gain has no error bars or repeated-split significance, and the binary accuracy is reported as both 94% and 96.7%.","rationale":"The reader's weakest_assumption identifies equal ordinal spacing as the load-bearing assumption. I see that as a modeling choice that could affect optimality, but not as the central condition for the paper's headline accuracy improvement. The more load-bearing issue is the validity of the evaluation: the severity and binary accuracy gains are claimed from single runs with no error bars, no specified split, and an internal inconsistency between the two reported binary accuracies. If the comparison is not repeated under matched conditions, the 4.4-point gain cannot be distinguished from split variance on a 917-image dataset. This is a concrete, testable concern rather than a theoretical objection. Because the paper's method is plausible and Figure 7 does hint at an AUC improvement, a conditional verdict requiring a rigorous repeated-split evaluation is appropriate. The ordinal spacing assumption is secondary; it could be tested by comparing with learned thresholds, but the first-order question is whether the reported gain is real. Thus I recommend keeping the reader's CONDITIONAL verdict, while shifting the stated weakest assumption to the evaluation protocol.","tokens_in":4769,"tokens_out":5187,"duration_ms":62814,"concrete_test":"Run 10 independent 80/20 splits with fixed seeds, identical preprocessing, data augmentation, and hyperparameters for both the classifier and classifier+regressor pipelines. For each split, record severity accuracy, binary accuracy, and mean AUC, then report mean +/- standard deviation and a paired test (e.g., Wilcoxon signed-rank). If the paired 95% confidence interval for the severity accuracy difference includes 0, or the observed 4.4-point gap is within one standard deviation of the split-to-split variation, the central claim is not supported. This same protocol should resolve the Section 3.2.3 vs Section 5 binary accuracy discrepancy by reporting the exact test-set value.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the MSE regression constraint improves severity accuracy from 70.1% to 74.5% and binary accuracy from 90.8% to 94%. For this claim to hold, those two numbers must come from a controlled comparison. Section 3.2.1 and 3.2.3 report each as a single figure, with no statement of the exact train/test split, no multiple seeds, and no error bars. The only multi-run evidence is the qualitative 4-fold AUC comparison in Figure 7, whose numeric values and paired statistics are not given. With 917 images, a typical 80/20 split yields roughly 183 test images, so a 4.4-point accuracy difference is about 8 images; under the class imbalance in Herlev, split-to-split variance can easily produce this size of difference. The ordinal equal-spacing assumption in Section 3.2.2 is a legitimate modeling choice but is not the main risk: even if the spacing were imperfect, the reported improvement could still be real or artifact of evaluation. The paper's own numbers are also internally inconsistent: Section 3.2.3 reports binary accuracy 94% with F1 0.96, while Section 5 reports 96.7% with F1 0.95. This inconsistency, together with the absence of released code, means the accuracy improvement is not yet established beyond split variance.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses automatic classification of squamous cervical cells from the public Herlev dataset. It proposes a 'classification + regression' pipeline in which a ResNet-101 classifier is trained with the standard cross-entropy loss plus an MSE penalty that encourages the softmax-derived expected score to match an ordinal severity score (1 to 5). The authors report that this constraint improves five-class severity accuracy from 70.1% to 74.5% and binary normal/abnormal accuracy from 90.8% to 94%, while preserving a probabilistic output. They also apply Integrated Gradients with a white baseline and use the dataset's nucleus/cytoplasm masks to quantify where the attribution falls, concluding that the model focuses increasingly on the nucleus as severity increases.","tokens_in":5040,"tokens_out":3078,"duration_ms":34386,"significance":"If the claimed accuracy improvement is statistically sound, the paper would make a modest but practically relevant contribution to automated cervical cancer screening, and the ordinal-constraint idea is a reasonable way to inject clinical knowledge into a standard classifier. The attribution analysis is a strength: it uses annotated cell masks to quantify the fraction of Integrated Gradient attribution inside the nucleus and cytoplasm, going beyond typical qualitative saliency maps. The paper also builds on a public dataset and reports comparisons with prior work. However, the central empirical claim currently rests on single-split point estimates without error bars or repeated-seed statistics, and the binary accuracy is reported inconsistently across sections. The attribution results are more convincing but cannot compensate for the lack of a rigorous evaluation of the headline accuracy improvement.","major_comments":[{"comment":"The central claim that the MSE regression constraint improves severity accuracy from 70.1% to 74.5% and binary accuracy from 90.8% to 94% is supported only by single-point estimates on an unspecified train/test split. With 917 images, a 4.4-point severity-accuracy gap corresponds to roughly 8 images on a typical 80/20 split, so split-to-split variance and class imbalance could easily produce this difference. Figure 7 shows 4-fold AUC distributions but does not report the numeric AUC values, per-fold results, or paired statistics, so it does not establish a significant difference for AUC either. Please provide repeated random-split results with means and standard deviations (or confidence intervals) for all reported metrics, and state the exact train/test protocol and number of seeds.","section":"Sections 3.2.1 and 3.2.3"},{"comment":"The binary accuracy is reported as 94% with F1 = 0.96 in Section 3.2.3, but Section 5 reports 96.7% with F1 = 0.95 for the same setting. This internal inconsistency makes it impossible to determine the actual headline result and must be corrected. If the two numbers refer to different evaluation conditions (e.g., different folds or a single best run), that distinction needs to be explicit.","section":"Section 3.2.3 vs. Section 5"},{"comment":"The method assumes that the five severity classes can be represented on an equally spaced interval scale (scores 1 through 5) for the MSE penalty. If the true clinical distances between classes are not equal, the regularizer could bias the learned representation. Since the regression constraint is the paper's main contribution, this assumption needs at least a discussion, and ideally a robustness check with alternative score spacing (e.g., ordinal logistic or unequal spacing) to show that the accuracy improvement is not an artifact of the specific score assignment.","section":"Sections 3.2.2 and 3.2.3"}],"minor_comments":[{"comment":"There are several typos: 'adresses' should be 'addresses' in the abstract, 'descibes' should be 'describes' in Section 3.2, and 'model agonistic' in Section 4 should be 'model agnostic'.","section":"Abstract and throughout"},{"comment":"The notation yx is used for both the one-hot label in the cross-entropy term and the scalar score in the MSE term. Please define separate symbols (e.g., y and s) to avoid ambiguity.","section":"Section 3.2.3, loss equation"},{"comment":"The statement 'our Resnet-101, Classifier + Regressor, makes less misclassifications than the classifier and lower MSE than the regressor' would benefit from specific numerical comparisons; as written, it is not clear which numbers support it.","section":"Section 3.2.3, text before Figure 5"},{"comment":"The notation \\sum_i could be clearer; specify that the sum runs over pixel indices in the corresponding region, and define N and C as sets of pixel indices rather than just 'nucleus and cytoplasm pixels'.","section":"Section 4, Equations (1) and (2)"},{"comment":"Several references are incomplete or inconsistently formatted (e.g., [6], [7], [9]); please ensure each entry includes author list, venue, year, and page numbers where applicable.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short, workshop-style contribution. The attribution analysis is a useful addition, but the classification claim needs a statistically rigorous evaluation before the paper can be considered for archival publication. The internal inconsistency in the binary accuracy is a red flag that must be resolved. I would not recommend reject, but the authors need to provide repeated-split results and clarify the evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one-sentence take: this is a sensible, small idea with a plausible accuracy bump, but the reported gain isn't statistically grounded, and the paper's own numbers disagree about how well the binary task went.\n\nWhat's actually new: applying a known ordinal-loss trick (CE + MSE on class indices) to the 5-class Herlev severity problem, and the quantitative attribution metrics AtN/AtC computed from the dataset's masks. The attribution analysis goes beyond the usual saliency maps by actually measuring where the gradients land relative to nucleus and cytoplasm. That is a useful, reproducible way to sanity-check what the model sees. The method itself is a simple combination of losses, so the novelty is modest, but the evaluation on Herlev severity with public masks is a legitimate addition.\n\nThe paper does a few things well. The loss formulation is clear and easy to reimplement. The AUC distributions in Figure 7, even if not fully numeric, suggest the regression constraint helps most on the extreme classes, which matches the confusion matrices. The motivation to avoid misclassifying carcinoma in situ as normal is medically sensible.\n\nNow the soft spots. The headline improvement—70.1% to 74.5% accuracy, 90.8% to 94% binary—comes from a single train/test split. There are no error bars, no multiple seeds, no statement of the split ratio. With 917 images, the 4.4-point gain is roughly 8 test images; split variance alone can generate that. The only multi-run evidence is Figure 7, but the numeric AUC values and paired statistics are missing. That is the load-bearing weakness. The ordinal equal-spacing assumption (1 to 5) is a legitimate modeling choice; it's not the main risk. Also, the binary accuracy is reported as 94% with F1 0.96 in Section 3.2.3 and 96.7% with F1 0.95 in the Conclusion. That inconsistency needs to be resolved before anyone can trust the abstract numbers. No code is released, which makes the exact split question impossible to audit.\n\nThe attribution results are suggestive, not conclusive. The AtN/AtC distributions are interesting, but without a baseline (e.g., a random model or a model trained without the regression constraint) it's hard to know whether the nucleus focus is specific to this loss. Minor, but worth saying.\n\nWho should read this: anyone working on ordinal classification for small medical image datasets, and people thinking about how to evaluate attribution methods quantitatively. It deserves a serious referee—the idea is reasonable and the dataset is public—but the revision needs repeated-split statistics and a corrected, consistent set of numbers. Send it to review, but expect to send it back.","headline":"A sensible ordinal-loss tweak with a plausible accuracy bump on Herlev, but the reported gain rests on a single split and the paper's own binary accuracy numbers disagree.","tokens_in":5552,"tokens_out":2287,"would_cite":false,"duration_ms":22551,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding an ordinal regression constraint to a cervical-cell classifier's loss function raises severity accuracy from 70.1% to 74.5% while keeping the model's class probabilities, and attribution maps show the model focuses on the nucleus.","keywords":["cervical cancer screening","Herlev dataset","ordinal regression","loss function design","integrated gradients","explainable AI","cytomorphological features","deep learning"],"falsifier":"A concrete check would be to compute a bootstrap confidence interval for the difference between 74.5% and 70.1% accuracy using the confusion matrices in Figures 1 and 6; if the interval includes 0, the reported improvement is not statistically established. Alternatively, replace the MSE regression target with a cumulative-link ordinal loss that does not assume equal spacing; if accuracy and AUC do not deteriorate, the equal-spacing assumption is not essential to the gain.","tokens_in":4546,"feed_emoji":"🔬","tokens_out":2814,"duration_ms":32492,"temperature":0.7,"pith_summary":"This paper tries to establish that a simple change to the training loss—adding an MSE penalty that encourages the softmax output to behave like a severity score on a 1-to-5 ordinal scale—improves a deep CNN's ability to grade cervical cell abnormalities. The authors argue this works because it explicitly encodes the ordinal structure of the WHO severity classification, which a plain cross-entropy loss ignores. They show that on the Herlev dataset this regression constraint lifts five-class severity accuracy from 70.1% to 74.5% and binary normal/abnormal accuracy from 90.8% to 94%, while also improving the model's focus on the nucleus as shown by integrated-gradient attributions. The practical payoff is a classifier that is both more accurate and more aligned with how cytotechnicians actually read slides.","feed_headline":"One loss tweak lifts cervical-cell grading from 70% to 74.5%","feed_subtitle":"Adding an ordinal-score penalty to the loss improves severity accuracy and shows the model looks at the nucleus.","key_machinery":"The central mechanism is the combined loss function $$L(x) = CE(p; y_x) + \\left(y_x - \\sum_{i=0}^{4} (i+1)\\,p_i\\right)^2,$$ where $p_i$ are the softmax class probabilities for a five-class severity label, $y_x$ is the ground-truth class index (1 to 5), and $CE$ is the cross-entropy loss. The second term is a regression constraint that turns the softmax distribution into an expected severity score and penalizes predictions that are far from the true ordinal class, effectively weighting misclassifications by their clinical distance. This simple addition is what carries the accuracy gain.","core_discovery":"The central claim is that coupling a standard cross-entropy classification loss with a mean-squared-error regression term on the expected severity score—computed as a fixed weighted sum of the softmax probabilities—yields a single architecture that outperforms either loss alone. On the Herlev severity dataset, the combined loss gives 74.5% overall accuracy and 94% binary normal/abnormal accuracy with an F1 of 0.96, while the plain classifier reaches 70.1% and 90.8%, respectively. The two extreme classes, normal and carcinoma in situ, show the largest AUC gains (0.98 and 0.97), correcting a known confusion between normal columnar and carcinoma in situ cells. The paper also claims that integrated-gradient attribution maps, evaluated against the dataset's nucleus and cytoplasm masks, show the malignancy scoring is driven mainly by the nucleus, with nucleus contribution increasing with severity.","pith_inferences":["The equal-spacing assumption (1 to 5) may be too coarse; if class distances are actually unequal, the regression penalty could distort the learned representation. A natural extension would be to learn the spacing or use an ordinal cumulative-link loss, then compare accuracy and calibration.","The attribution result—that nucleus contribution rises with severity—could be turned into a weak localization signal, potentially flagging images where the model looks outside the nucleus as cases deserving manual review.","The approach is not limited to cancer screening; any classification problem with ordered classes and a natural score interpretation could benefit from the same loss, though the magnitude of the gain would likely depend on how well the classes align with the score scale.","The reported accuracy gain of 4.4 percentage points is based on a single test split; re-running the comparison with multiple random seeds and a bootstrap confidence interval would tell whether the gain is robust or partly noise."],"forward_implications":["If this loss works as claimed, the same trick of adding an ordinal regression constraint to cross-entropy could be applied to other medical grading tasks with ordered categories, such as diabetic retinopathy or skin lesion severity.","The improvement in extreme classes suggests that the constraint reduces confusion between normal and carcinoma in situ cells, which is the most clinically dangerous error; this could make automated screening safer.","The quantitative attribution metrics (AtN, AtC) provide a way to verify that a classifier is using medically relevant features, potentially serving as a quality-control step before deployment.","The paper's binary accuracy of 94% matches the earlier reported binary result on Herlev, suggesting the method preserves binary performance while adding explainable severity grading."],"supporting_citations":[{"why":"Supplies the integrated-gradient attribution method used to localize discriminative features in the images.","marker":"[8]"},{"why":"The Bethesda guidelines define the cytomorphological features (nucleus-centric) that the attribution analysis is compared against.","marker":"[9]"},{"why":"Earlier CNN work on Herlev that reported the confusion between normal columnar and carcinoma in situ cells, providing the baseline problem the combined loss addresses.","marker":"[6]"},{"why":"ImageNet pretraining supplies the initial weights for the ResNet-101 backbone used in all experiments.","marker":"[10]"},{"why":"The ResNet-101 architecture with skipped connections is the model family whose classification head is modified with the regression constraint.","marker":"[5]"}],"fun_headline_variants":["MSE regression loss lifts cervical-cell accuracy from 70.1% to 74.5%","Adding severity-score regression improves cervical cancer classifier","Regression penalty on severity score boosts cervical cell grading","Combined loss improves cervical cancer screening accuracy to 74.5%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the five severity classes can be represented as equally spaced ordinal scores (1 to 5), so that a mean-squared-error penalty on the expected score is a meaningful regularizer; if the true clinical distances between classes are unequal, this constraint could distort the learned representation.","fun_headline_variants_meta":{"raw":{"variants":["MSE regression loss lifts cervical-cell accuracy from 70.1% to 74.5%","Adding severity-score regression improves cervical cancer classifier","Regression penalty on severity score boosts cervical cell grading","Combined loss improves cervical cancer screening accuracy to 74.5%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1283,"prompt_tokens":814,"completion_tokens":469,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":395}},"tokens_in":430,"tokens_out":469,"duration_ms":5164,"temperature":1.0,"reasoning_tokens":395,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:38:13.134572+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check would be to compute a bootstrap confidence interval for the difference between 74.5% and 70.1% accuracy using the confusion matrices in Figures 1 and 6; if the interval includes 0, the reported improvement is not statistically established. Alternatively, replace the MSE regression target with a cumulative-link ordinal loss that does not assume equal spacing; if accuracy and AUC do not deteriorate, the equal-spacing assumption is not essential to the gain.","supporting_citations":[{"cited_title":"Forslid and H","cited_arxiv_id":null,"evidence_quote":"Supplies the integrated-gradient attribution method used to localize discriminative features in the images."},{"cited_title":"ICML 2017","cited_arxiv_id":null,"evidence_quote":"The Bethesda guidelines define the cytomorphological features (nucleus-centric) that the attribution analysis is compared against."},{"cited_title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","cited_arxiv_id":null,"evidence_quote":"Earlier CNN work on Herlev that reported the confusion between normal columnar and carcinoma in situ cells, providing the baseline problem the combined loss addresses."},{"cited_title":"Solomon and D","cited_arxiv_id":null,"evidence_quote":"ImageNet pretraining supplies the initial weights for the ResNet-101 backbone used in all experiments."},{"cited_title":"Hinton ImageNet Classification with Deep Convolutional Neural Networks","cited_arxiv_id":null,"evidence_quote":"The ResNet-101 architecture with skipped connections is the model family whose classification head is modified with the regression constraint."}],"review_version":1}