{"id":"bb41e7ef-58bd-4d96-b535-3e186b249d92","arxiv_id":"2411.15592","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using SVM on pretrained ResNet-50 features improved test accuracy and erythroblast precision over a fine-tuned ResNet-50 when only 1% of the blood smear dataset was used for training.","lead":"The paper tests whether replacing a deep network's final classification layer with a traditional machine learning classifier, such as SVM, improves blood cell classification when training images are scarce. On a public 8-class blood smear dataset, SVM on ResNet-50 features reached 86.75% test accuracy with only 1% of the training data, slightly above the 82.03% of a fine-tuned ResNet-50 alone.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SVM is not 'consistently' superior in the paper's own Table 3: it trails ResNet-50 at the 7.5%, 10%, and 20% splits and trails KNN at 30%, so the central claim is overbroad; the 1% advantage also has no error bars or repeated splits.","rationale":"The reader's weakest assumption focused on test-set leakage through backbone selection. My most load-bearing concern is different but related: the paper's own Table 3 contradicts the 'consistently surpasses' wording, and the single-split design means even the 1% SVM advantage is not established as reliable. The reader's rationale did mention overclaimed consistency and lack of error bars, so there is partial agreement. These issues are addressable with repeated holdout splits and corrected claims, so a conditional verdict remains appropriate; I would not move to reject. The paper does provide a concrete dataset, code link, and a falsifiable limited-data comparison, which are real strengths, but the central claim needs to be narrowed to what the evidence supports.","tokens_in":9483,"tokens_out":6768,"duration_ms":59359,"concrete_test":"Recompute the comparison on a truly held-out test set, separate from backbone selection, with 10 random 1% training draws (168 images total, 21 per class, different random seeds, same preprocessing). Report mean plus/minus standard deviation of top-1 accuracy and erythroblast precision, recall, and F1 for ResNet-50 and ResNet-50-SVM. If the SVM advantage is not consistently positive and outside the observed split-to-split variation, the abstract's 'consistently surpasses' claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim that 'ResNet50-SVM classifier consistently surpasses other models' overall test accuracy and erythroblast detection accuracy' is contradicted by the paper's own results in Table 3. For overall test accuracy, SVM trails plain ResNet-50 at the 7.5% split (96.07 vs 96.29), at 10% (96.00 vs 96.14), and at 20% (97.30 vs 97.66); at 30%, KNN is the best (98.45 vs 98.42 for SVM). At 1%, SVM's erythroblast recall is 0.812 versus ResNet-50's 0.857, and its F1 is 0.892 versus 0.917, so SVM does not consistently win on erythroblast detection either. The only clear SVM wins are top-1 accuracy at the 1% and 2.5% splits, and the 4.7 percentage-point advantage at 1% is a single split with no error bars, repeated-seed experiments, or confidence intervals, making it impossible to distinguish from sampling noise. Because the central contribution is that SVM improves limited-data performance, the evidence as reported does not establish 'consistently surpasses'; it supports at most a qualified claim about one split. The fixed 4,000-image test set is also used to select the ResNet-50 backbone in Section 4.1, so no fully held-out evaluation is reported, which compounds the single-split problem.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid pipeline in which a pre-trained ResNet-50 is used as a fixed feature extractor and the resulting features are fed to classical machine learning classifiers (SVM, XGBoost, KNN, Random Forest) for eight-class peripheral blood cell classification, with emphasis on erythroblast detection under small training fractions (1% to 30% of the dataset). The authors report that the ResNet-50-SVM combination consistently surpasses both plain fine-tuned ResNet-50 and the other classifier hybrids in test accuracy and erythroblast detection, and they argue this is useful for resource-constrained clinical settings. The manuscript includes a backbone comparison, a fixed 4,000-image test set, per-split accuracy and erythroblast precision/recall/F1 tables, and a discussion of limitations related to overlapping cells and staining variability.","tokens_in":9869,"tokens_out":2682,"duration_ms":25066,"significance":"If the central claim were established, the finding would be practically relevant: a data-efficient blood-cell classifier that maintains high accuracy and erythroblast detection with only 168 images per class could benefit low-resource hematology settings. The paper also has concrete strengths: it uses a public dataset, reports code and data availability, compares several classifiers against a fine-tuned deep baseline, and openly discusses clinical failure modes such as overlapping cells and staining variation. However, the main empirical claim rests on a single split with no uncertainty quantification and on a test set that appears to have been used for model selection, so the current evidence does not support the strength of the stated conclusion.","major_comments":[{"comment":"The abstract states that 'ResNet50-SVM classifier consistently surpasses other models' overall test accuracy and erythroblast detection accuracy,' but Table 3 contradicts this. Plain ResNet-50 has higher test accuracy than ResNet-50-SVM at the 7.5% split (96.29 vs. 96.07), the 10% split (96.14 vs. 96.00), and the 20% split (97.66 vs. 97.30), while at the 30% split KNN is highest (98.45 vs. 98.42 for SVM). For erythroblast detection at the 1% split, SVM has lower recall (0.812 vs. 0.857) and lower F1 (0.892 vs. 0.917) than plain ResNet-50. The evidence therefore supports at most a qualified claim about low-data test accuracy, not consistent superiority.","section":"Abstract and Table 3"},{"comment":"All reported results in Table 3 appear to come from a single partition of the data into one training set, one validation set, and one fixed 4,000-image test set. No error bars, repeated subsampling, multiple seeds, or statistical significance tests are provided, so the headline 4.7-percentage-point advantage of SVM over ResNet-50 at the 1% split cannot be distinguished from sampling noise. The authors should report means and standard deviations over repeated runs or, at minimum, confidence intervals for the key comparisons.","section":"Section 4.2 and Table 3"},{"comment":"The fixed 4,000-image test set appears to be used both for selecting the backbone and for evaluating the final classifiers. Section 4.1 selects ResNet-50 as the best backbone based on Table 1 'test accuracy,' and Section 4.2 then fixes the same 4,000-image testing set for all split experiments. This means no fully held-out evaluation is reported, and the reported SVM advantage may be inflated by selection on test-set performance. The authors should either reserve a separate test set untouched by any model selection or explicitly show that the conclusions are stable across multiple random splits.","section":"Sections 4.1 and 4.2"},{"comment":"The hyperparameter optimization procedure is described only by the generic equation GridSearch = arg max over accuracy, with no details of the candidate grids, the validation protocol used during grid search, or the final hyperparameter values for SVM, KNN, Random Forest, and XGBoost. If the grid search was evaluated on the same 4,000-image test set, this would be additional test-set leakage; if it was evaluated on the validation set, the authors should say so and report the chosen hyperparameters for reproducibility.","section":"Section 4.3"}],"minor_comments":[{"comment":"There is a typo in 'Preproccesing'; it should be 'Preprocessing'.","section":"Section 4.3"},{"comment":"In the related work paragraph on Chola et al., 'suing' should be 'using'.","section":"Section 2"},{"comment":"The cross-entropy loss expression lacks an explicit summation index; it should be L(y, y_hat) = -sum_i y_i log(y_hat_i) or an equivalent class-indexed sum.","section":"Equation (1)"},{"comment":"The caption says 'Test accuracy V/S percentage of the test data,' but the x-axis appears to denote the percentage of training data used; please correct the wording.","section":"Figure 3 caption"},{"comment":"The manuscript describes 5-fold cross-validation over the validation data but then reports results on a fixed 4,000-image test set in Table 3; the relationship between the cross-validated metrics and the reported test metrics should be clarified.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a worthwhile applied problem and is transparent about data and code availability, but the central claim is overbroad relative to the reported evidence. The main fixes—proper held-out evaluation, uncertainty quantification, and aligning the abstract/conclusion with the actual results—are within the scope of a revision, so I do not recommend rejection. Please also ensure the authors do not select models on the same test set used for final accuracy reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this one. The new bit is a clean benchmark of classic classifiers (SVM, KNN, RF, XGB) on deep features from ResNet-50 for eight-class blood-cell classification across training splits from 1% to 30% on the public Mendeley PBC dataset. That is genuinely useful for labs with very small labeled sets. The catch is that the paper's headline claim—'consistently surpasses'—does not survive contact with its own Table 3.\n\nAt 1% and 2.5% splits, SVM does beat the plain ResNet-50 baseline (86.75 vs 82.03 at 1%), and erythroblast precision is slightly better. But at 7.5%, 10%, and 20% the plain ResNet-50 is actually ahead on top-1 accuracy, and at 30% KNN is the best. So the support for 'consistently surpasses' is weak. The 1% gap is also a single split with no error bars, no repeated seeds, and no confidence intervals. It could be real, but the paper doesn't give you a way to tell.\n\nThe experimental protocol is under-specified. Section 4.1 selects ResNet-50 based on top-1 test accuracy on a 15% split, then Section 4.2 fixes a different 4000-image test set for all split experiments. The relationship between those two test sets isn't stated. The methodology also describes 5-fold cross-validation with mean and standard deviation, but Table 3 reports single numbers, so it's unclear whether the results are averages, one run, or something else. That needs to be cleaned up before anyone should trust the ranking.\n\nTo be fair, the paper does several things right. It uses a public dataset, links the code, reports per-class erythroblast precision/recall/F1, and is honest in the discussion about clinical limitations (overlapping cells, staining variability, cropped images). The features come from ImageNet pretraining, so the result is not forced by construction; the SVM-at-1% finding is a legitimate empirical observation worth reporting.\n\nWho should read it: anyone doing blood-cell classification with little data, and anyone teaching experimental hygiene in ML-for-medicine. Who shouldn't: people looking for a rigorous comparison or a deployed model. I'd send it to peer review, because the question is real and the flaws are fixable; the authors need to rerun with multiple seeds, report means and intervals, and rewrite the abstract to match the data.","headline":"A useful small-data benchmark and a cautionary tale: the SVM advantage is real only at the smallest split, and the paper overstates its consistency.","tokens_in":10392,"tokens_out":3278,"would_cite":false,"duration_ms":28747,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a pretrained ResNet-50 used for feature extraction plus an SVM classifier outperforms fine-tuning the network on blood-smear images when training data is scarce, reporting 86.75% top-1 accuracy with 168 images per…","keywords":["erythroblast","nucleated red blood cells","blood smear classification","ResNet-50","support vector machine","limited training data","transfer learning"],"falsifier":"Re-run the 1% and 2.5% training splits while setting aside an untouched test set used only at the very end, with network architecture chosen by validation accuracy alone; if ResNet-50-SVM no longer beats the fine-tuned ResNet-50 by a clear margin, the central claim fails.","tokens_in":9325,"feed_emoji":"🩸","tokens_out":9196,"duration_ms":72213,"temperature":0.7,"pith_summary":"The paper tries to establish that a two-stage pipeline—a pretrained deep network used only to extract features, followed by a traditional classifier like an SVM—identifies erythroblasts in blood smear images more accurately than fine-tuning the deep network itself when labeled images are scarce. It reports that with just 168 images per class (1% of the available data), the ResNet-50+SVM version reaches 86.75% top-1 test accuracy and 98.9% erythroblast precision, against 82.03% and 98.6% for a fine-tuned ResNet-50. The authors argue this matters because high-accuracy blood-cell classification could support diagnosis of hematological disorders in resource-limited settings, where large labeled datasets are rarely available.","feed_headline":"Bolting an SVM onto ResNet-50 beats fine-tuning at 1% training data","feed_subtitle":"With only 168 images per class, the hybrid model hits 86.75% test accuracy versus 82.03% for the network alone.","key_machinery":"The central object is the two-stage classification pipeline: the last pooling layer of a pretrained ResNet-50 (a deep residual network trained on natural images) produces a compact feature vector for each blood-cell image, and a support vector machine with grid-searched hyperparameters replaces the network's final classification layer. The SVM supplies a decision boundary that can be exactly tuned to the few hundred training examples, while the fixed feature extractor supplies transferable visual structure without being re-optimized on the small dataset. Training splits at 1%, 2.5%, 5%, 7.5%, 10%, 20%, and 30% of the full dataset, all evaluated against the same 4,000-image test set, make the data-scarcity comparison quantitative.","core_discovery":"On a fixed 4,000-image test set from an eight-class peripheral blood cell dataset, the paper reports that a pretrained ResNet-50 used to extract features, followed by a support vector machine trained on those features, reaches top-1 accuracy of 86.75% at the 1% training split (168 images per class), against 82.03% for the same network fine-tuned end-to-end. Erythroblast precision is 98.9% for the SVM version and 98.6% for the baseline. The advantage shrinks as training data grows and is essentially gone at 30% (98.42% vs 98.36%). The central claim is that the classifier-enhanced pipeline is a data-efficient alternative for differentiating erythroblasts from white blood cells in low-data settings.","pith_inferences":["A natural extension, not reported in the paper, is to rerun the 1% and 2.5% comparisons across multiple random splits; this would show whether the SVM advantage is stable or depends on the single split reported.","Because the improvement is carried by a shallow classifier on deep features, the same recipe should transfer to other pretrained networks and other small medical image datasets; that is a testable extrapolation, not stated by the authors.","If the SVM advantage survives independent replication, a practical consequence is that hospitals with limited labeled data could share a fixed feature extractor and train only the cheap classifier locally, which would also reduce the amount of patient data that needs to move.","The paper does not quantify how staining variation or overlapping cells shift accuracy, so a natural follow-up is to measure the SVM advantage on data from a second site or scanner."],"forward_implications":["At the 1% split (168 images per class), the ResNet-50-SVM pipeline reports 86.75% top-1 test accuracy, beating the fine-tuned ResNet-50 baseline at 82.03% while keeping erythroblast precision at 98.9%.","The classifier advantage is largest at small splits and narrows as data grows; at 30% both configurations sit near 98% test accuracy.","KNN on the same features also beats the baseline at 1% and 2.5%, so the improvement is not unique to SVM.","Using a feature extractor plus a shallow classifier avoids retraining the deep network, which is computationally cheap in resource-constrained clinical settings.","The same two-stage recipe can be applied to other small medical image datasets without extra segmentation or annotation effort."],"supporting_citations":[{"why":"It supplies the 17,092-image peripheral blood cell dataset that defines the eight classes and the limited-data splits.","marker":"[2]"},{"why":"It defines ResNet-50, the feature-extracting network used as the backbone in both baselines and classifier variants.","marker":"[12]"},{"why":"It introduces the SVM classifier whose two-stage combination with ResNet-50 is the paper's main proposed enhancement.","marker":"[13]"},{"why":"It provides the pretrained weights and normalization statistics that allow transfer learning from natural images to blood cells.","marker":"[9]"},{"why":"It describes the discriminative fine-tuning and one-cycle learning-rate schedule used to train the baseline network.","marker":"[14]"},{"why":"It justifies the grid-search procedure that tunes SVM, KNN, random forest, and XGBoost hyperparameters.","marker":"[22]"}],"fun_headline_variants":["SVM on ResNet-50 beats fine-tuning at 1% data","86.75% accuracy from just 168 images per class","Erythroblast ID: SVM layer makes ResNet-50 data-efficient","One classifier swap lifts ResNet-50 from 82% to 86.75%","For scarce data, a simple SVM improves deep learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume the fixed 4,000-image test set was not used to decide which network architecture to use in the first place; if that selection was influenced by the same test images, the SVM's apparent advantage at 1% data would not be out-of-sample.","fun_headline_variants_meta":{"raw":{"variants":["SVM on ResNet-50 beats fine-tuning at 1% data","86.75% accuracy from just 168 images per class","Erythroblast ID: SVM layer makes ResNet-50 data-efficient","One classifier swap lifts ResNet-50 from 82% to 86.75%","For scarce data, a simple SVM improves deep learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000801,"raw_usage":{"total_tokens":3527,"prompt_tokens":953,"completion_tokens":2574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":2477}},"tokens_in":569,"tokens_out":2574,"duration_ms":18649,"temperature":1.0,"reasoning_tokens":2477,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:07:37.090786+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 1% and 2.5% training splits while setting aside an untouched test set used only at the very end, with network architecture chosen by validation accuracy alone; if ResNet-50-SVM no longer beats the fine-tuned ResNet-50 by a clear margin, the central claim fails.","supporting_citations":[{"cited_title":"In: 2019 Second International Conference on Ad- vanced Computational and Communication Paradigms (ICACCP)","cited_arxiv_id":null,"evidence_quote":"It justifies the grid-search procedure that tunes SVM, KNN, random forest, and XGBoost hyperparameters."}],"review_version":1}