{"id":"d31fdcb1-3f87-44af-aefb-3c5eb472e5e5","arxiv_id":"2505.13923","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"On 1,658 images of six African foods, a fine-tuned ResNet50 and an SVM using raw pixels both reach about 81 percent accuracy, with SVM slightly ahead on macro F1.","lead":"This paper compares two computer vision models, a deep neural network named ResNet50 and a classic support vector machine, on a small dataset of African food photos. Both reach about 81 percent accuracy, and the simpler SVM is slightly more consistent across the six dish categories.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set integrity and label quality are untested; the SVM-versus-ResNet50 conclusion is read entirely from one 251-image split.","rationale":"The reader's verdict is CONDITIONAL with the same central concern: the paper's conclusion is read entirely from one test split whose label quality and near-duplicate leakage were not checked. My stress-test pass independently confirms this is the most load-bearing assumption. I agree with the reader's identification of the weakest assumption, and I recommend keeping the verdict CONDITIONAL. The paper itself does not provide code or data inspection in the manuscript, and no formal verification or repeated runs are reported. The fairer baseline issue raised by the reader is real but secondary; the SVM with raw RGB is a deliberately simple baseline, and the claim that it is competitive does not depend on it being optimally tuned. The most concrete check is to rerun the SVM on the actual split and to check for leakage and label noise. If those pass, the modest qualitative claim holds.","tokens_in":8147,"tokens_out":1167,"duration_ms":10375,"concrete_test":"Download the Mendeley dataset, recompute per-class and overall accuracy for a raw-RGB RBF SVM (C=1, gamma=scale) on the exact 251-image test split described in Table I, and compute the maximum pairwise similarity (e.g., SSIM or perceptual hash) between each test image and every training image. If the test set contains near-duplicates of training images or the SVM accuracy differs by more than ~2 points from 80.88%, the central comparison is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that raw-RGB SVM and fine-tuned ResNet50 both reach ~81% accuracy on the African Foods dataset, with SVM slightly ahead on macro F1. The data source (Mendeley, rrzhwbg3kw) was not independently inspected or re-run during review. The paper reports custom splits in Table I that do not obviously match the source dataset's original split: the authors state the original split was 70/15/15, yet they combined the original training and validation sets and re-split with 5-fold cross-validation, without verifying that the source dataset's 251 test images are representative or free from near-duplicate leakage. If the test set contains near-duplicates of training images or mislabeled examples, both the absolute 81% figures and the SVM/ResNet50 comparison shift. The paper actually concedes that no independent verification was performed: the authors only state the split was 'provided by the source.' The claim of SVM competitiveness is an inference from a single test set, so the weakest link is the unverified premise that this test split is clean and representative.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents an empirical comparison between a fine-tuned ResNet50 and a default RBF-kernel SVM using raw RGB pixel features for classifying six African food classes from a public Mendeley dataset of 1,658 images. The authors combine the original training and validation splits, use five-fold cross-validation for the ResNet50 and select the fold with the highest validation accuracy, train the SVM on the same combined training set, and evaluate both models on the original 251-image test split. They report overall accuracy of about 81% for both models, with the SVM obtaining a slightly higher macro-averaged F1 score (0.82 versus 0.81), and they conclude that a classical model can remain competitive with a transfer-learned deep network on this small dataset. The paper also provides per-class precision, recall, F1, and confusion matrices, and it releases code for the ResNet50 pipeline.","tokens_in":8492,"tokens_out":5458,"duration_ms":55023,"significance":"If the empirical claims hold, this is a modest but useful benchmark for African food recognition. The explicit comparison of a raw-pixel SVM with a fine-tuned CNN on a public African foods dataset is a data point that the community can build on, and the paper has the concrete strengths of using a publicly available dataset, reporting per-class metrics rather than only overall accuracy, and releasing code for the deep learning pipeline. However, the central comparative claim currently rests on a single small test split, a single selected validation fold, and an untuned SVM baseline, so the quantitative conclusion that SVM is more consistent than ResNet50 is not yet firmly established.","major_comments":[{"comment":"The ResNet50 result is obtained from the single fold with the highest validation accuracy among five folds, rather than from repeated runs, averaged folds, or an ensemble. Selecting the best validation fold is an additional model-selection step on validation noise, so the reported 81% test accuracy does not represent the expected performance of the fine-tuning procedure. Please report per-fold test results and the mean plus/minus standard deviation, or use an ensemble over the folds, and also run multiple random seeds.","section":"Section II-A and Table III"},{"comment":"The headline comparative claim that the SVM is more consistent than ResNet50 rests on a macro-F1 difference of 0.01 (0.82 versus 0.81) and equal overall accuracy on a single 251-image test set. With per-class test sizes of 31 to 59 images, one or two misclassifications change a class-level F1 score by roughly 0.02 to 0.03, so the observed gap is within the magnitude of sampling noise. Bootstrap confidence intervals or a significance test are needed before concluding that one model is more consistent.","section":"Section III-C and Table III"},{"comment":"The SVM is trained with default scikit-learn hyperparameters (C=1.0, gamma='scale'), while the ResNet50 pipeline was manually tuned through a hyperparameter-selection process. This comparison confounds the model family with the amount of tuning effort. A simple grid search over C and gamma, or an explicit framing of the SVM as an untuned baseline, is needed before the paper can conclude that classical methods 'can still hold their ground' as a general alternative to deep learning.","section":"Section II-B and Section III-C"},{"comment":"All conclusions are read from the original 251-image test split, which the authors state they kept unchanged but did not audit. The paper reports no check for near-duplicate images across the training/test boundary, no label-noise analysis, and no representativeness analysis of the test images. Accidental leakage or systematic label errors would inflate both reported 81% accuracy figures, so a duplicate and near-duplicate check and, if feasible, a label audit should be added.","section":"Sections I-A, II-A, and II-B"}],"minor_comments":[{"comment":"There are several language and typographical issues: 'Food recognition systems has advanced' should be 'have advanced', 'african' should be capitalized, and 'Ghanian' should be 'Ghanaian'.","section":"Abstract and text"},{"comment":"The phrase 'Randomly rotate images vertically (0.2)' is ambiguous; clarify whether this is a random rotation range or a different transformation, and distinguish it from the listed horizontal flip.","section":"Section II-A"},{"comment":"The hyperparameter selection is described only as 'manually selected' with no search ranges or number of trials, which limits reproducibility of the tuning process.","section":"Section II-A"},{"comment":"The paper states that the full codebase is available, but only a GitHub link for the ResNet50 fine-tuning code is provided; no SVM or evaluation scripts are linked. Please add the missing code or adjust the claim.","section":"Contributions and Section II-A"},{"comment":"Reference [18] is cited for the use of HOG features with SVMs, but the cited paper is titled 'Image classification using random forests and ferns'; please verify and correct this citation.","section":"References"},{"comment":"The plot shows average training and validation loss across five folds but does not show fold-to-fold variance; adding a shaded standard-deviation band would make the claim of stable convergence more informative.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"This is a short applied paper rather than a methodological contribution. The central empirical claim is plausible but is currently supported by a single test split and a single selected fold. If the authors add per-fold results, uncertainty quantification, and a basic SVM hyperparameter search, the paper would be suitable for a venue accepting concise empirical studies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:2505.13923 is that it's a small, honest empirical comparison: a fine-tuned ResNet50 and a plain RBF SVM on raw RGB pixels both land at about 81% accuracy on a six-class African foods dataset. That result is plausible and useful for anyone building recognition systems for under-represented cuisines. What it is not is a proof that SVMs are generally competitive with deep networks; it's a single benchmark on a small, imbalanced dataset.\n\nWhat the paper does well: it gives per-class precision/recall/F1, not just overall accuracy, which lets you see where each model fails. It releases code, which is more than most papers at this level do. The authors also keep the claims modest; they call it explorative and they do not offer a theory for why SVM wins.\n\nThe soft spots are real but not disqualifying. There are no error bars or repeated runs; the ResNet50 result comes from the best of five validation folds, which inflates the expected accuracy. The SVM uses scikit-learn defaults, so the comparison is not perfectly fair—a tuned SVM might do better, or worse, depending on the data. And the test set, 251 images from the Mendeley African Foods dataset, was not independently checked for near-duplicates or label noise. If there is leakage, both numbers go down. The stress-test note is right to flag this; it is the weakest link. That said, the paper's conclusion is about this dataset, not a universal truth, so the issue limits generalization rather than invalidating the comparison.\n\nCitation pattern is fine. The self-citations are to the authors' own prior work, but they are not load-bearing; the related work on African food is appropriately cited.\n\nWho is this for? Practitioners in low-resource settings who want to know whether a simple SVM can match a fine-tuned CNN on a small dataset. It is not a methods paper. It deserves a serious referee because it is a legitimate empirical contribution with open code, but it needs a revision that adds variance estimates and acknowledges the test-set limitation.\n\nI would send it to a workshop or a short-paper track, not a main conference. On the question of peer review: yes, give it a proper review; just ask the authors to tighten the evaluation.","headline":"A modest, honest empirical comparison of SVM and ResNet50 on a small African foods dataset; the result is plausible, but the evaluation is thin on variance and the test set is unvalidated.","tokens_in":8900,"tokens_out":2361,"would_cite":false,"duration_ms":23272,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On six African dishes, a fine-tuned ResNet50 and a classical SVM both reach about 81% accuracy, with the SVM slightly ahead on macro F1.","keywords":["African food classification","ResNet50","Support Vector Machine","Transfer learning","Food image recognition","Comparative analysis","Raw pixel features","Imbalanced dataset"],"falsifier":"Re-running the two pipelines on the same train/test split while excluding any test images that are near-duplicates of training images, or re-testing on an independently collected set of the same six dishes, would settle the claim: if accuracy of either model falls well below 81% under that check, the reported comparison is an artifact of the dataset split.","tokens_in":7957,"feed_emoji":"🍲","tokens_out":4406,"duration_ms":37657,"temperature":0.7,"pith_summary":"This paper asks a practical question: on a small, imbalanced dataset of six popular African dishes, does a deep convolutional network fine-tuned by transfer learning actually beat a classical SVM trained on raw pixel values? The authors report that both approaches reach about 81% overall accuracy on the same 251-image test set, with the SVM achieving a slightly higher macro-averaged F1 score (0.82 versus 0.81). The point is that on this dataset, a simple classical pipeline holds its ground against a modern deep model, while the two models differ in which classes they confuse. If the finding holds, it matters for food recognition in low-resource settings, where lightweight classical models are easier to deploy.","feed_headline":"SVM matches fine-tuned ResNet50 at 81% on African foods","feed_subtitle":"On six popular dishes, a simple pixel-based classical model ties deep transfer learning and beats it on macro F1.","key_machinery":"The comparison rests on two pipelines. The SVM pipeline uses raw RGB pixel intensities resized to 100×100, flattened into 30,000-element vectors, and classified by an RBF-kernel SVM with default settings. The ResNet50 pipeline fine-tunes the last four layers of an ImageNet-pretrained ResNet50 with a global average pooling layer, a 1,024-neuron dense layer with dropout 0.5 and L2 regularization, and a 6-way softmax head, trained with 5-fold cross-validation and data augmentation. The 5-fold validation-based model selection and the fixed public test split are what make the two pipelines comparable.","core_discovery":"The central claim is empirical and comparative. On the African Foods dataset of 1,658 images across six classes, a fine-tuned ResNet50 and a radial-basis-function SVM fed with flattened 100×100 RGB pixels both achieve an overall test accuracy of about 81%. Per-class results diverge: ResNet50 is strongest on Ndole (precision 1.00, F1 0.91) and Palm-nut soup (recall 0.90), while the SVM is more consistent, leading in Ekwang and Eru and posting the better macro F1 (0.82 vs 0.81). The paper reads these results as evidence that classical methods remain competitive on small, imbalanced food datasets and that per-class behaviour, not just overall accuracy, should drive model choice.","pith_inferences":["The result is specific to a 251-image test set; a natural next test is whether the SVM's edge survives on a larger, more balanced African food dataset, which the paper itself lists as future work.","The large confusion between palm-nut soup and ekwang suggests a visually similar pair that could be targeted with class-aware losses or hierarchical classification.","Because the SVM uses only raw pixels, adding handcrafted features or color histograms might improve it further, a direction the paper does not test.","The paper's reliance on a single public dataset means the 81% figure should be read as a benchmark for this dataset, not a claim about African food recognition in general."],"forward_implications":["If the 81% figures are accurate, a lightweight SVM with raw pixels is a viable baseline for African food recognition on small datasets.","The macro F1 advantage of the SVM suggests that classical models can generalize more evenly across imbalanced classes in this setting.","ResNet50's strengths on particular classes imply that ensembles or hybrid pipelines could exploit the complementary errors of the two models.","The dataset-specific comparison indicates that without large, balanced data, deep transfer learning may not deliver a decisive advantage over a well-tuned classical method."],"supporting_citations":[{"why":"Supplies the African Foods dataset of 1,658 images across six classes, the empirical basis of the whole comparison.","marker":"[22]"},{"why":"Establishes ResNet50 as a prebuilt model trained on ImageNet, which the fine-tuning pipeline starts from.","marker":"[24]"},{"why":"Defines support-vector networks, the foundation of the classical SVM classifier used in the comparison.","marker":"[30]"},{"why":"Provides the transfer-learning procedure for fine-tuning a pretrained ResNet50 to a new classification task.","marker":"[26]"},{"why":"Demonstrates an earlier SVM-based food classification system, positioning the classical approach the paper revives.","marker":"[17]"}],"fun_headline_variants":["SVM ties ResNet50 on African foods, wins macro F1","Classic SVM beats deep net on African food macro F1","Pixel-only SVM matches ResNet50 at 81% on African dishes","Deep learning vs SVM: 81% tie, SVM leads macro F1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire comparison is read from a single 251-image test split of the African Foods dataset; if those images are mislabeled or contain near-duplicates of training images, both 81% figures would be inflated and the two models could no longer be compared as reported.","fun_headline_variants_meta":{"raw":{"variants":["SVM ties ResNet50 on African foods, wins macro F1","Classic SVM beats deep net on African food macro F1","Pixel-only SVM matches ResNet50 at 81% on African dishes","Deep learning vs SVM: 81% tie, SVM leads macro F1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1389,"prompt_tokens":825,"completion_tokens":564,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":486}},"tokens_in":441,"tokens_out":564,"duration_ms":5071,"temperature":1.0,"reasoning_tokens":486,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:07:02.903131+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-running the two pipelines on the same train/test split while excluding any test images that are near-duplicates of training images, or re-testing on an independently collected set of the same six dishes, would settle the claim: if accuracy of either model falls well below 81% under that check, the reported comparison is an artifact of the dataset split.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the African Foods dataset of 1,658 images across six classes, the empirical basis of the whole comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines support-vector networks, the foundation of the classical SVM classifier used in the comparison."},{"cited_title":"A., and Adebanjo, A","cited_arxiv_id":null,"evidence_quote":"Provides the transfer-learning procedure for fine-tuning a pretrained ResNet50 to a new classification task."},{"cited_title":"(2009, November)","cited_arxiv_id":null,"evidence_quote":"Demonstrates an earlier SVM-based food classification system, positioning the classical approach the paper revives."}],"review_version":1}