{"id":"f2ebc7c5-56a9-4510-9678-d780f73b5705","arxiv_id":"2412.20370","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A hybrid ensemble of YOLOv5, YOLOv8, and DETR with differential evolution weights and weighted boxes fusion improves mAP on a pre-made dish dataset.","lead":"This paper combines three object detection models, YOLOv5, YOLOv8 and DETR, into one ensemble that uses differential evolution to tune how much each model matters, and then merges their predictions. It reports improved accuracy on a dataset of pre-made dish photos with 11 ingredient categories, which may help automate quality checks in the pre-made food industry.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim of a significant DE-optimized gain is not yet testable: the paper reports no train/validation/test split, no error bars, and no equal-weight WBF ensemble baseline, so the 90.92 vs 88.41 mAP50 gap may reflect selection bias or ensemble benefit rather than DE optimization.","rationale":"The reader's weakest assumption correctly identified the risk that the same data influenced both DE weight selection and final evaluation. My stress-test concurs but goes slightly further: even with a clean split, the paper's central claim would still be under-supported because no equal-weight WBF baseline is reported, so the observed gain could come from ensembling three models rather than from the differential evolution component. The strongest claim is empirical, and the missing pieces are standard reproducibility items: split sizes, error bars, and an ablation. I found no internal mathematical inconsistency; the complexity analysis and algorithm description are plausible as far as they go. The paper does not provide code, data, or formal verification, so independent support is limited to the reported table. Because the concerns are addressable in a revision and the reader already issued a conditional verdict, I do not recommend changing the verdict. The agreement is partial because the reader focused on split leakage while I additionally emphasize the missing equal-weight control; the two concerns reinforce each other rather than conflict.","tokens_in":12342,"tokens_out":2885,"duration_ms":31518,"concrete_test":"Fix a split protocol (e.g., 60/20/20) and rerun all models; optimize DE weights only on the validation partition, then evaluate once on the held-out test partition. Add a control ensemble of the same three base models using equal weights in WBF, and repeat each configuration for at least five seeds reporting mean and standard deviation of mAP50. If DEIHDL's test mAP50 does not exceed both the best single model (YOLOX) and the equal-weight WBF ensemble by more than the seed-to-seed variation, the central claim should be weakened. Ideally the authors should release the split indices and code to make the check reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For the central claim to hold, the DE-optimized fusion weights must generalize to a test set that did not influence weight selection, and they must add value over simply fusing the same three base models with equal weights. The manuscript establishes neither condition. Section III.B states that the performance of each weight vector is evaluated on the validation set (Eq. 13), and Section IV.B reports final accuracy in Table V without stating whether those numbers come from a separate, never-seen test set or from the same validation set used for selection. Dataset statistics in Table III give only total images (2200) and instances (4609), with no train/validation/test split sizes. With a 2.51-point mAP50 gap and no repeated runs or confidence intervals, the difference could easily be within run-to-run noise. Furthermore, no experiment combines YOLOv5, YOLOv8, and DETR with equal WBF weights; without that ablation, any observed gain cannot be attributed to DE optimization rather than to ensembling. The claim of 'significantly outperforms' is therefore unsupported by the reported protocol, though the flaw is empirical and fixable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEIHDL, an ensemble object detector for pre-made dish ingredients that combines YOLOv5, YOLOv8, and DETR using weighted boxes fusion (WBF) with integration weights optimized by differential evolution (DE). Section III presents the three base models, the DE operations (initialization, mutation, crossover, selection), and WBF, including pseudo-code and complexity analysis. Section IV reports experiments on a Dish Ingredients dataset of 2,200 images and 11 classes, comparing DEIHDL with six detectors. DEIHDL achieves mAP50 of 90.92% and mAP50-95 of 72.25%, outperforming the best single model YOLOX (88.41% mAP50). Section IV.C analyzes the influence of population size NP and generations G. The authors claim significant improvement and discuss limitations and future work.","tokens_in":12614,"tokens_out":4510,"duration_ms":39344,"significance":"If the reported gain is real, the paper offers a straightforward, practical recipe for improving detection accuracy in a niche domain by combining off-the-shelf detectors with a standard optimizer, which could be useful for food-industry applications. The manuscript provides algorithm-level details and complexity estimates, and it is commendable that the authors expose the DE hyperparameter influence. However, the scientific claim hinges entirely on the empirical comparison, and the current protocol does not yet establish that the DE-tuned WBF ensemble outperforms a simple equal-weight WBF ensemble or that the gain is statistically reliable. Thus the significance is conditional on the missing experiments.","major_comments":[{"comment":"The paper does not state whether the mAP values reported in Table V come from a held-out test set or from the same validation set used by DE for weight selection. Equation (13) explicitly defines the selection criterion as performance on the validation set, so if Table V is computed on that validation set, the reported superiority could be an artifact of selection. Please specify the exact train/validation/test split of the 2,200 images, and report the DEIHDL results on a test set that is never used for weight selection or hyperparameter choice.","section":"Section IV.A and Table V"},{"comment":"The comparison includes only the three base models plus three other single models; there is no ensemble baseline with equal weights. Since WBF itself typically improves mAP over individual models, the 90.92 versus 88.41 gap cannot be attributed to the DE-optimized weights without also reporting a WBF ensemble with equal (or hand-set) weights on the same three base models. Add this ablation to isolate the contribution of DE.","section":"Section IV.B, Table V"},{"comment":"All results are single-point estimates without variance, confidence intervals, or tests of significance. Given the 2.51-point gap between DEIHDL and YOLOX and the variability typical of object detection training, repeated runs (e.g., 3-5 seeds) and, if appropriate, a paired statistical test over the test images are needed to substantiate the claim that DEIHDL 'significantly outperforms' the base models.","section":"Section IV.B, Table V"},{"comment":"The hyperparameter analysis for NP and G is presented as influencing DEIHDL performance, but the manuscript does not specify whether these values were selected using the same validation set and whether the final Table V entry uses those tuned values. This creates a possible selection-on-validation bias. Please describe the hyperparameter selection protocol and, ideally, use an independent validation split for the DE and hyperparameter tuning.","section":"Section IV.C"}],"minor_comments":[{"comment":"Several equations are corrupted by incomplete or missing symbols (e.g., undefined \\lambda*, \\tau, \\alpha, and garbled subscripts in Eq. (10)); please regenerate them with a proper equation editor to make the method verifiable.","section":"Section III.A, Eqs. (1), (2), (3), (10)"},{"comment":"Tables I and II, referenced as the pseudo-code of DEIHDL and WBF, do not appear in the provided manuscript; please include them in the final version.","section":"Section III.C, Tables I and II"},{"comment":"The individual X_{i,g} is represented by three weights, but no constraints such as non-negativity or sum-to-one are stated; please describe the search space and any normalization applied within the WBF step.","section":"Equation (8), Section III.B"},{"comment":"The figures in Section IV.C do not state which metric is plotted on the y-axis; please specify whether it is mAP50, mAP50-95, or another metric.","section":"Figures 2-4"},{"comment":"References [2] and [30] are the same paper (Grab, Pay, and Eat); one duplicate should be removed.","section":"References [2] and [30]"},{"comment":"The conclusion states that the model is 'limited by data integrity' but gives no details; if the dataset has missing or noisy labels, please describe them and how they may affect the comparison.","section":"Section V, Conclusion"},{"comment":"The dataset 'Dish Ingredients' is not described beyond counts; at minimum, state class names, image distribution, and availability to enable reproducibility.","section":"Section IV.A, Dataset"}],"recommendation":"major_revision","confidential_remarks":"The ambiguity about the evaluation set is the central issue; if the authors can show that Table V reports results on an untouched test set and add an equal-weight WBF baseline, the main claim may become credible. Note that the reference list contains a large fraction of self-citations, which is not a reason to reject but may warrant editorial scrutiny regarding novelty presentation. The paper's novelty for a top-tier venue is modest, but it could be suitable for an applied journal after the empirical gaps are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a competent but routine ensemble paper whose reported advantage is not yet demonstrated. The method—DE-tuned weights for weighted boxes fusion over YOLOv5, YOLOv8, and DETR—is a specific combination that I don't think appears elsewhere, and the authors apply it to a new food-ingredient dataset. That is the extent of the novelty, and it is enough for an applied venue if the experiments are solid. The experiments are not yet solid.\n\nWhat the paper does well: the related work on food recognition is relevant and reasonably thorough; the DE integration is described with enough formalism (init/mutation/crossover/selection, complexity) to be followed; and the conclusion honestly lists data integrity and hyperparameter tuning as limitations.\n\nThe soft spots are all in the evaluation protocol, and they are the load-bearing ones. Eq. (13) says the DE fitness is evaluated on the validation set, but the paper never states whether Table V reports a held-out test set or the same validation set. That is the single most important missing sentence. If the final mAP numbers come from validation, the 90.92 vs 88.41 gap is just selection bias. Even if a separate test set exists, the paper must say so explicitly and describe the split. Second, there is no equal-weight WBF baseline. Without it, you cannot attribute the gain to DE weight optimization rather than to ensembling per se. Third, there are no repeated runs or error bars. A 2.51 mAP point gap can easily be within run-to-run noise for deep detectors. Fourth, the hyperparameter analysis (NP, G) is presumably on the same validation set, which adds another layer of selection. Finally, the pseudo-code tables (I and II) appear empty in the manuscript, so the exact algorithm steps are missing.\n\nThese are all fixable, and the underlying idea is not absurd. But as submitted, the claim of significant improvement is unsupported. I would send this to peer review because the combination is new and the domain is practical, but I would expect the reviewers to demand a proper validation/test split, an equal-weight WBF ablation, and variance estimates across runs. The audience is practitioners in food computing; researchers in ensemble methods will find little here. I would not cite the current result until the protocol is fixed.","headline":"Reasonable ensemble idea, but the reported evaluation does not support the headline gain; the paper needs a clear test/validation separation, an equal-weight WBF baseline, and error bars.","tokens_in":13089,"tokens_out":4223,"would_cite":false,"duration_ms":39451,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that combining three diverse object detectors with differential evolution–tuned weights and weighted boxes fusion yields higher accuracy on pre-made dish images than any single detector alone, reaching 90.92 mAP50 against…","keywords":["object detection","ensemble learning","differential evolution","weighted boxes fusion","pre-made dishes","food detection","YOLO","DETR"],"falsifier":"Run the same DE weight search on the test set (or perform repeated cross-validation) and compare against uniform average weights of the three base models; if the mAP advantage over the uniform average shrinks to near zero, the claim that DE-tuned weights are the source of the gain is falsified. Alternatively, reshuffle the dataset into a different train/validation/test split and check whether DEIHDL still beats YOLOX by a comparable margin.","tokens_in":12192,"feed_emoji":"🥘","tokens_out":4423,"duration_ms":37914,"temperature":0.7,"pith_summary":"The paper claims that no single object detector works well enough on pre-made dishes, where ingredients overlap, look alike, and appear in poor lighting. Its proposed DEIHDL model combines three diverse detectors—YOLOv5, YOLOv8, and DETR—by giving each a weight found through differential evolution and merging their boxes with weighted boxes fusion. On a real dataset of 2,200 dish images across 11 ingredient categories, the ensemble reaches 90.92 mAP50, above the best single model YOLOX at 88.41. The authors argue this shows ensemble integration is a practical way to raise detection accuracy in complex food scenes.","feed_headline":"Ensemble of three detectors lifts pre-made dish detection to 90.92 mAP","feed_subtitle":"Differential evolution picks fusion weights for YOLOv5, YOLOv8, and DETR, beating the best single model.","key_machinery":"The mechanism is a three-way ensemble whose fusion weights are tuned by differential evolution. Each individual in the population is a triple of weights for the three detectors; mutation (DE/rand/1 with an adaptive scaling factor), arithmetic crossover, and selection on validation performance drive the search. Weighted boxes fusion then merges the three models' predicted boxes, assigning scores to candidate boxes rather than discarding overlapping detections. The load-bearing identity is that the combined mAP is higher than any single model's because the weight search is guided by the metric being optimized.","core_discovery":"The central claim is that an ensemble of heterogeneous detectors, fused by weights optimized with differential evolution and then merged by weighted boxes fusion, can outperform every single detector it is built from. The authors build three base models—YOLOv5, a single-stage CNN detector; YOLOv8, an anchor-free single-stage detector; and DETR, a transformer-based detector—to capture different inductive biases. Differential evolution searches for the three fusion weights by evaluating each candidate weight set's performance on a validation set according to the weighted boxes fusion result. The final DEIHDL model reports mAP50 of 90.92 and mAP50-95 of 72.25, versus 88.41 and 68.27 for the best single model (YOLOX). The paper's point is that this integration, not any single architecture, is what handles the overlapping-occlusion and low-light difficulties of pre-made dish scenes.","pith_inferences":["The same ensemble recipe could be tried on other occlusion-heavy detection tasks, such as medical image analysis or warehouse picking, where a single model struggles.","A natural test would be to compare DE-tuned weights against simple uniform averaging of the three base models' WBF outputs; if uniform averaging matches the reported mAP, the differential evolution step may be adding little.","The reported gain is on a single dataset of 2,200 images; evaluating on larger public food datasets or cross-domain shifts would show whether the advantage generalizes.","Since the hyperparameter analysis varies one parameter at a time, an automatic joint tuning of population size and generations could change the optimal settings."],"forward_implications":["DEIHDL outperforms each of its three base models on both mAP50 and mAP50-95, so the ensemble gain is consistent across metrics.","The differential evolution search converges over generations, showing the fusion weights stabilize rather than wander.","The best population size is small (5 to 15 individuals) and the best generation count is 40, indicating the weight search is computationally cheap.","The weighted boxes fusion step means confidence scores are recomputed from the ensemble, which the paper argues reduces false positives from overlapping ingredient boxes."],"supporting_citations":[{"why":"YOLOv5 is one of the three base detectors, contributing a classical single-stage CNN approach.","marker":"[10]"},{"why":"YOLOv8 is the anchor-free single-stage base detector, adding diversity through its decoupled head.","marker":"[11]"},{"why":"DETR is the transformer-based base detector, providing global-context modeling that the CNN models lack.","marker":"[13]"},{"why":"Weighted boxes fusion is the strategy that merges the base models' predictions, and the paper's selection step evaluates candidate weights through this fusion.","marker":"[23]"},{"why":"The scaling factor local search (SFLSDE) method supplies the adaptive scaling factor used in the DE mutation operator.","marker":"[20]"},{"why":"YOLOX is the strongest single-model baseline, and the paper's claim of superiority is measured against it.","marker":"[7]"},{"why":"Prior work using hierarchical DE in model training motivates the choice of differential evolution for weight optimization.","marker":"[8]"}],"fun_headline_variants":["Evolved ensemble fuses YOLOv5, YOLOv8, DETR for dish detection","Differential evolution unlocks 90.92 mAP on pre-made dish recognition","Three-detector blend beats every single model on complex dish scenes","Weighted fusion via evolution sharpens pre-made dish detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported mAP gain assumes the test set was not used, directly or indirectly, to choose the differential evolution weights or the hyperparameters; if the same images influenced both the weight search and the final score, the gain could be an artifact of overfitting.","fun_headline_variants_meta":{"raw":{"variants":["Evolved ensemble fuses YOLOv5, YOLOv8, DETR for dish detection","Differential evolution unlocks 90.92 mAP on pre-made dish recognition","Three-detector blend beats every single model on complex dish scenes","Weighted fusion via evolution sharpens pre-made dish detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1329,"prompt_tokens":1003,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":244}},"tokens_in":619,"tokens_out":326,"duration_ms":4136,"temperature":1.0,"reasoning_tokens":244,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:22:48.821609+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same DE weight search on the test set (or perform repeated cross-validation) and compare against uniform average weights of the three base models; if the mAP advantage over the uniform average shrinks to near zero, the claim that DE-tuned weights are the source of the gain is falsified. Alternatively, reshuffle the dataset into a different train/validation/test split and check whether DEIHDL still beats YOLOX by a comparable margin.","supporting_citations":[{"cited_title":"Yolov5 release v7.0","cited_arxiv_id":null,"evidence_quote":"YOLOv5 is one of the three base detectors, contributing a classical single-stage CNN approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"YOLOv8 is the anchor-free single-stage base detector, adding diversity through its decoupled head."},{"cited_title":"Scale factor local search in differential evolution,","cited_arxiv_id":null,"evidence_quote":"The scaling factor local search (SFLSDE) method supplies the adaptive scaling factor used in the DE mutation operator."}],"review_version":1}