{"id":"96a746b9-5f11-4998-ad59-7accb067cb08","arxiv_id":"1908.08244","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CenterNet with an Hourglass-104 backbone, high-resolution multi-scale testing, and horizontal flip achieves competitive VisDrone2019 aerial detection results, finishing 7th on images and 5th on videos.","lead":"The paper benchmarks CenterNet on the VisDrone aerial dataset using different backbones and test-time augmentation. It reports 7th place on the image track and 5th place on the video track of the VisDrone2019 challenge, with the Hourglass-104 backbone performing best.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Class-wise APs in Table 6 do not average to the reported overall mAPs in Tables 4/5, so the headline benchmark numbers are internally inconsistent.","rationale":"The reader's weakest assumption concerned validation-set representativeness and possible overfitting of test-time augmentation, but the leaderboard results are externally generated and would not be invalidated by a non-representative validation split. A more immediate and concrete problem is that the paper's own tables contradict each other arithmetically. The central claim is the quantitative performance on VisDrone2019, and that claim is presented through Tables 4, 5, and 6. If the class-wise APs in Table 6 are averaged, they do not reproduce the reported overall APs in Tables 4 and 5. Because the discrepancies are in opposite directions, no simple methodological adjustment explains them; at least one set of numbers is incorrect. This internal inconsistency is load-bearing because it makes the headline results unreliable without external verification. It does not require assuming bad intent; it could be a transcription error or an undocumented evaluation detail, but either way the paper as written does not support its own numbers. The reader's conditional verdict remains appropriate, so no verdict change is needed, but the reason for caution should be sharper.","tokens_in":5030,"tokens_out":7494,"duration_ms":73232,"concrete_test":"Recompute the mean of the ten per-class AP values in Table 6 for each track and compare with the AP values in Tables 4 and 5. If the means differ by more than 0.1 point from the reported overall APs, check the official VisDrone2019 leaderboard and evaluation code for the exact averaging procedure. If the official leaderboard gives 27.83/21.58, Table 6 is erroneous; if it gives 28.11/20.97, Tables 4/5 are erroneous. The paper must correct or explicitly explain the discrepancy before the headline benchmark numbers can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper reports in Table 4 an overall Track1 AP of 27.83 and in Table 5 an overall Track2 AP of 21.58. Table 6 lists per-class AP values for the same runs. Averaging the ten Track1 class values (31.05, 12.99, 9.08, 51.92, 38.33, 31.14, 24.24, 21.06, 40.94, 20.35) yields 28.11, not 27.83. Averaging the ten Track2 values (27.86, 6.59, 12.47, 33.92, 29.91, 40.55, 13.99, 12.91, 24.48, 6.98) yields 20.97, not 21.58. The discrepancies go in opposite directions, so a single global explanation like maxDets or ignore-region handling cannot reconcile them. Since the central claim is the quantitative leaderboard result, this internal inconsistency means at least one of Tables 4/5 or Table 6 is not correctly transcribed or was not computed with the stated metric. This directly undermines the reliability of the reported numbers that the paper's conclusion rests on.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates CenterNet, a keypoint-based single-stage detector, on the VisDrone2019 aerial object detection benchmark. It compares three backbones (ResNet-18, DLA-34, Hourglass-104) at 512x512 input and reports validation mAP, then studies test-time augmentation (horizontal flip) and multi-scale inference at different input resolutions. Based on these experiments, the authors select a 2048x2048 input with scales 0.5-1.5 and horizontal flip, and report 58.03 mAP on the validation set. On the challenge test set, they report 7th place on Track 1 with overall mAP 27.83 and 5th place on Track 2 with mAP 21.58, using the same model without fine-tuning. The paper concludes that CenterNet is competitive for small-object detection in aerial imagery.","tokens_in":5301,"tokens_out":5918,"duration_ms":51910,"significance":"If the reported leaderboard results are accurate, this paper provides useful empirical evidence that CenterNet, originally designed for general object detection, can rank competitively on a challenging aerial benchmark with dense small objects. The external evaluation by the challenge organizers is a strength, as it reduces concerns about self-reported test-set numbers. However, the internal inconsistency between the class-wise APs in Table 6 and the overall APs in Tables 4/5, together with the conflicting validation set sizes in Table 1 and Fig. 2, substantially undermine confidence in the reported quantitative claims. No code or detailed evaluation protocol is provided, which limits reproducibility. The scientific value is therefore contingent on correcting and verifying the reported numbers.","major_comments":[{"comment":"The class-wise APs in Table 6 do not average to the overall APs reported for the same runs. Averaging the ten Track 1 values (31.05, 12.99, 9.08, 51.92, 38.33, 31.14, 24.24, 21.06, 40.94, 20.35) gives 28.11, not 27.83 as reported in Table 4; averaging the ten Track 2 values (27.86, 6.59, 12.47, 33.92, 29.91, 40.55, 13.99, 12.91, 24.48, 6.98) gives 20.97, not 21.58 as reported in Table 5. The discrepancies are in opposite directions, so a single systematic explanation (e.g., maxDets or ignore-region handling) cannot reconcile them. Since Tables 4 and 5 are the paper's headline quantitative claims, at least one set of numbers is incorrect or computed under a different metric than stated. Please provide the exact evaluation protocol and correct the tables accordingly.","section":"§4.5, Tables 4, 5, and 6"},{"comment":"The size of the validation set is reported as 548 images in Table 1 and §4.2 but as 528 images in the caption of Fig. 2. This discrepancy affects the reproducibility of the validation mAP numbers in Tables 1-3, which in turn motivate the choice of input resolution, scale range, and flip augmentation used for the leaderboard submission. Please state the exact number of validation images used in each experiment and explain the source of the difference.","section":"§3.2.1, Table 1, Fig. 2"}],"minor_comments":[{"comment":"The sentence \"We use the VisDrone2019 DET dataset for object detection in videos\" appears to be a typo; the DET benchmark is for image detection, while the VID dataset is for video detection. Please correct.","section":"§3.1"},{"comment":"The sentence \"The performance of the evaluated on a variety of data-set\" is grammatically incomplete and should be revised.","section":"§4.2"},{"comment":"The captions state that red entries denote the best performance, but the tables are printed in black in the PDF. Please add color or otherwise indicate the best entries in the text.","section":"Tables 4 and 5"},{"comment":"The claim that scaling beyond 2048 (e.g., to 4096) does not improve performance is not supported by any reported experiment in Table 3. Please include the result or qualify the statement as an observation from unreported experiments.","section":"§4.4"},{"comment":"The gain from horizontal flip is described as \"1 %\" in the text; please clarify whether this is an absolute or relative improvement (the table shows 31.97 to 32.99).","section":"§4.3, Table 2"},{"comment":"The paper does not describe how multi-scale predictions and the flipped predictions are fused (e.g., NMS across scales or averaging of heatmaps). Please add a sentence describing the inference post-processing.","section":"§4.4"},{"comment":"The comparison to YOLOv3 uses a different input resolution (416x416 for YOLOv3 vs 512x512 for CenterNet), which confounds the claimed advantage over YOLOv3. Please report YOLOv3 at 512x512 or explicitly state that the comparison is not controlled for input resolution.","section":"§4.2, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The main reason for major revision is the valid internal inconsistency between Table 6 and Tables 4/5 identified by the skeptical reader; this directly affects the reliability of the headline leaderboard numbers. The validation set size discrepancy (548 vs 528) further weakens confidence in the experimental chain. The paper is a short challenge report, and the issues appear fixable by re-running the evaluation and correcting the tables, but as written the central quantitative claims are not internally consistent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's headline numbers don't survive a calculator. The class-wise APs in Table 6 average to 28.11 for Track 1 and 20.97 for Track 2, not the 27.83 and 21.58 reported in Tables 4 and 5. The errors go in opposite directions, so this isn't one rounding slip; at least one of those tables doesn't match the run it claims to describe. That's a real problem for a paper whose main contribution is a leaderboard number.\n\nCredit where it's due: this is a straightforward, honest parameter sweep of CenterNet on VisDrone2019. The authors try different backbones, input resolutions, multi-scale ranges, and test-time flip. The conclusion that high resolution plus multi-scale around 1x helps is plausible and consistent with what others see in aerial detection. The validation mAP progression (31.97 to 58.03) is internally consistent. And the leaderboard positions (7th on Track 1, 5th on Track 2) are externally documented, so the qualitative story isn't invented.\n\nThe soft spots beyond the arithmetic: no error bars, no description of the train/validation split, and the validation set size changes from 548 images in Table 1 to 528 in Figure 2 without comment. The YOLOv3 baseline is confounded because it runs at 416x416 while CenterNet gets 512. No code is released. These are typical for a challenge report, but together they mean the precise numbers should be treated as approximate.\n\nThe central qualitative claim — CenterNet is competitive on VisDrone with the right backbones and test-time settings — survives. But I wouldn't quote the exact mAP as gospel until the tables are reconciled.\n\nThis paper is for someone working on drone imagery who wants a quick comparison point. It's not a conceptual advance. It deserves a serious referee because the empirical question is real and the leaderboard results are verifiable, but the authors need to fix the inconsistency, clarify the split, and ideally ship code. I'd accept it for review with a requested major revision.","headline":"The paper's per-class APs don't average to its headline mAP, so the central numbers are internally inconsistent; still a plausible benchmark data point.","tokens_in":5785,"tokens_out":3187,"would_cite":false,"duration_ms":29258,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CenterNet, a keypoint-estimation detector that models each object as its bounding-box center, achieves 58.03 validation mAP on VisDrone2019 and places 7th on the image-detection track and 5th on the video-detection track.","keywords":["CenterNet","keypoint estimation","aerial object detection","VisDrone2019","test-time augmentation","multi-scale testing","small object detection","HourGlass-104"],"falsifier":"Compute the same training and inference pipeline on several held-out subsets of VisDrone2019 (for instance, the 528-image subset shown in the class-wise figure versus the 548-image table) and compare class-wise mAP; if the validation composition shifts the 58.03 result by more than the 1–2 points attributed to test-time augmentation, the reported gain is partly a property of the chosen sample rather than the method.","tokens_in":4865,"feed_emoji":"🚁","tokens_out":5279,"duration_ms":49927,"temperature":0.7,"pith_summary":"CenterNet, a single-stage detector that treats every object as the center point of its bounding box, can be made to work well on dense, small-object aerial imagery. The paper shows that with an HourGlass-104 backbone, training at 1024x1024, and inference at 2048x2048 with multi-scale testing plus horizontal flip, CenterNet reaches a validation mAP of 58.03 on VisDrone2019 and lands 7th on the image-detection track and 5th on the video-detection track without retraining. This matters because small, densely packed objects in drone images are exactly where anchor-based single-stage detectors such as YOLOv3 and SSD struggle, and CenterNet's keypoint formulation sidesteps exhaustive box proposals.","feed_headline":"CenterNet ranks 7th on the VisDrone drone challenge","feed_subtitle":"Modeling each object as one center point, with high-resolution multi-scale testing, reaches 58.03 validation mAP.","key_machinery":"The load-bearing object is CenterNet's keypoint-estimation head: a fully convolutional network predicts a heatmap of object center locations and regresses the bounding-box size and a small offset for each center; no anchor boxes or region proposals are enumerated. The HourGlass-104 backbone provides repeated down- and up-sampling that preserves spatial detail for small objects. The inference setup—resizing to 2048x2048 and averaging predictions over scales 0.5, 0.75, 1, 1.25, 1.5 and a horizontal flip—is what carries the final numbers.","core_discovery":"The central claim is that modeling an object as a single point—its bounding-box center—is sufficient for competitive detection in aerial imagery, provided the input resolution is high enough and inference uses the right scales. The authors report that HourGlass-104 beats ResNet-18 and DLA-34 backbones, that horizontal flip adds about one percentage point, and that at 2048x2048 the scale range 0.5–1.5 outperforms 1–2, yielding 58.03 validation mAP. On the VisDrone2019 challenge test set the same model reaches 27.83 AP on images and 21.58 AP on videos, and it is the top entry at 500 detections per image on the image track.","pith_inferences":["Taken together, the 512-vs-2048 results imply that effective object resolution, not raw input size, drives the gain; a tiling or cropping strategy on native-resolution imagery might capture most of the 58.03 result at lower memory cost.","The paper does not ablate backbone capacity at 2048 input; if the resolution jump is the dominant factor, a smaller backbone at 2048 could approach HourGlass-104 performance much faster.","For the video track, adding temporal linking or tube aggregation across frames could plausibly push the 21.58 video AP closer to the image-track 27.83, since the detector alone already transfers without retraining."],"forward_implications":["Keypoint-based detection is a viable alternative to anchor-based detectors for dense small-object scenes such as drone footage.","Input resolution and scale range matter more than backbone choice for aerial images: resolution is what lets small objects survive feature extraction.","Multi-scale test-time averaging plus horizontal flip is worth roughly 2–3 mAP points at 2048 resolution; removing the flip drops mAP by about one point.","The same image-trained model transfers to video frames without fine-tuning and still ranks 5th, so temporal redundancy is not needed for a competitive video result."],"supporting_citations":[{"why":"Supplies CenterNet itself: the objects-as-points keypoint-estimation formulation that the paper adapts to aerial imagery.","marker":"[9]"},{"why":"Provides the COCO pretrained weights used to initialize all CenterNet backbones before VisDrone training.","marker":"[5]"},{"why":"YOLOv3 is the one-stage anchor-based baseline whose 416x416 result is compared against CenterNet in the backbone evaluation.","marker":"[7]"},{"why":"Supplies the ImageNet normalization statistics used in preprocessing training and test images.","marker":"[1]"}],"fun_headline_variants":["CenterNet reaches 58.03 mAP on VisDrone using HourGlass-104","CenterNet tops 500-detection-per-image limit on VisDrone track","Single-point detection: CenterNet hits 58 mAP on VisDrone","HourGlass-104 beats ResNet and DLA for CenterNet on VisDrone","CenterNet ranks 7th, leads at 500 detections on VisDrone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline validation number rests on the assumption that the particular set of validation images used to pick the 2048-pixel, 0.5–1.5 scale settings is representative of the hidden challenge test set; the paper lists 548 validation images in one place and 528 in another, and reports no error bars.","fun_headline_variants_meta":{"raw":{"variants":["CenterNet reaches 58.03 mAP on VisDrone using HourGlass-104","CenterNet tops 500-detection-per-image limit on VisDrone track","Single-point detection: CenterNet hits 58 mAP on VisDrone","HourGlass-104 beats ResNet and DLA for CenterNet on VisDrone","CenterNet ranks 7th, leads at 500 detections on VisDrone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001063,"raw_usage":{"total_tokens":4368,"prompt_tokens":766,"completion_tokens":3602,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":382,"completion_tokens_details":{"reasoning_tokens":3494}},"tokens_in":382,"tokens_out":3602,"duration_ms":23194,"temperature":1.0,"reasoning_tokens":3494,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:44:55.377984+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the same training and inference pipeline on several held-out subsets of VisDrone2019 (for instance, the 528-image subset shown in the class-wise figure versus the 548-image table) and compare class-wise mAP; if the validation composition shifts the 58.03 result by more than the 1–2 points attributed to test-time augmentation, the reported gain is partly a property of the chosen sample rather than the method.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies CenterNet itself: the objects-as-points keypoint-estimation formulation that the paper adapts to aerial imagery."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the COCO pretrained weights used to initialize all CenterNet backbones before VisDrone training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"YOLOv3 is the one-stage anchor-based baseline whose 416x416 result is compared against CenterNet in the backbone evaluation."}],"review_version":1}