{"id":"d8a0a8b6-9cec-4cb7-96ad-ca0f5b1d14f5","arxiv_id":"2411.13186","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adapting the number of aggregated LiDAR frames per object, based on speed and density, improves 3D detection over fixed multi-frame aggregation on Waymo.","lead":"This paper proposes VADet, which gives each detected object its own number of past LiDAR frames to merge, chosen from the object's speed and point density. On the Waymo benchmark it reports higher detection accuracy than fixed-frame aggregation, using ordinary single-stage detectors.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3's VADet-vs-fixed comparison is confounded: VADet receives an extra fine-tuning epoch, so the reported gains may reflect more training rather than variable aggregation.","rationale":"The reader's weakest assumption concerns failure when an object is missed in the previous frame, which is a real limitation of the mechanism. However, the single most load-bearing threat to the central empirical claim is the missing control for the extra fine-tuning epoch. Table 3 is the primary evidence for 'VADet consistently exceeds fixed aggregation,' and every VADet row is produced after an additional epoch of training on top of the baseline used for all fixed-aggregation columns. This is a textbook confound: the comparison conflates the aggregation policy with a training-compute difference. The paper even acknowledges that RAT-based training requires more epochs to reach its reported performance, so the direction and magnitude of the confound are plausible. The previous-detection failure mode, by contrast, is a limitation that would only reduce gains for missed objects; it does not undermine the comparison structure. The proposed control experiment is inexpensive relative to the original training budget and would settle whether variable aggregation itself is responsible for the gains. Given this unresolved confound, the reader's CONDITIONAL verdict remains appropriate, but the condition should explicitly include this control. I disagree with the reader's choice of weakest assumption because a confounded comparison is more fundamental than an unanalyzed failure mode: if the control shows no gain, the mechanism's benefit is unestablished regardless of how well it handles previous-frame misses.","tokens_in":12813,"tokens_out":6661,"duration_ms":71500,"concrete_test":"Add a control experiment for each architecture: take the RAT-pretrained baseline and apply the exact VADet fine-tuning protocol (one epoch, cosine LR, mini-sequence shuffling, cached offline predictions), but set eta to a fixed constant for all objects, e.g., k=3 and k=best-fixed-frame-count. If the fixed-k fine-tuned controls match VADet's AP/APH within 0.1-0.2 points, the variable aggregation contributes little beyond extra training; if VADet still exceeds both controls by the reported margins, the confound is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3.2 states that VADet models are initialized from the RAT baselines and fine-tuned for an additional epoch with cosine decay, while Table 3's fixed-aggregation columns report the baselines after their initial 20-epoch RAT training with no such fine-tune. The treatment therefore differs by both the aggregation policy and an extra epoch of training. This matters because Section 3.1 already shows that a single RAT model needs extra epochs to match separately trained fixed-configuration models. The +0.5, +0.7, and +1.3 AP margins in Table 3 could plausibly come from the fine-tuning epoch alone. The paper does not report a control where the RAT baseline is fine-tuned for one epoch under fixed k-frame aggregation using the same mini-sequence shuffling, cosine schedule, and cached offline predictions. Without that control, the central claim that VADet consistently exceeds fixed aggregation is not isolated from additional compute. A secondary compounding issue is that VADet training uses cached offline predictions from a 3-frame baseline, while inference uses online predictions from the VADet model itself; this train/inference mismatch is not analyzed and further complicates attribution of the reported gains.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VADet, an input-level per-object aggregation scheme for multi-frame LiDAR 3D detection. It first trains a detector with randomly varying numbers of aggregated frames (RAT), then constructs a lookup table η that maps an object's estimated speed and point density to the number of past frames to aggregate, and finally aggregates per-object regions centered on previous-frame detections propagated by a constant-velocity model. VADet is applied to CenterPoint, VoxelNeXt, and a pillar variant of DSVT and evaluated on Waymo vehicle detection, reporting consistent gains over fixed-aggregation baselines and competitive or superior results against recent multi-frame detectors.","tokens_in":13083,"tokens_out":7230,"duration_ms":69922,"significance":"If the reported gains are robust, VADet is a valuable result: it shows that input-level adaptive aggregation can capture much of the benefit of more complex feature-level temporal fusion at low latency and without architecture-specific changes. The paper's strengths include the RAT training scheme (which makes broad frame-count studies feasible), the well-structured speed/density breakdown analysis in Section 5.3, and the explicit discussion of subset-evaluation metrics in Section 4.2. The central comparison in Table 3, however, is confounded by an extra fine-tuning epoch, and the policy η is fit to the training split with several unspecified hyperparameters; these issues need to be addressed before the central claim can be accepted.","major_comments":[{"comment":"The VADet-versus-fixed comparison in Table 3 is confounded. VADet models are initialized from the RAT baselines and fine-tuned for an additional epoch with cosine decay and mini-sequence shuffling (Sec. 4.3.2), while the fixed-aggregation columns report the RAT baselines without this extra training. Section 3.1 and Table 1 show that additional training can change AP by amounts comparable to the reported margins (e.g., RAT versus separate training differs by up to 0.7 AP). The paper does not report a control in which a fixed k-frame aggregation model receives the same extra fine-tuning epoch; without such a control, the +0.5, +0.7, and +1.3 AP margins in Table 3 cannot be attributed to variable aggregation. The additional mismatch between cached offline predictions used in training and online VADet predictions at inference (Sec. 4.3.2) further complicates attribution. No error bars are reported anywhere, which is especially important because the headline margins are small.","section":"§4.3.2, Table 3"},{"comment":"The lookup table η is fit to the training split: for each speed-density bin, the frame count maximizing AP on the training split is selected over 3–16 frames. This is a high-dimensional model-selection procedure with no multiplicity correction, no reported stability of the selected frame counts, and no sensitivity analysis with respect to the bin thresholds chosen from training statistics. The validation and test results are therefore not an evaluation of an a priori policy, and the claim that η gives the 'empirically best' number of frames needs support in the form of cross-validated η construction or a report of how much the selected frame counts and final AP change under perturbation of the thresholds or the training split.","section":"§3.2.1, §4.3.3"},{"comment":"The per-object aggregation region is computed from the previous frame's detection b_{τ−1} and its velocity estimate v_{τ−1}; if an object is not detected in the previous frame, no extra frames are aggregated for it, so the claimed benefits for sparse or occluded objects (e.g., the qualitative example in Fig. 2) presuppose a prior detection and an accurate velocity. The paper does not analyze this failure mode, nor the sensitivity of the results to errors in v_{τ−1}. In addition, the enlargement factor σ in Eqs. (5)–(7) is introduced but its value is never specified and no ablation or sensitivity analysis is provided; this is a reproducibility issue for a hyperparameter that directly controls the aggregation region.","section":"§3.2.2, Algorithm 1, Eqs. (3)–(8)"}],"minor_comments":[{"comment":"The speed threshold list contains a likely typo: 81.6 m/s should almost certainly be 8.16 m/s; as printed, the thresholds are not monotonic.","section":"§4.3.3"},{"comment":"The set-builder notation in the line defining the filtered box is confusing, as it appears to define a set in terms of itself; please rewrite it to indicate that the box is kept when η(b_{τ−1}) > i.","section":"Algorithm 1"},{"comment":"The DSVT-P variant used in the experiments is not the original DSVT-pillar: it uses static voxelization and reduced channel widths. This should be stated where Table 4 is discussed, since it affects the interpretation of the comparison to DSVT-pillar.","section":"§4.3.1, Table 4"},{"comment":"The statement 'we do not anticipate any obvious limitations' is at odds with the acknowledged failure mode in the same section and with the method's dependence on previous-frame detections; please revise.","section":"§6"},{"comment":"The RAT-versus-separate-training comparison is reported only for VoxelNeXt and without variance estimates; please clarify whether the equivalence claim is established for the other two architectures.","section":"§3.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the experiments are broad, but the headline comparison in Table 3 needs a proper control (fixed aggregation with the same extra fine-tuning epoch), and the construction of η needs a sensitivity or cross-validation analysis. These are fixable within the scope of the manuscript, so I would not recommend rejection; I would, however, ask the authors to address the confound and the unspecified σ before the paper is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain take: this paper has a genuine idea—per-object variable aggregation decided by speed and point density—and it shows carefully that different object types have different optimal frame counts. But the central comparison in Table 3 is confounded by an extra fine-tuning epoch, and the η lookup table is fitted on the training split. The core claim is plausible but not yet isolated.\n\nWhat is new and good: RAT training lets one model handle many frame counts, which is useful. The point-density trade-off is a real observation and the breakdown tables are informative. Applying the method to three backbones is the right way to show generality. The 50 ms latency claim is concrete and the qualitative examples support the intended mechanism.\n\nSoft spots, in proportion:\n\n1. The stress-test note is correct. Table 3 compares VADet (RAT + one extra fine-tuning epoch with cosine decay, mini-sequence shuffling, cached predictions) against RAT baselines that did not get that epoch. Table 1 already shows RAT with more epochs can beat separately trained fixed models. So the +0.5 to +1.3 AP margins could mostly be the extra training. The paper needs a control: fine-tune the RAT baseline for one epoch under fixed k-frame aggregation with the same schedule and report that.\n\n2. η is built by evaluating the RAT model on the training split and picking the best frame count per (speed, density) bin. That is a legitimate design choice, but it means the per-bin allocation is selected, not predicted. The validation numbers are a selected configuration. The comparison would be fairer if fixed-aggregation baselines also got the benefit of choosing the frame count on the training split, with the same fine-tuning.\n\n3. σ is never specified, no error bars are given anywhere, and the code is not released. Individually these are minor, but together they hurt reproducibility.\n\n4. The method depends on the previous frame's detection b_{τ−1}. If an object was missed there, it gets no aggregation benefit. The paper does not analyze this failure mode. Minor for the overall claim, but relevant for sparse or occluded objects.\n\nVerdict: the idea is worth pursuing and the paper deserves referee time, but not acceptance as is. I would send it to review with a request to fix the control experiment and to report the fine-tuned fixed-aggregation baseline.","headline":"The per-object variable aggregation idea is real and worth a look, but the main empirical claim is undersupported until the extra-epoch confound in Table 3 is controlled.","tokens_in":13605,"tokens_out":2469,"would_cite":false,"duration_ms":26975,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that per-object, variable numbers of aggregated LiDAR frames—chosen by speed and point density—outperform fixed whole-scene aggregation and can beat far more complex multi-frame detectors.","keywords":["3D object detection","LiDAR point clouds","multi-frame aggregation","variable aggregation","random aggregation training","speed and point density trade-off","Waymo Open Dataset"],"falsifier":"Evaluate VADet on Waymo validation separated by whether the object was detected in the previous frame and by the error of the velocity estimate; if objects missed in the previous frame show no AP gain over 3-frame fixed aggregation, the benefit is conditional on prior detections. A companion experiment feeding ground-truth previous boxes instead of detector boxes would show how much of the gain depends on the detector's own upstream recall.","tokens_in":12589,"feed_emoji":"🚗","tokens_out":6419,"duration_ms":57682,"temperature":0.7,"pith_summary":"The paper claims that the standard practice of aggregating a fixed number of LiDAR frames for the entire scene is fundamentally limited, because the optimal frame count differs by object type: stationary objects benefit from many frames, fast objects are hurt by over-aggregation, and sparse objects behave differently from dense ones. VADet instead aggregates each detected object with its own number of frames, chosen by a lookup table that maps the object's estimated speed and point density to the frame count that maximizes average precision. The authors show this per-object input-level modification works across three single-stage detectors and, with a VoxelNeXt backbone, reaches 76.1 L2 APH on Waymo validation and 79.4 on the test split, beating the far more complex two-stage MPPNet and slightly exceeding the camera-LiDAR fusion method LoGoNet. A careful reader would care because it suggests input construction alone, not architectural attention, can capture most of the multi-frame gain at a fraction of the latency.","feed_headline":"Per-object frame counts beat fixed LiDAR aggregation","feed_subtitle":"VADet gives each object the frames it needs, beating complex multi-frame detectors on Waymo with low latency.","key_machinery":"The central mechanism is the aggregation-count function $\\eta(b_{\\tau-1})$, which maps a previous-frame detection's estimated speed $\\|v_{\\tau-1}\\|$ and point density $\\rho_{\\tau-1}$ (points per unit surface area of the predicted box) to an integer frame count. The function is learned empirically: after RAT training, the training set is split into speed and density bins, and for each bin the frame count with the highest average precision becomes the lookup value. At inference, Algorithm 1 computes the object's current position with a constant-velocity model, enlarges the box by speed and frame count to capture motion smudges, crops the ego-corrected past frames inside that region, and concatenates the object points while leaving background points at a fixed 3-frame aggregation. The same mechanism also introduces a reweighted subset precision metric so that per-speed and per-density AP numbers are comparable across subsets of different sizes.","core_discovery":"On the paper's own terms, the discovery is that fixed aggregation has a per-object performance trade-off that can be removed by variable per-object aggregation. Using Random Aggregation Training (RAT), a single detector is trained on scenes with randomly varying numbers of aggregated frames, which lets the authors cheaply measure how average precision varies with frame count for objects binned by speed and point density. The resulting function $\\eta$ is a piecewise lookup table that assigns each object a frame count, and the input is built by propagating the object's previous detection forward with a constant-velocity model and cropping the ego-corrected past point clouds inside an enlarged box. Applied to CenterPoint, VoxelNeXt, and DSVT-P, VADet exceeds every fixed frame count for the same architecture, and the best variant, VADet-VoxelNeXt, outperforms the attention-based MPPNet by a large margin while adding about 50 ms of overhead versus MPPNet's 900–2500 ms.","pith_inferences":["Because $\\eta$ is a static lookup table, a learned regressor from raw object point clouds to frame counts could capture interactions between speed and density that the hand-chosen bins miss; the paper names this as a possible extension.","The background points are kept at fixed 3-frame aggregation, so the variable aggregation principle could be pushed further by making background density adaptive too, which might reduce computation and confusion in dense scenes.","The reweighted subset precision metric proposed for speed and density breakdowns could be applied to any future per-group evaluation on Waymo, since the official metric's treatment of unmatched false positives makes subset sizes incomparable.","Per-object variable aggregation should transfer to tracking and motion forecasting pipelines, because those tasks already require per-object temporal association; a testable extension is whether the same $\\eta$ improves downstream tracking accuracy on Waymo."],"forward_implications":["For each of the three tested architectures, VADet's L2 vehicle AP exceeds the best fixed aggregation count on the Waymo validation split, so the gain is not tied to one backbone.","VADet-VoxelNeXt achieves 76.1 L2 APH on validation and 79.4 on the test split, surpassing MPPNet and slightly exceeding LoGoNet despite using LiDAR only and no test-time augmentation or ensemble.","The overhead of VADet's input aggregation is about 50 ms, roughly 20 to 50 times less than the 900–2500 ms second stage of MPPNet, making the gain available for real-time use.","RAT lets one detector evaluate and serve many frame counts, so it can be a cheap pretraining strategy and a tool for studying input-level temporal fusion before designing architecture-level attention."],"supporting_citations":[{"why":"Establishes fixed aggregation and the velocity regression head that VADet reuses for per-object speed estimation.","marker":"[1]"},{"why":"Provides the attention-based multi-frame SOTA MPPNet, the main performance baseline and a source of the motion-distortion trade-off.","marker":"[2]"},{"why":"Supplies VoxelNeXt, the fully sparse backbone used in the best VADet configuration.","marker":"[3]"},{"why":"Provides the offline-prediction training technique and the FSD++ SOTA baseline used for comparison.","marker":"[5]"},{"why":"Supplies the Waymo Open Dataset, its metrics, and all training, validation, and test splits used in the experiments.","marker":"[16]"},{"why":"Supplies DSVT, the transformer-based detector that VADet is applied to and compared against.","marker":"[18]"},{"why":"Prior work attributing aggregation degradation to object motion; its 3D-MAN results serve as a comparison baseline.","marker":"[19]"},{"why":"Supplies CenterPoint, the dense voxel-based detector that VADet is applied to and compared against.","marker":"[21]"},{"why":"Supplies CenterFormer, a SOTA multi-frame detector used as a comparison baseline.","marker":"[22]"}],"fun_headline_variants":["Adaptive per-object frame counts beat fixed LiDAR aggregation","VADet gives each object its own frame count for LiDAR 3D","Variable aggregation for LiDAR: better detection with less","Per-object frame selection outperforms fixed LiDAR multi-frame"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme is driven by the previous frame's detection: the region to aggregate is computed from that detection and its velocity, so an object that was missed in the previous frame, or has a wrong velocity estimate, receives no extra frames or is aggregated in the wrong place, and the paper does not analyze how often that happens.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive per-object frame counts beat fixed LiDAR aggregation","VADet gives each object its own frame count for LiDAR 3D","Variable aggregation for LiDAR: better detection with less","Per-object frame selection outperforms fixed LiDAR multi-frame"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000257,"raw_usage":{"total_tokens":1548,"prompt_tokens":883,"completion_tokens":665,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":594}},"tokens_in":499,"tokens_out":665,"duration_ms":6624,"temperature":1.0,"reasoning_tokens":594,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:44:24.780721+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate VADet on Waymo validation separated by whether the object was detected in the previous frame and by the error of the velocity estimate; if objects missed in the previous frame show no AP gain over 3-frame fixed aggregation, the benefit is conditional on prior detections. A companion experiment feeding ground-truth previous boxes instead of detector boxes would show how much of the gain depends on the detector's own upstream recall.","supporting_citations":[{"cited_title":"Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krish- nan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom","cited_arxiv_id":null,"evidence_quote":"Establishes fixed aggregation and the velocity regression head that VADet reuses for per-object speed estimation."},{"cited_title":"MPPNet: Multi-frame feature intertwining with proxy points for 3D temporal object detection","cited_arxiv_id":null,"evidence_quote":"Provides the attention-based multi-frame SOTA MPPNet, the main performance baseline and a source of the motion-distortion trade-off."},{"cited_title":"V oxelNeXt: Fully sparse V oxelNet for 3D object detection and tracking","cited_arxiv_id":null,"evidence_quote":"Supplies VoxelNeXt, the fully sparse backbone used in the best VADet configuration."},{"cited_title":"Super sparse 3D object detection","cited_arxiv_id":null,"evidence_quote":"Provides the offline-prediction training technique and the FSD++ SOTA baseline used for comparison."},{"cited_title":"Scalability in perception for autonomous driving: Waymo Open Dataset","cited_arxiv_id":null,"evidence_quote":"Supplies the Waymo Open Dataset, its metrics, and all training, validation, and test splits used in the experiments."},{"cited_title":"DSVT: Dynamic sparse voxel transformer with ro- tated sets","cited_arxiv_id":null,"evidence_quote":"Supplies DSVT, the transformer-based detector that VADet is applied to and compared against."},{"cited_title":"3D-MAN: 3D multi-frame attention network for object detection","cited_arxiv_id":null,"evidence_quote":"Prior work attributing aggregation degradation to object motion; its 3D-MAN results serve as a comparison baseline."},{"cited_title":"Center-based 3D object detection and tracking","cited_arxiv_id":null,"evidence_quote":"Supplies CenterPoint, the dense voxel-based detector that VADet is applied to and compared against."},{"cited_title":"CenterFormer: Center- based transformer for 3D object detection","cited_arxiv_id":null,"evidence_quote":"Supplies CenterFormer, a SOTA multi-frame detector used as a comparison baseline."}],"review_version":1}