{"id":"581ee6ea-b39e-4f81-99b1-566f3fa1f61e","arxiv_id":"2509.05780","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"3DPillars is a two-stage pillar-based 3D detector using axis-decomposed 2D convolutions (SVFM) and a memory-augmented context-aware RoI head (S2CFM), achieving 29.6 Hz with strong results on small objects.","lead":"This paper introduces a two-stage 3D object detector that keeps the speed of pillar-based LiDAR methods while adding a refinement stage with scene-wide context. On KITTI, Waymo Open, and nuScenes it reports real-time inference with accuracy near slower voxel-based state-of-the-art detectors, especially for small objects.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SVFM's sparse-volume processing is underspecified; the claimed efficiency and parameter reduction hinge on an unverified implementation of the axis-decomposed 2D convolutions.","rationale":"The paper's headline claim has two components: accuracy approaching voxel-based two-stage detectors, and efficiency inherited from pillar-based pseudo-image processing. The accuracy component is supported by extensive experiments and ablations, and I see no internal inconsistency there. The efficiency component, however, rests on the SVFM construction, and the manuscript's description is ambiguous exactly at the point that determines cost. If 'applying 2D convolutions to each slide' means processing each z-slice (and similarly for other axes) with a shared 2D filter, the compute scales with the number of slices, making the reported 8.9 ms backbone runtime highly surprising given a 16-slice volume and three views. If instead the height dimension is collapsed into channels before the 2D convolution, the operation is not equivalent to the stated 1×k×k separable kernel, and its cost is not described. The paper also asserts a 45% parameter reduction that fails arithmetically for k=3. These are not cosmetic issues: a reader cannot verify the core efficiency claim without knowing the actual sparse/dense processing scheme. This matches the reader's weakest_assumption. I therefore agree with the CONDITIONAL verdict; the authors must clarify (and ideally release code for) the SVFM implementation before the efficiency claims can be taken at face value. The other issues (overstated 'first' claims, no error bars) are real but secondary to this load-bearing technical gap.","tokens_in":28616,"tokens_out":9990,"duration_ms":112221,"concrete_test":"Obtain the actual implementation (code release) or, failing that, reimplement SVFM from Section 3.1.2 on KITTI at the stated voxel resolution (W=70.4m, L=80m, H=4m; vW=vH=0.16m, vL=0.25m → 440×500×16 volume). Measure FLOPs, parameter count, and runtime for one SVFM block under both readings: (a) per-slice 2D convolutions with shared weights, and (b) height-collapsed 2D convolutions. Compare the backbone runtime against the reported 8.9 ms and the total runtime against 29.6 Hz. Also verify the parameter count: for k=3, three kernels of shapes 1×3×3, 3×1×3, and 3×3×1 have 27C parameters, equal to a 3D conv's 27C; if the claimed 45% reduction exists, it must come from a different architecture (e.g., depthwise/grouped convolutions) that the authors should specify and that should be used in the FLOPs measurement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim—that 3DPillars extracts voxel-based features without 3D convolutions while retaining real-time speed—rests entirely on how SVFM processes the sparse 3D feature volume. Section 3.1.2 first says SVFM 'applies 2D convolutions to each slide' of the stack of pseudo images, but then says it 'reshapes the 4D features ... by collapsing the height dimension' and applies 2D convolutions along X-Y. These are different operations: per-slice 2D convolutions would scale FLOPs by the number of slices (e.g., 16 z-slices on KITTI, plus hundreds of slices for the other axes), making the reported 8.9 ms backbone time implausible; collapsing height into channels would yield filters that span the full Z-extent, contradicting the stated 1×k×k kernel shape. The paper never specifies how empty voxels are handled, whether shared/grouped convolutions are used, or the actual FLOPs/memory compared with sparse 3D convolutions. Moreover, the claimed '45% parameter reduction' for k=3 is arithmetically false: three 2D kernels of sizes 1×3×3, 3×1×3, and 3×3×1 contain 3×9=27 parameters, identical to a single 3×3×3 kernel. Thus the two quantitative pillars of the efficiency story—parameter reduction and runtime—depend on an unverified or incorrect description. Without code or a precise specification, the reader cannot judge whether SVFM is genuinely cheaper than sparse 3D convolutions or merely a dense approximation that happens to fit on the reported benchmark.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 3DPillars, a two-stage LiDAR-based 3D object detector built on pseudo image representations. Its first stage (SVFM) treats the sparse voxel feature volume as stacks of pseudo images and applies 2D convolutions along the X, Y, and Z axes instead of 3D convolutions, in order to preserve 3D structure while retaining efficiency. The second stage (S2CFM) aggregates multi-scale voxel features into a sparse scene feature and refines RoI features with a key-value memory that provides global context. Experiments are reported on KITTI, Waymo Open, and nuScenes, with a runtime of 29.6 Hz on KITTI and substantial gains over PointPillars. The paper's central claim is that this is the first two-stage pseudo-image-based 3D detector that narrows the accuracy gap to voxel-based state-of-the-art methods while retaining real-time speed.","tokens_in":29005,"tokens_out":3966,"duration_ms":42214,"significance":"If the method and measurements hold up, the paper would offer a useful speed/accuracy trade-off for pillar-based 3D detection: a two-stage detector that avoids 3D convolutions and still reaches real-time on KITTI. The paper has genuine strengths: it evaluates on three datasets (KITTI, Waymo, nuScenes), provides component-wise ablations (Tables 6–9), reports a runtime breakdown (Table 11), and the design of the RoI head with a memory module is a plausible contribution. However, the central efficiency and novelty claims contain errors and ambiguities that must be resolved before the contribution can be assessed: the SVFM description is underspecified, the stated parameter reduction is arithmetically false, and the 'first' and 'best at real-time' claims are contradicted by the paper's own tables.","major_comments":[{"comment":"The SVFM specification is ambiguous and load-bearing. The text says SVFM 'applies 2D convolutions to each slide' but then says it 'reshapes the 4D features ... by collapsing the height dimension' and applies 2D convolutions along X-Y. These are different operations: per-slice convolutions scale FLOPs with the number of slices, while collapsing height changes the filter's receptive field and contradicts the stated 1×k×k kernel shape. The paper also never states how empty voxels are handled, whether the convolutions are sparse, dense, or submanifold, or what the actual FLOPs/memory are. Without this specification, the reported 8.9 ms backbone time and the claim that SVFM avoids 3D convolutions cannot be verified.","section":"§3.1.2, Fig. 3"},{"comment":"The claimed '45% parameter reduction compared to the 3D case when k = 3' is arithmetically incorrect. Three 2D kernels with sizes 1×k×k, k×1×k, and k×k×1 contain 3·k^2 parameters; a single 3D kernel k×k×k contains k^3. For k=3 both are 27, so there is zero parameter reduction. A reduction only appears for k>3. This quantitative pillar of the efficiency story needs to be corrected or removed.","section":"§3.1.2, parameter count"},{"comment":"The claim of being the 'first two-stage 3D detection framework exploiting pseudo image representations' is contradicted by Table 1, which lists LiDAR R-CNN as a BEV-based two-stage method (35.6 Hz). In the same table, the statement that 'ours achieves the best mAP across all classes at real-time speed' is falsified by Voxel R-CNN (25.2 Hz) with higher Car Easy/Moderate mAP, and by PVT-SSD (20.4 Hz) with higher Car Easy/Moderate/Hard mAP, both above the 20 Hz real-time threshold used in the paper. These claims should be narrowed or qualified.","section":"Abstract, §1, Table 1"},{"comment":"On Waymo Open, the gap to recent fully sparse voxel methods is substantial (e.g., vehicle L2 mAP 70.42 vs 73.10 for ScatterFormer and 72.70 for SAFDNet; pedestrian L2 mAP 73.12 vs 77.00 and 77.30). The text acknowledges this, but the abstract's 'narrowing the performance gaps ... while retaining its efficiency' overstates the case without also reporting the corresponding efficiency comparison for those methods. The accuracy/efficiency trade-off should be presented more carefully.","section":"§4.2.2, Table 3"}],"minor_comments":[{"comment":"The confidence loss is denoted \\mathcal{L}_{cfd} but the text says 'confidence (cfd)'; likely a typo for \\mathcal{L}_{conf} or similar.","section":"§3.3, Eq. (10)"},{"comment":"The paper says SVFM 'reduces the number of parameters significantly by replacing the expensive 3D operations with 2D convolutions,' but no comparison of SVFM's parameter count against a 3D-convolution baseline is actually reported. Table 10 compares whole detectors, not the SVFM vs a 3D-convolution backbone under matched design.","section":"§4.3.6, Table 10"},{"comment":"Typos: 'perfomance' and 'PoinPillars' should be corrected.","section":"§4.2.2"},{"comment":"The four SVFM variants are described in the caption and text, but it would help to specify the exact tensor reshape/grouping used for each variant, especially for the sequential-parallel and parallel-sequential cases, to make the ablation reproducible.","section":"Fig. 4"},{"comment":"Key and value memory sizes are tuned on the validation split (K=10,V=50 on KITTI; K=20,V=200 on Waymo). This should be stated as validation-tuned hyperparameters, which the paper does, but the sensitivity analysis in Fig. 6 is only for V and only on KITTI. Reporting sensitivity to K would strengthen the memory module's robustness claim.","section":"§4.1.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical architecture study, but the novelty and efficiency claims are overstated. The 'first two-stage pseudo-image detector' is contradicted by LiDAR R-CNN in the paper's own table, and the parameter-reduction arithmetic is wrong for k=3. These are fixable with rewriting and with an explicit specification of SVFM's sparse/dense handling and FLOPs, but they are load-bearing for the contributions as stated. I would not reject if the authors can provide the missing details and correct the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a legitimate engineering contribution, not a breakthrough, and it is mostly well executed. What is new: treating the sparse voxel feature volume as stacked pseudo images along three axes with axis-decomposed 2D convolutions (SVFM), plus a memory-augmented context-aware RoI head (S2CFM). The ablation suite is properly done—each component is isolated—and the results hold consistently on KITTI, Waymo, and nuScenes, with strong small-object and far-object gains at 29.6 Hz. That is enough to make the paper worth a serious look.\n\nThe soft spots are mostly in the framing and the specification. The 'first two-stage pseudo-image framework' is contradicted by LiDAR R-CNN, which sits in their own Table 1 as a BEV two-stage method. The 'best mAP across all classes at real-time speed' is contradicted by Voxel R-CNN (25.2 Hz) and PVT-SSD (20.4 Hz) for the car class. The stated '45% parameter reduction' for k=3 is arithmetically wrong: three 2D kernels of size k^2 total 27 parameters, the same as one 3D kernel of size k^3; the reduction only appears for larger k. None of these sink the core result, but they need correction.\n\nThe more substantive concern is that SVFM is underspecified. Section 3.1.2 first says 2D convolutions are applied 'to each slide' of the stack, then says the height dimension is 'collapsed' before X-Y convolution. Those are different operations with very different FLOPs, and the paper never says how empty voxels are handled or compares FLOPs/memory against sparse 3D convolutions. The 8.9 ms backbone runtime suggests they did something efficient, but the text does not pin down what. This is the piece that needs clarification, or released code, before the efficiency claim is fully testable.\n\nMinor: no error bars, code promised but not available, and memory sizes are tuned on the validation split. Standard practice, but worth stating.\n\nBottom line: the empirical case is credible and the architecture is worth knowing. With corrected claims and a precise SVFM description, this is a solid submission. I would send it to review, and I would point the referees at the three flagged issues.","headline":"Solid, well-ablated two-stage pillar detector; the headline claims oversell it and SVFM's exact computation needs to be pinned down before the efficiency story is credible.","tokens_in":29558,"tokens_out":3732,"would_cite":true,"duration_ms":36061,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a two-stage pillar-based 3D detector that processes voxel features as stacks of pseudo images can match slower voxel-based detectors while running in real time.","keywords":["3D object detection","LiDAR","point clouds","pseudo image representation","pillar-based detection","two-stage detector","separable voxel feature module","autonomous driving"],"falsifier":"Run the same two-stage detector on KITTI with the SVFM backbone replaced by standard sparse 3D convolutions at matched channel counts, keeping the S2CFM RoI head and training schedule fixed. If the 3D-convolution version matches or exceeds the reported mAP while the separable version's claimed memory and latency advantage fails to appear under a profiler, the central claim that 2D axis-decomposed convolutions preserve needed 3D structure at lower cost is not supported. A second, quicker check: disable the side- and front-view pseudo-image stacks, keeping only the BEV stack; if accuracy hardly","tokens_in":28483,"feed_emoji":"🚗","tokens_out":6498,"duration_ms":63091,"temperature":0.7,"pith_summary":"The paper tries to establish that a pillar-style 3D detector, the fastest known family, does not have to give up accuracy to keep its speed. It claims the two reasons PointPillars lags behind state-of-the-art detectors are fixable: the pseudo-image representation collapses vertical structure, and 3D proposals cannot be refined in a second stage on a 2D feature map. The paper's answer is 3DPillars, a backbone that treats the 3D voxel feature volume as three stacks of pseudo images (bird's-eye, side, front) and runs cheap 2D convolutions on each, plus an RoI head that pools multi-scale features and a shared memory of scene contexts to refine proposals. On KITTI it reports large gains over PointPillars while running at 29.6 Hz, and it reports top cyclist and pedestrian results on the KITTI validation set. The significance, if true, is that real-time 3D perception for driving does not require choosing between speed and close-to-voxel accuracy.","feed_headline":"Two-stage pillar detector closes the accuracy gap at 29.6 Hz","feed_subtitle":"Slicing voxel features into three pseudo-image views adds 3D structure and richer context to real-time LiDAR detection.","key_machinery":"Two named modules carry the argument. SVFM (separable voxel feature module) treats the 4D voxel feature volume as a stack of pseudo images and decomposes a 3D convolution of size k×k×k into three 2D convolutions, 1×k×k over the X-Y plane, k×1×k over Y-Z, and k×k×1 over X-Z, so view-specific features are learned with 2D filters; sequential ordering of the three views outperforms parallel ordering in the paper's ablations. S2CFM (sparse scene context feature module) concatenates initial VFE features with trilinearly interpolated multi-scale backbone features at non-empty voxel positions to build a sparse scene feature, pools sub-RoI features via the voxel RoI pooling scheme, and uses a key-val","core_discovery":"On the paper's own terms, the discovery is that a two-stage 3D object detector can be built entirely on pseudo image representations, something previously available only to voxel-based or point-based pipelines. The authors state this is the first two-stage framework exploiting pseudo images, and they attribute its success to two components: SVFM, which splits the sparse voxel feature volume along the X, Y, and Z axes and applies 2D convolutions (1×k×k, k×1×k, k×k×1) to each stack, preserving 3D structure without 3D convolutions; and S2CFM, which forms a sparse scene feature from the initial voxel features plus multi-scale backbone features, pools RoI features from it, and enriches them with","pith_inferences":["The axis-decomposed 2D convolution idea is a general recipe: any 3D feature volume on a dense grid could be processed as three view stacks, which may transfer to other perception tasks such as 3D segmentation or occupancy prediction at lower cost than 3D convolutions.","Because the paper leaves open how empty voxels are handled in SVFM, the reported efficiency depends on whether the volume is processed densely; a sparse-aware implementation of the same decomposition could be a natural next step and might close the remaining gap to fully sparse voxel transformers.","The paper's arithmetic for parameter reduction (45% for k=3) does not match 3·k² versus k³, so the actual savings come from channel and depth structure rather than the kernel decomposition alone; a fair comparison would report FLOPs and memory, not just parameter counts.","The shared key-value memory of scene prototypes suggests a learned prior over driving environments; one testable extension is to update or condition the memory on the current scene's layout, such as map or lane information, to reduce context ambiguity like the reported garden-car false positive."],"forward_implications":["Two-stage refinement becomes available to any pillar-based or pseudo-image detector, not just sparse-3D-CNN pipelines.","Small and distant objects, such as cyclists, pedestrians, and far vehicles, benefit most because multi-scale pooling and global context compensate for sparse point clouds.","Real-time operation is preserved: 29.6 Hz on KITTI with 8.1M parameters, and the added RoI head costs about 6 ms and 115 MB over the single-stage backbone.","The same architecture transfers across KITTI, Waymo Open, and nuScenes with competitive BEV-family results, indicating the gains are not dataset-specific.","Sequential rather than parallel processing of the three pseudo-image views yields better accuracy, so how the views interact matters as much as their existence."],"supporting_citations":[{"why":"Baseline PointPillars detector whose pseudo-image representation and efficiency the paper inherits and whose accuracy gaps it measures against.","marker":"(Lang et al., 2019)"},{"why":"Supplies the two-stage voxel RoI pooling method and the Voxel R-CNN baseline that S2CFM adapts and outperforms.","marker":"(Deng et al., 2021)"},{"why":"PV-RCNN, the two-stage hybrid baseline whose VSA module is compared to S2CFM and whose RPN/RoI loss formulation is followed.","marker":"(Shi et al., 2020a)"},{"why":"Provides the VFE layer and voxel grid formulation used to build the sparse feature volume and initial voxel features.","marker":"(Zhou & Tuzel, 2018)"},{"why":"Waymo Open Dataset used to validate generalization beyond KITTI, including distance-stratified vehicle results.","marker":"(Sun et al., 2020)"},{"why":"KITTI benchmark providing the main test and validation comparisons for car, cyclist, and pedestrian classes.","marker":"(Geiger et al., 2012)"},{"why":"SECOND, source of the direction-classification loss, sparse convolution baseline, and ground-truth sampling augmentation used in training.","marker":"(Yan et al., 2018)"}],"fun_headline_variants":["First two-stage LiDAR detector on pseudo images","Two-stage pillar detector runs at 29.6 Hz","Splitting voxels into 3 views gives 3D structure without 3D convs","Aggregating sparse scene context refines 3D proposals","3DPillars: Two-stage detector with 2D convolutions only"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole accuracy-versus-speed trade-off rests on the claim that splitting the 3D voxel feature grid into three stacks of pseudo images and applying 2D convolutions along each axis captures enough 3D structure to match 3D convolutions, yet the paper never specifies how empty voxels are treated, what exactly collapsing the height dimension does, or the actual FLOPs and memory compared with sparse 3D convolutions.","fun_headline_variants_meta":{"raw":{"variants":["First two-stage LiDAR detector on pseudo images","Two-stage pillar detector runs at 29.6 Hz","Splitting voxels into 3 views gives 3D structure without 3D convs","Aggregating sparse scene context refines 3D proposals","3DPillars: Two-stage detector with 2D convolutions only"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000811,"raw_usage":{"total_tokens":3459,"prompt_tokens":875,"completion_tokens":2584,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":2501}},"tokens_in":619,"tokens_out":2584,"duration_ms":20330,"temperature":1.0,"reasoning_tokens":2501,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T04:58:38.054838+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same two-stage detector on KITTI with the SVFM backbone replaced by standard sparse 3D convolutions at matched channel counts, keeping the S2CFM RoI head and training schedule fixed. If the 3D-convolution version matches or exceeds the reported mAP while the separable version's claimed memory and latency advantage fails to appear under a profiler, the central claim that 2D axis-decomposed convolutions preserve needed 3D structure at lower cost is not supported. A second, quicker check: disable the side- and front-view pseudo-image stacks, keeping only the BEV stack; if accuracy hardly","supporting_citations":[{"cited_title":"H., V ora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O","cited_arxiv_id":null,"evidence_quote":"Baseline PointPillars detector whose pseudo-image representation and efficiency the paper inherits and whose accuracy gaps it measures against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the two-stage voxel RoI pooling method and the Voxel R-CNN baseline that S2CFM adapts and outperforms."},{"cited_title":"& Tuzel, O","cited_arxiv_id":null,"evidence_quote":"Provides the VFE layer and voxel grid formulation used to build the sparse feature volume and initial voxel features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Waymo Open Dataset used to validate generalization beyond KITTI, including distance-stratified vehicle results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"KITTI benchmark providing the main test and validation comparisons for car, cyclist, and pedestrian classes."}],"review_version":1}