{"id":"e63a8d54-192b-4b48-9c35-1ee5514a2a75","arxiv_id":"2412.18870","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A three-stage active learning sampler using category entropy, graph-based scene similarity, and mixture density uncertainty improves 3D object detection with fewer labeled scenes.","lead":"TSceneJAL is an active learning method that chooses which traffic scenes to label for 3D object detection, aiming to cut annotation cost while keeping detection accuracy. It scores scenes by class balance, redundancy, and prediction uncertainty, then picks the most informative ones in three stages.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Early-round pseudo-label noise is unquantified; all three selection metrics are computed from the current predictor, so the claimed mechanism may not be operating in the low-label regime where it must.","rationale":"The reader's weakest assumption singles out pseudo-label reliability, and the manuscript text supports that concern: Sec. VII-C1 admits the initial predictor is weak, and the only safeguard is a confidence threshold plus one motivating example. I considered the flawed theoretical derivation around Eqs. (8)-(10) as a possible load-bearing issue, but the central claim is empirical, and the theory is post-hoc motivation; a corrected derivation would not change the tables. I also considered the absence of error bars in Table I, but Table VI provides some variance evidence for the similarity component, and the pseudo-label issue is more directly tied to whether the method does what it claims. The concrete test would settle the mechanism: if pseudo-label-based and oracle-based rankings agree in round 1, the concern is resolved; if they disagree, the paper should report how selection quality evolves across rounds and should qualify the claim that the three metrics drive the gains from the first iteration. Since the reader already assigned CONDITIONAL, my assessment does not move the verdict; it reinforces that CONDITIONAL was the right call and points to the specific experiment the authors should run.","tokens_in":25883,"tokens_out":7599,"duration_ms":72874,"concrete_test":"On KITTI, run the round-1 selection and compute all three metrics twice for the unlabeled pool: once from the actual round-0 predictor's pseudo-labels and once from oracle ground-truth annotations on the same frames. Measure Kendall's tau between the two category-entropy rankings and the overlap of the top-K1*Nr candidates entering stage 2, and also the overlap of the final Nr selected scenes under the full pipeline. If the rank correlation is near zero or the top-set overlap is close to random, early-round selection is not implementing the claimed balance/diversity/complexity criteria, and the headline gains need to be re-attributed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that category entropy, graph similarity, and MDN uncertainty jointly select scenes that improve 3D detection. All three metrics are computed from the current predictor's pseudo-labels: category entropy (Eq. 12) uses predicted class counts, scene graphs (Sec. V-B) use predicted categories and boxes, and MDN uncertainty (Eq. 23) comes from the same regression head. In the first KITTI round, the predictor is trained on only 200 scenes (about 5% of the training set), so pseudo-labels are noisy. The paper explicitly acknowledges that the AL predictor is 'hardly capable of detecting objects in the initial learning process' (Sec. VII-C1, Fig. 9) and applies a confidence threshold (Eq. 14) plus one illustrative example and a threshold ablation (Table X). It never measures how selection rankings degrade as pseudo-label noise grows. If false positives inflate category entropy for simple scenes and false negatives hide minority classes, the stage-1 and stage-2 rankings are not the intended balance/diversity rankings. Figure 5 shows TSceneJAL is not better in the first two KITTI rounds but improves later; this is consistent with the metrics becoming reliable only after the predictor improves, which would mean the reported advantage is not evidence that the claimed mechanism works from the start. This is an internal robustness gap, not a disagreement with consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TSceneJAL, an active learning (AL) framework for 3D object detection in autonomous driving. The method combines three sampling criteria in a fixed three-stage pipeline: category entropy to encourage class balance, graph-based scene similarity with farthest-point sampling to reduce redundancy, and mixture density network (MDN) uncertainty to select complex scenes. The AL predictor is a PointPillars model with the regression head modified to an MDN, so that aleatoric and epistemic uncertainties are obtained in one forward pass. The method is evaluated on KITTI, Lyft, nuScenes, and SUScape against Random, Confidence, MC Dropout, Coreset, Badge, and Crb. Table I reports that TSceneJAL achieves the best final mAP3D/mAPBEV on most settings, with gains over Random of roughly 1.2–2.5 mAP points depending on dataset and difficulty level. Ablations study each metric, the confidence threshold, stage order, initial data quantity, and selection complexity.","tokens_in":26220,"tokens_out":7283,"duration_ms":67905,"significance":"If the empirical claims hold, the contribution is a practical, multi-criteria AL pipeline for LiDAR-based 3D detection, with the useful engineering property that uncertainty estimation is obtained in a single forward pass. The paper's strengths are its breadth of datasets, the inclusion of a released code repository, and ablations that separately test each of the three metrics, the stage order, and the confidence threshold. The main limitations are that the headline comparison rests on single runs without error bars or significance tests, that the proposed mechanism is not verified in the low-label regime on which AL depends, and that the theoretical derivation in Section III is not sound as written. Overall, the empirical direction is promising and the issues are addressable, but the central claim needs additional support before publication.","major_comments":[{"comment":"The central empirical claim that TSceneJAL outperforms all baselines is not statistically supported. Table I reports only single-run mAP values, while the gains over Random are small (e.g., roughly 1.2 mAP3D points on nuScenes). The only repeated-run results in the paper, Table VI, show standard deviations of 0.65–1.47 mAP3D for a closely related selection pipeline, which is the same magnitude as several of the headline gains. Please report results over multiple seeds with means and error bars, and ideally paired significance tests, for at least the KITTI and nuScenes comparisons.","section":"Section VII-B, Table I, and Table VI"},{"comment":"Pseudo-label noise in early AL rounds is acknowledged but never quantified. All three metrics are computed from the current predictor's pseudo-labels: category entropy uses predicted class counts, scene graphs use predicted classes and boxes, and MDN uncertainty comes from the same regression head. In the first KITTI round the predictor is trained on only 200 scenes, and Section VII-C1 states that the AL predictor is 'hardly capable of detecting objects in the initial learning process.' The confidence threshold in Eq. (14) and its ablation in Table X address one symptom, but the paper does not measure how pseudo-label noise degrades the entropy, similarity, or uncertainty rankings. Figure 5 shows that TSceneJAL is not better in the first two KITTI rounds and improves later, which is consistent with the metrics becoming reliable only after the predictor improves. Please add a quantitative analysis of ranking stability or pseudo-label noise sensitivity, or temper the mechanistic interpretation of the gains.","section":"Section V (Eqs. 12–14), Section VII-C1, Figures 5 and 9"},{"comment":"The theoretical derivation is not sound as written. Eq. (8) states D*_S3 = arg max H(DS3) = arg max [H(DS3|ω) + H(ω) − H(ω|DS3)], and Eq. (10) concludes D*_S3 ≈ arg min H(ω|DS3) from the assertion H(ω|DS3) ≪ H(DS3|ω). If the conditional entropy is negligible, dropping it from a maximization does not yield a minimization of that term; the conclusion would require the other two terms to be constant, which is neither stated nor established. In addition, Eq. (3) introduces a decomposition of the distribution discrepancy into three KL terms with assumed target distributions (uniform, Gaussian, uniform) and no derivation; this appears to be a post hoc mapping onto the three proposed metrics. Because the empirical comparison is independent of this derivation, the issue is fixable by rewriting Section III as a design rationale rather than a proof.","section":"Section III, Eqs. (3), (8)–(10)"},{"comment":"The reported improvement ranges are inconsistent with Table I. The abstract promises improvements 'up to 12%,' and the conclusion lists average improvements of 2.5%–6.6%, 4.4%–9.0%, 3.0%–10.9%, and 0.5%–6.2% for the four datasets. Direct comparisons to Random in Table I are about 2.2–2.6 points for KITTI, 1.7–2.5 for Lyft, 1.0–1.5 for nuScenes, and 1.1–1.5 for SUScape; no value is near 9–12%. Please correct the abstract and conclusion, or specify exactly which baseline and metric produce the larger ranges.","section":"Abstract, Section VII-B, and Section VIII"}],"minor_comments":[{"comment":"The entropy term log(p_c + ζ) assigns a large positive entropy to a scene with no detections (p_c = 0 for all c), since H = −C log ζ. Please state how empty scenes are handled in the category entropy computation.","section":"Section V-A, Eq. (12)"},{"comment":"The kernel K(v, v') = ξ(v = v')/2 assigns similarity 1/2 between identical nodes rather than 1; please clarify whether this normalization is intentional, and correct the phrase 'marginalize kernel' to 'marginalized kernel' throughout the paper.","section":"Section V-B, Eq. (15)–(17)"},{"comment":"For the anchor-based residual encoding used by PointPillars, the propagation of variance for width, length, and height should include the anchor dimensions (or the actual box dimensions), not only the residual means squared. Please clarify the notation and verify the formula.","section":"Section V-C, Eq. (22)"},{"comment":"The caption states that numerical values are presented in 10^-2 order, but the table lists values such as 0.663 and 1.732; please make the stated scale consistent with the displayed numbers.","section":"Section VII-C2, Table VII"},{"comment":"The paper contains several typographical errors, including 'sence sampling' in the index terms, 'correponding' in the Fig. 4 caption, and 'pesudo-labels' in the introduction; a careful proofreading pass is recommended.","section":"General"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nWhat you should know first: the real contribution is an engineering integration, not new theory. Three existing ideas — category entropy, marginalized graph kernel similarity, and MDN uncertainty — are staged into a joint sampler for LiDAR 3D detection. The paper cites the prior art clearly. The new bit is the three-stage policy and the breadth of evaluation: KITTI, Lyft, nuScenes, and SUScape, with consistent mAP gains over Random and over Crb, the closest state of the art. That is a legitimate contribution, if not a deep one. The code link and the complexity table help.\n\nThe empirical core is solid enough to take seriously. The ablations are informative: each metric adds something, the confidence threshold helps, stage order matters, and Table X shows the authors noticed the pseudo-label problem. Figure 5 honestly shows no early-round advantage, with gains appearing later — consistent with the metrics becoming reliable as the predictor improves. The paper admits as much in Sec. VII-C1. I would call the central empirical claim plausible, not proven.\n\nSoft spots, in proportion. First, no error bars or significance tests on the headline numbers. The gains are 1 to 2.5 mAP points; without variance, part of that could be noise. Second, the theory section in Sec. III has a shaky jump from maximizing H(D) to minimizing H(omega|D) in Eqs. (8)-(10). It reads post hoc, with the three KL terms in Eq. (3) mapped onto the three metrics. The empirical results do not depend on that derivation, but it should be fixed or cut. Third, the pseudo-label noise concern is real: all three metrics are computed from the current predictor's outputs, and the first KITTI round trains on only 200 scenes. The threshold in Eq. (14) and the single example in Fig. 9 are not a quantitative analysis of how selection rankings degrade as noise grows. This does not sink the paper, but it is an under-analyzed gap. Fourth, the abstract's 'up to 12%' is not supported by the main tables; the conclusion's ranges are the honest summary. Minor: the code has no commit hash and the graph kernel bandwidth is unspecified, so exact reproduction is not yet possible.\n\nWho is this for: anyone building active learning pipelines for LiDAR detection or curating AD datasets. It deserves a serious referee. I would send it out, asking for variance reporting, a corrected theory section, and a direct study of pseudo-label noise in early rounds. With those, it could become a reliable reference.\n\nRecommendation: peer review, conditional on those revisions.","headline":"A workmanlike empirical AL paper: known ingredients, new integration, consistent moderate gains on four datasets; theory is weak and early-round pseudo-label noise is under-examined, but the empirical core probably survives.","tokens_in":26713,"tokens_out":2556,"would_cite":true,"duration_ms":25911,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TSceneJAL proposes that a three-stage sampler — category entropy, graph-based scene similarity, and mixture-density-network uncertainty — selects traffic scenes that train better 3D object detectors than existing active learning methods…","keywords":["active learning","3D object detection","autonomous driving","category entropy","scene similarity","mixture density network","uncertainty estimation","point clouds"],"falsifier":"Run the same three-stage sampler twice from a very small initial labeled pool — once scoring scenes from the detector's pseudo-labels and once scoring the same scenes from oracle ground-truth labels — and compare the mAP3D of the final trained detectors; if the pseudo-label version loses most or all of its advantage over Random in early rounds, the central claim that the three metrics are robust selection signals would be falsified.","tokens_in":25650,"feed_emoji":"🚗","tokens_out":7943,"duration_ms":64203,"temperature":0.7,"pith_summary":"TSceneJAL aims to solve a practical problem: under a fixed annotation budget, which unlabeled traffic scenes should be labeled to train the best 3D object detector. The paper's answer is a three-stage sampler that filters scenes in sequence — first those with the most balanced object-category mix, then the most diverse among those by graph-based scene similarity, then the most complex by a mixture-density-network uncertainty score. The paper claims this joint policy beats Random, Confidence, MC Dropout, Coreset, Badge, and Crb on KITTI, Lyft, nuScenes, and SUScape, with average mAP3D gains over Random of about 2.45%, 2.13%, 1.24%, and 1.25%, respectively, and gains over the strongest prior method, Crb, between roughly 0.9 and 1.6 points. The practical stakes are that labeling 3D driving data is expensive, and the paper also reports that on KITTI its selected sets contain roughly half the objects of Random's sets at comparable or better accuracy, and that sampling 60–80% of the data can match fully supervised training.","feed_headline":"Active learner lifts 3D detection using fewer labeled scenes","feed_subtitle":"Filtering by class balance, diversity, and uncertainty beats Random and SOTA baselines on four AD benchmarks.","key_machinery":"The mechanism is the three-stage cascade itself. Stage one ranks unlabeled scenes by Eq. (12), a class-proportion entropy of pseudo-labels thresholded at confidence 0.3, and keeps the top $K_1N_r$; stage two builds a directed scene graph with one node per detected object plus the ego vehicle, edge weights $1/\\text{distance}$, computes scene similarity through the marginalized kernel of Eq. (17), and applies farthest-point sampling (Algorithm 1) to keep $K_2N_r$ diverse scenes; stage three scores each remaining scene with Eq. (23), the mean over boxes and seven box attributes of aleatoric plus epistemic variance from the MDN regression head, and keeps the $N_r$ most uncertain. Algorithm 2 stitches these stages together and retrains the PointPillars predictor each round.","core_discovery":"The central claim, stated on the paper's own terms, is that optimal active learning for 3D detection should be treated as a three-part divergence-minimization problem — balance, diversity, and complexity — and that a stage-by-stage greedy cascade is the right way to minimize all three at once. Each scene is scored three ways: category entropy of the predicted class distribution filters for balance; a directed graph whose nodes are detected objects and whose edges encode inverse 3D distance, compared with the marginalized kernel and sampled by farthest-point selection, filters for diversity; and a mixture density network attached to the regression head outputs per-attribute aleatoric and epistemic variances whose sum scores complexity. Running these metrics in the order entropy → similarity → uncertainty produces the reported results: consistent mAP3D/mAPBEV improvements over all baselines on all four datasets, with the largest redundancy-related gains on the sequential Lyft data and up to 12% improvement reported overall.","pith_inferences":["A testable extension is to make the confidence threshold of Eq. (14) adaptive across active-learning rounds; the paper's own sweep shows a 0.9 threshold degrades to near-random, so a noise-aware threshold could extend the gains to very early rounds where pseudo-labels are least reliable.","If pseudo-label noise is the binding constraint, coupling the three-stage sampler with a pseudo-label refinement step — for example, ensembling or temporal aggregation of predictions before scoring — could preserve more of the advantage in low-budget regimes, a direction the paper leaves open.","Because the scene-graph similarity uses only 3D positions and class names, the same diversity stage should transfer to other geometry-heavy perception tasks, such as indoor 3D object detection or LiDAR segmentation, where redundant frames also inflate labeling costs.","The KITTI redundancy result (60–80% of data matching fully supervised accuracy) suggests a standalone benchmark: measure how much pure near-duplicate removal from raw sensor streams can cut annotation budgets before any active-learning loop is added."],"forward_implications":["If the central claim holds, a fixed annotation budget buys more detection accuracy: the paper's TSceneJAL beats Random by 2.45 mAP3D on KITTI, 2.13 on Lyft, 1.24 on nuScenes, and 1.25 on SUScape at the end of the active learning process.","The reported KITTI object-count statistics imply annotation cost is roughly halved for the same model quality, because TSceneJAL's selected scenes contain about half as many objects as Random's and still yield higher mAP.","Sequential, redundant data benefits most from the joint policy: the Lyft dataset, collected as continuous drives, shows the largest similarity-stage gains (+1.61 to +2.01 mAP3D over the no-similarity variant), suggesting redundancy removal is where the biggest wins lie.","The stage order matters: entropy-first beats uncertainty-first by a clear margin in Table VIII, so practitioners should prioritize class balance before complexity sampling.","On KITTI, matching or exceeding fully supervised training requires only 60–80% of the data, implying roughly 20–40% of many existing AD datasets may be redundant for training."],"supporting_citations":[{"why":"Supplies the generalization-risk upper bound in Eq. (1) that frames active learning as minimizing distribution disparity, and provides the three-stage sampling structure and Crb baseline that TSceneJAL extends.","marker":"[12]"},{"why":"Provides the marginalized kernel algorithm used to compute graph-based scene similarity in Eq. (17).","marker":"[40]"},{"why":"Provides the mixture density network variance modeling and the aleatoric/epistemic uncertainty decomposition used in Eqs. (20)-(21).","marker":"[22]"},{"why":"Provides the PointPillars detector whose regression head is converted into the MDN and which serves as the backbone for all baselines.","marker":"[42]"},{"why":"Supplies the zero-training-error assumption and the diversity-based coreset baseline that the paper contrasts with and outperforms.","marker":"[9]"},{"why":"Supplies the MC Dropout regression-uncertainty baseline and motivates the paper's efficiency argument for single-forward-pass MDN uncertainty.","marker":"[10]"},{"why":"Supplies the Badge hybrid uncertainty-diversity baseline, which the paper argues is unsuitable for regression tasks like 3D box prediction.","marker":"[11]"},{"why":"Supplies the Bayesian active learning objective used to derive the uncertainty-maximization selection rule in Eq. (11).","marker":"[39]"}],"fun_headline_variants":["Joint active learning balances, diversifies, and targets hard scenes","TSceneJAL: entropy, kernels, and MDN pick smarter 3D scenes","Three-filter active learning boosts 3D detection by up to 12%","Balanced, diverse, uncertain: the triple filter for active 3D learning","Cut labeling cost with joint scene sampling for 3D detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that the current detector's pseudo-labels — class confidences, box regressions, and MDN variances — are reliable enough that the three metrics rank scenes correctly, an assumption that is most strained in the first active-learning rounds when the labeled pool is tiny and false positives can, as the paper's Fig. 9 shows, turn a one-car scene into a high-entropy scene.","fun_headline_variants_meta":{"raw":{"variants":["Joint active learning balances, diversifies, and targets hard scenes","TSceneJAL: entropy, kernels, and MDN pick smarter 3D scenes","Three-filter active learning boosts 3D detection by up to 12%","Balanced, diverse, uncertain: the triple filter for active 3D learning","Cut labeling cost with joint scene sampling for 3D detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000531,"raw_usage":{"total_tokens":2568,"prompt_tokens":964,"completion_tokens":1604,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1505}},"tokens_in":580,"tokens_out":1604,"duration_ms":10481,"temperature":1.0,"reasoning_tokens":1505,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:22:43.687146+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same three-stage sampler twice from a very small initial labeled pool — once scoring scenes from the detector's pseudo-labels and once scoring the same scenes from oracle ground-truth labels — and compare the mAP3D of the final trained detectors; if the pseudo-label version loses most or all of its advantage over Random in early rounds, the central claim that the three metrics are robust selection signals would be falsified.","supporting_citations":[{"cited_title":"Marginalized kernels between labeled graphs,","cited_arxiv_id":null,"evidence_quote":"Provides the marginalized kernel algorithm used to compute graph-based scene similarity in Eq. (17)."},{"cited_title":"Uncertainty-aware learning from demonstration using mixture density networks with sampling-free variance modeling,","cited_arxiv_id":null,"evidence_quote":"Provides the mixture density network variance modeling and the aleatoric/epistemic uncertainty decomposition used in Eqs. (20)-(21)."},{"cited_title":"Pointpillars: Fast encoders for object detection from point clouds,","cited_arxiv_id":null,"evidence_quote":"Provides the PointPillars detector whose regression head is converted into the MDN and which serves as the backbone for all baselines."},{"cited_title":"Deep active learning for efficient training of a lidar 3d object detector,","cited_arxiv_id":null,"evidence_quote":"Supplies the MC Dropout regression-uncertainty baseline and motivates the paper's efficiency argument for single-forward-pass MDN uncertainty."}],"review_version":1}