{"id":"cddddc9b-84cd-425f-a8ea-35d893860408","arxiv_id":"2507.19239","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A cooperative tracking framework that fuses learned instance features from vehicle and infrastructure through learnable graph-based association, achieving SOTA on V2X-Seq.","lead":"CoopTrack introduces an end-to-end framework for cooperative 3D multi-object tracking where a vehicle and a roadside unit transmit sparse instance-level features and learn to associate them. It reports state-of-the-art results on the V2X-Seq benchmark, with a 9.5 point higher mAP than the previous best end-to-end method while using less bandwidth.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The learned association module is trained on pseudo-labels whose quality is asserted but never quantified; without a label-quality or oracle comparison, the SOTA gain cannot be attributed to the central contribution.","rationale":"The reader's weakest assumption and my own converge on the same load-bearing point: the association module, the paper's central novelty, is trained from automatically generated labels whose reliability is asserted but never measured. Section 3.5 explicitly claims that the first-stage model's competent tracking performance validates the labels, yet no quantitative evidence is provided, and the manuscript's appendix does not remedy this. My controlled test would settle the question by comparing pseudo-label-driven training against oracle-label-driven training; if the results are equivalent, the concern is largely resolved, while a divergence would show that the SOTA numbers depend on the label-generation pipeline rather than on the learned association itself. I therefore keep the reader's CONDITIONAL verdict, because the experiments and ablations are otherwise reasonably strong and the identified gap is addressable with additional analysis rather than being a fundamental flaw. The paper would become fully acceptable if the authors add label-quality statistics and the oracle-supervision comparison, or otherwise demonstrate that pseudo-label noise does not drive the reported gains.","tokens_in":20988,"tokens_out":4067,"duration_ms":44710,"concrete_test":"Rerun Stage 2 fixing every other component and varying only the association supervision: (a) current Hungarian pseudo-labels, (b) oracle labels defined by center-distance or IoU matching of ground-truth boxes across agents, and (c) no association supervision (lambda_asso = 0). Compare the final Table 4 row across these three conditions. Also report association-label precision, recall, and F1 on the V2X-Seq validation split, with per-sequence variance. If oracle supervision does not materially improve over pseudo-label supervision, or if pseudo-label F1 is substantially below 0.85 with high variance, the learned-association claim and the reported SOTA delta need to be reinterpreted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The second-stage association loss (Eq. 8) is supervised by labels generated by Hungarian-matching first-stage predictions to ground truth, as described in Section 3.5. The only evidence offered for their reliability is the sentence: 'The results demonstrate that the first-stage model achieves competent tracking performance, thereby validating the reliability of the generated labels,' plus a BEV visualization. No precision, recall, F1, per-sequence statistics, or comparison against oracle labels is reported. Because the graph-based association module (GBA) is the paper's advertised advance over rule-based association in UniV2X, the learned-association advantage may encode systematic biases of the stage-1 model, such as duplicate detections, false positives, or identity switches, rather than a genuinely better association mechanism. This is a correctness-risk issue: if the pseudo-labels are noisy in a favorable direction, the reported 9.5 mAP and 4.1 AMOTA improvements over UniV2X could be partly artifacts of the label-generation procedure. The concern is not that the mechanism is impossible; it is that the central claim is under-evidenced at exactly the point where the method differs most from prior work.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoopTrack, an instance-level end-to-end framework for cooperative 3D multi-object tracking from camera inputs. The method transmits sparse instance-level features from the infrastructure agent to the ego vehicle, aligns cross-agent features with a learned Cross-Agent Alignment module, and associates them with a Graph-Based Association module before decoding cooperative outputs. The framework is trained in two stages; the second stage supervises the association module with labels generated by Hungarian matching between first-stage model predictions and ground truth. Experiments on V2X-Seq report state-of-the-art results (39.0% mAP, 32.8% AMOTA) and a transmission cost lower than UniV2X, while experiments on Griffin show improved tracking over a UniV2X baseline. The paper also provides ablations, robustness studies on latency and pose noise, and a public project page.","tokens_in":21256,"tokens_out":5466,"duration_ms":48298,"significance":"If the central claims hold, CoopTrack is a meaningful advance for cooperative sequential perception: it is among the first fully end-to-end cooperative 3D trackers with a learnable association mechanism, it demonstrates a favorable performance-bandwidth trade-off on V2X-Seq, and it includes practical robustness analyses (communication latency, pose noise, input loss). The paper ships detailed ablations and releases code, which are strengths. However, the evidence that the learnable association module is the source of the improvement is incomplete because the association labels used for training are never validated, and the headline numbers come from single runs without variance reporting.","major_comments":[{"comment":"The second-stage association loss is supervised by labels generated by Hungarian matching between first-stage model predictions and ground truth. The manuscript supports the reliability of these labels only with the sentence in Section 3.5 and a BEV visualization; no precision, recall, F1, per-sequence statistics, or comparison against oracle labels is provided. Because the Graph-Based Association module is the paper's main novelty over UniV2X and the reported gains are attributed to it, the authors should quantify the accuracy of the generated association labels, or provide an ablation that trains with oracle labels instead. Without such evidence, the improvement could be driven by systematic biases of the first-stage model rather than by a better learned association mechanism.","section":"Section 3.5, Eq. (8)"},{"comment":"In the Griffin experiment, CoopTrack's transmission cost is 1.17e5 bytes, more than double UniV2X's 5.58e4, but the text states that CoopTrack outperforms UniV2X 'with similar transmission costs.' This is internally inconsistent. The authors should either correct the wording or discuss why the higher transmission cost on Griffin is an acceptable trade-off, especially because the abstract and introduction emphasize low transmission costs as a general property of the method.","section":"Section 4.2, Table 2"},{"comment":"All reported results are from single runs with no error bars or multiple seeds. The paper's headline comparisons—9.5 mAP on V2X-Seq in Table 1, 4.6 mAP on the 10Hz ResNet50 setting in Table 3, and the module ablations in Table 4—would be more convincing with at least three seeds and standard deviations. Without this, it is difficult to assess whether the reported margins are statistically reliable. The authors should provide seed variance for the main comparisons and key ablations, or explicitly justify why this is infeasible.","section":"Section 4.2 and Section 4.3, Tables 1-6"},{"comment":"The ablation table has no row labels. As printed, it is impossible to determine which configuration each row corresponds to (for example, which two-checkmark row is Pipeline+MDFE versus Pipeline+CAA versus Pipeline+GBA). Please add explicit configuration labels to each row or split the table so that the incremental contributions of the pipeline, MDFE, CAA, and GBA are unambiguous.","section":"Section 4.3, Table 4"},{"comment":"In the historical-frame ablation, moving from 4 to 5 frames increases mAP from 0.356 to 0.367 but decreases AMOTA from 0.346 to 0.324. The text states that performance 'begins to saturate at 4 frames' and that 4 is chosen for efficiency; the non-monotonic behavior of the tracking metric deserves an explicit comment, since it suggests a possible trade-off between detection and tracking performance rather than simple saturation.","section":"Section 4.3, Table 5"}],"minor_comments":[{"comment":"The comparison against V2X-ViT in the text states a 12.2% mAP gain and a 4.1% AMOTA gain; the 12.2% appears to be 0.390 - 0.268, but the AMOTA gain is 0.328 - 0.287 = 0.041. The sentence is internally consistent, but the units (percentage points) should be stated explicitly to avoid confusion.","section":"Section 4.2, Table 1 interpretation"},{"comment":"The edge features are defined as an MLP over pairwise differences of reference points, but the text says 'the pair-wise differences D' without specifying whether the difference is absolute value or signed; Eq. (6) then adds edge features to the attention score. Please clarify the exact definition of D and how the edge feature dimension aligns with the node feature dimension.","section":"Section 3.4, Eq. (6)"},{"comment":"The runtime table reports total inference time of 121.88 ms for the ResNet50 model and 207.99 ms for ResNet101, but the text says 'nearly real-time performance at approximately 10Hz.' The reported total time of 121.88 ms corresponds to 8.2 Hz, not 10 Hz; please reconcile this statement with the measured runtime.","section":"Appendix C.2, Table 7"},{"comment":"The association label generation process is described for the positive case (ground truth matched by both agents) and negative cases, but the exact rule for labels when one agent's prediction is matched to a ground truth and the other agent's prediction is matched to a different ground truth is only stated as 'different IDs.' This is clear in principle, but a pseudocode block or a precise algorithmic description would improve reproducibility.","section":"Figure 6 and Section 3.5"},{"comment":"References [13], [20], [58] are described as providing comparisons on V2X-Seq, but it would be helpful to state more explicitly which of these were re-implemented with BEVFormer rather than using their original backbones, since the evaluation protocol in Section B.2 says all methods except CoCa3D use BEVFormer.","section":"Related Work and References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer vision or intelligent vehicle venue. The Griffin dataset is an arXiv preprint from the same group (reference [46]); the authors should establish in the final version that the benchmark is publicly available and documented. I also note a number of self-citations to prior works from the same group; this is not improper, but the related-work section should ensure that independent works are represented fairly. The main revision pressure should be on the pseudo-label validation and the single-run reporting, as these directly affect the credibility of the central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CoopTrack is a genuine step forward for cooperative 3D multi-object tracking. It combines three things that haven't been put together before: an end-to-end query-based tracker, a fusion-after-decoding pipeline, and a learnable graph-based cross-agent association module. The experiments are thorough: two datasets, ablations, latency and pose-noise robustness studies, and the ablation table shows each component contributes. The 9.5 mAP gain over UniV2X on V2X-Seq is plausible, and the Griffin results show it generalizes beyond vehicle-infrastructure. I'd believe the central result, but with one real caveat.\n\nThe caveat is the pseudo-label issue. The second-stage association loss is supervised by labels generated by Hungarian-matching first-stage predictions to ground truth. The paper's only evidence for their reliability is one sentence and a BEV visualization. No precision/recall numbers, no per-sequence statistics, no oracle-label comparison. Since GBA is the main advertised difference from UniV2X's rule-based association, this is a soft spot. It isn't fatal: the ablation shows GBA still helps when trained on those labels, and the final evaluation uses ground truth. But it leaves a real question about how much of the SOTA gain is attributable to the learned association as opposed to the training signal. The fix is straightforward: report label quality, and ideally add an oracle-label ablation. That would close the loop.\n\nMinor concerns: no error bars or multiple seeds, so the exact margins come from a single run. That's common in this subfield but still worth noting. The transmission-cost numbers are reported but I'd like a clearer statement of what bytes are counted. The code link is good; I couldn't verify the repo from the preprint, but at least it's there.\n\nOverall: this paper deserves a serious referee. It presents a meaningful architectural advance and backing experiments. I would not desk-reject it. I'd send it to review with a request for pseudo-label analysis and variance estimates before acceptance. The core contribution looks solid.","headline":"CoopTrack is a real advance in cooperative tracking, but the learned association's pseudo-label supervision is the key soft spot; it deserves peer review with a request for label-quality analysis.","tokens_in":21726,"tokens_out":3617,"would_cite":true,"duration_ms":30432,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoopTrack sets a new state of the art on cooperative 3D multi-object tracking, reaching 39.0% mAP and 32.8% AMOTA on V2X-Seq.","keywords":["cooperative perception","3D multi-object tracking","end-to-end learning","instance-level feature fusion","learnable association","graph attention","vehicle-infrastructure cooperation","autonomous driving"],"falsifier":"Corrupt the first-stage tracker's predicted boxes with calibrated noise before generating the second-stage association labels and measure CoopTrack's AMOTA; if a small amount of added noise causes a large performance drop, the reported gains depend on clean pseudo-labels rather than on the learned association mechanism itself.","tokens_in":20835,"feed_emoji":"🚗","tokens_out":8063,"duration_ms":69879,"temperature":0.7,"pith_summary":"This paper argues that cooperative 3D multi-object tracking should be solved end to end at the instance level: instead of exchanging dense bird's-eye-view maps or fusing features before decoding, each agent should send sparse features describing the objects it already sees, and the receiving agent should learn which features from different agents correspond to the same object. CoopTrack implements this by extracting two complementary representations per candidate object (semantic features for identity and class, motion features built from 3D box geometry), aligning infrastructure features into the vehicle's latent space, and using a graph-attention module to predict an affinity matrix that guides fusion. On the V2X-Seq benchmark the authors report 39.0% mAP and 32.8% AMOTA, beating the previous best end-to-end method by 9.5 points of mAP and 4.1 points of AMOTA while transmitting fewer bytes per frame. The practical significance is a much better performance-bandwidth trade-off for connected driving, with the same architecture also showing gains in aerial-ground cooperation.","feed_headline":"CoopTrack lifts cooperative 3D tracking to 39.0% mAP","feed_subtitle":"Instance-level fusion with learned association beats prior cooperative trackers while sending fewer bytes.","key_machinery":"The load-bearing mechanism is the learned cross-agent association built on multi-dimensional instance features. Every candidate object carries a semantic feature vector and a motion feature vector; the motion vector is produced by a shared point-set encoder over the 3D corners of the coarse box, and both vectors are temporally refined by a transformer block over a sliding window of history. Cross-Agent Alignment treats the feature-domain gap between agents as a latent affine transformation, predicting a rotation and translation from the known relative pose. Graph-Based Association then computes a pairwise affinity from projected node features plus edge features derived from reference-point differences, and this affinity matrix drives Hungarian pairing and feature aggregation. The design's role is to make the decision 'which roadside box is the same as this vehicle box' a differentiable part of the end-to-end tracker.","core_discovery":"The central claim is that the bottleneck in cooperative tracking is not communication size but poor association, and that association should be learned from rich instance representations rather than decided by spatial distance. CoopTrack first decodes each agent's image stream into instance queries, then derives a semantic feature and a motion feature for every instance, with the motion feature coming from the 3D corners of the coarse predicted box and both features refined through a short temporal history. Cross-Agent Alignment maps the roadside features into the vehicle's latent space via a learned rotation and translation, and the Graph-Based Association module builds a fully connected graph over the two instance sets, using node features and reference-point differences as edge features to predict an affinity matrix. Aggregation merges matched pairs and keeps unmatched instances, and the result is decoded into cooperative boxes that propagate their tracking queries to the next frame. The paper supports the claim by ablations in which the fusion-after-decoding pipeline, multi-dimensional features, alignment, and graph association each add measurable gains.","pith_inferences":["Because the association module is trained on multi-dimensional features rather than raw reference-point distance, it should tolerate larger pose-estimation errors than the reference-point baselines; this could be tested by measuring ID switches under increasing rotation noise.","The same latent alignment and graph-association machinery could transfer to other cross-sensor fusion settings (camera-radar, camera-lidar, or agents with different intrinsics), where the domain gap is larger and spatial-only association is known to be fragile.","Since only sparse instance features are transmitted, per-frame communication cost should scale with the number of detected objects rather than scene size; in dense traffic the cost would rise, and the paper reports only average bytes per frame, so cost scaling is an open question.","The reliance on Hungarian-generated pseudo-labels could be removed with a soft, differentiable assignment loss trained jointly, which would eliminate the two-stage training loop the paper itself notes is complex."],"forward_implications":["Sparse instance-level features are enough for strong cooperative tracking: CoopTrack transmits 5.64e4 bytes per frame on V2X-Seq and still beats dense BEV-feature fusion methods on both mAP and AMOTA.","A fusion-after-decoding pipeline avoids the ambiguity of merging queries before decoding, yielding a 2.7-point mAP gain over the baseline in the paper's ablation.","Learnable graph-based association contributes beyond spatial proximity: adding it on top of multi-dimensional features and alignment brings a 1.4-point AMOTA gain in the ablation.","The method is not limited to vehicle-infrastructure settings: on the Griffin aerial-ground dataset it outperforms the previous end-to-end method by 6.0 mAP and 3.2 AMOTA.","Performance scales with backbone strength: replacing the default backbone with a newer convolutional backbone raises mAP by 2.3 points and AMOTA by 10.1 points on V2X-Seq."],"supporting_citations":[{"why":"The previous end-to-end cooperative tracking method that CoopTrack must beat; its rule-based association and fusion-before-decoding baseline is the direct comparison.","marker":"[71]"},{"why":"Supplies the V2X-Seq sequential vehicle-infrastructure dataset and the tracking-by-cooperative-detection baselines used for evaluation.","marker":"[70]"},{"why":"The query-based detection architecture whose decoder and image features CoopTrack reuses inside its Multi-Dimensional Feature Extraction.","marker":"[23]"},{"why":"Source of the temporal transformer block that gives CoopTrack's instance features their historical context.","marker":"[34]"},{"why":"Provides the latent linear alignment formulation and segmented mapping used by Cross-Agent Alignment.","marker":"[8]"},{"why":"The Hungarian algorithm used both to pair instances for aggregation and to generate the second-stage association labels.","marker":"[17]"},{"why":"Inspiration for decoupled task-dependent queries and differentiable association, which the learnable association design builds on.","marker":"[7]"},{"why":"The continuous 6D rotation representation used to convert the relative pose rotation into a learnable latent rotation.","marker":"[78]"},{"why":"The graph-transformer attention mechanism that the Graph-Based Association module adapts for instance affinity.","marker":"[6]"},{"why":"Supplies the Griffin aerial-ground cooperative dataset used to validate transfer beyond vehicle-infrastructure settings.","marker":"[46]"}],"fun_headline_variants":["CoopTrack: end-to-end cooperative 3D tracking with learned association","Cooperative tracking goes end-to-end with instance-level features","CoopTrack sets SOTA on V2X-Seq with efficient cooperative tracking","Instance-level fusion boosts cooperative 3D tracking to 39 mAP","CoopTrack: low-cost cooperative tracking via graph-based association"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The second training stage relies on association labels generated by matching first-stage tracker predictions to ground truth with the Hungarian algorithm; if those predictions are noisy, the learned association module is trained on wrong correspondences, and the paper does not quantify how much label noise is present.","fun_headline_variants_meta":{"raw":{"variants":["CoopTrack: end-to-end cooperative 3D tracking with learned association","Cooperative tracking goes end-to-end with instance-level features","CoopTrack sets SOTA on V2X-Seq with efficient cooperative tracking","Instance-level fusion boosts cooperative 3D tracking to 39 mAP","CoopTrack: low-cost cooperative tracking via graph-based association"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1170,"prompt_tokens":949,"completion_tokens":221,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":127}},"tokens_in":565,"tokens_out":221,"duration_ms":3032,"temperature":1.0,"reasoning_tokens":127,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:56:32.432821+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Corrupt the first-stage tracker's predicted boxes with calibrated noise before generating the second-stage association labels and measure CoopTrack's AMOTA; if a small amount of added noise causes a large performance drop, the reported gains depend on clean pseudo-labels rather than on the learned association mechanism itself.","supporting_citations":[{"cited_title":"End-to-end autonomous driving through v2x cooperation","cited_arxiv_id":null,"evidence_quote":"The previous end-to-end cooperative tracking method that CoopTrack must beat; its rule-based association and fusion-before-decoding baseline is the direct comparison."},{"cited_title":"Bevformer: 9 learning bird’s-eye-view representation from lidar-camera via spatiotemporal transformers","cited_arxiv_id":null,"evidence_quote":"The query-based detection architecture whose decoder and image features CoopTrack reuses inside its Multi-Dimensional Feature Extraction."},{"cited_title":"Standing between past and future: Spatio-temporal modeling for multi-camera 3d multi- object tracking","cited_arxiv_id":null,"evidence_quote":"Source of the temporal transformer block that gives CoopTrack's instance features their historical context."},{"cited_title":"Star- track: Latent motion models for end-to-end 3d object track- ing with adaptive spatio-temporal appearance representa- tions","cited_arxiv_id":null,"evidence_quote":"Provides the latent linear alignment formulation and segmented mapping used by Cross-Agent Alignment."},{"cited_title":"Ada-track: End-to-end multi-camera 3d multi-object tracking with alternating detection and association","cited_arxiv_id":null,"evidence_quote":"Inspiration for decoupled task-dependent queries and differentiable association, which the learnable association design builds on."},{"cited_title":"3dmotformer: Graph trans- former for online 3d multi-object tracking","cited_arxiv_id":null,"evidence_quote":"The graph-transformer attention mechanism that the Graph-Based Association module adapts for instance affinity."}],"review_version":1}