{"id":"d81c8e01-4898-4215-b9c7-b511092ba019","arxiv_id":"2411.16799","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PolyInter is a single prompt-conditioned feature interpreter that adapts to new heterogeneous perception agents by fine-tuning only a per-agent prompt, reporting higher AP than PnPDA and MPDA on OPV2V.","lead":"PolyInter uses one shared neural interpreter with a small per-agent prompt to align features from different vehicle perception networks, so new car types join by learning only their own prompt. The method reports gains over two prior interpreters on collaborative 3D detection benchmarks, but the headline improvement numbers are hard to verify from the paper's own tables.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Generalization to new agent types is not demonstrated: all Phase II LiDAR agents are variants of Phase I families, and the '+' rows retest a Phase I agent.","rationale":"The reader identified the central weakness as the limited generality of the Phase II test agents, and I agree that this is the load-bearing issue. My analysis sharpens it in two ways. First, the '+' rows in Table 2 are confounded because pp4 appears in the Phase I combination pp8-pp4-vn4, making that condition a test of seen-agent retention rather than novel-agent generalization. Second, the channel selection module's frozen projection weights provide a concrete mechanism by which an architecturally novel encoder could fail to be interpreted correctly, because the trainable prompt is applied after the similarity-based channel reorganisation and cannot correct a wrong selection. The supplementary multi-modal experiments with EfficientNet and ResNet partially mitigate the concern by demonstrating cross-family transfer between two 2D CNN image backbones, but they are not part of the headline LiDAR claims and do not cover a radically different LiDAR encoder. These considerations do not invalidate the core idea, but they do mean the paper's extensibility claim is only weakly evidenced as presented. A targeted held-out-backbone experiment would resolve the question. The reader's conditional verdict remains appropriate: the paper should be accepted only if such a test confirms that prompt-only fine-tuning transfers to a truly novel architecture family.","tokens_in":16884,"tokens_out":15107,"duration_ms":139657,"concrete_test":"Use the Phase I base pp8-vn4-sd2 only (which contains no pp4, sd1 or vn6), then introduce a LiDAR encoder from a genuinely different architecture family, e.g., a sparse 3D transformer such as VoTr or a range-view CNN, as the new neighbor. Fine-tune only the new specific prompt and resizer for this agent under the Phase II protocol, and compare the resulting AP@0.7 with (a) full fine-tuning of the entire PolyInter on the same agent, and (b) the best PnPDA/MPDA baseline. If the prompt-only result falls more than 5 AP points below full fine-tuning, the claim that the shared interpreter learns a general alignment mechanism rather than memorising Phase I encoders is falsified; the same experiment should also be repeated with the '+' rows removed so that no Phase I agent reappears as a 'new' test agent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that a single PolyInter interpreter trained in Phase I transfers to genuinely new agent types with only specific-prompt fine-tuning is not supported by the main experiments. In Section 4.1 the Phase II test agents pp4, sd1 and vn6 are resolution variants of the three backbone families (PointPillar, SECOND, VoxelNet) already present in the Phase I combinations pp8-vn4-sd2 and pp8-pp4-vn4. The '+' rows in Table 2 are particularly problematic: when the Phase I combination is pp8-pp4-vn4, the test agent pp4 was already a Phase I neighbor, so that row measures retained performance on a seen encoder, not generalization to a new type. The frozen channel selection module (Eq. 2-3) applies W_q and W_k learned on Phase I encoders to compute the similarity matrix M for a new encoder; the trainable specific prompt is added only after M has reorganised the channels (Eq. 4), so it cannot repair a mis-selection of semantically mismatched channels. This mechanism-level limitation is partially tested in the supplementary multi-modal experiments (Table 3), which swap EfficientNet and ResNet image encoders and show cross-family transfer, but those are both 2D CNN backbones, the gains are smaller, and they are excluded from the abstract's headline. Consequently, the paper overclaims architectural extensibility: prompt-only fine-tuning has been shown only for within-family resolution changes and for two related image CNN families.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PolyInter, a polymorphic feature interpreter for immutable heterogeneous collaborative perception. A single interpreter network is trained in Phase I on a small set of neighbor encoders, with a shared general prompt and per-agent specific prompts; in Phase II, new agents are added by fine-tuning only the specific prompt and a resizer. Experiments on OPV2V (and in the supplement V2XSet, DAIR-V2X, and multi-modal settings) compare against PnPDA and MPDA, reporting improved AP@0.5/0.7 and reduced trainable parameters.","tokens_in":17323,"tokens_out":7241,"duration_ms":57064,"significance":"If the extensibility claim holds, PolyInter is a practically valuable contribution: it reduces the cost of adding new agent types to a collaborative perception system and avoids the cumulative loss of two-stage interpreters. The paper provides reproducible code, external datasets, ablations of each module, and parameter-count analysis. The central open question is whether prompt-only fine-tuning transfers to architecturally novel encoders, which the current experiments do not establish.","major_comments":[{"comment":"The claim of extensibility to new agent types is not supported by the experimental design. In Phase I combination pp8-vn4-sd2, the Phase II agents pp4, sd1 and vn6 are resolution variants of the PointPillar, SECOND and VoxelNet families already present in Phase I (Table 1); in combination pp8-pp4-vn4, pp4 is literally a Phase I neighbor, so the '+' row for pp8-pp4 in Table 2 measures retained performance on a seen encoder rather than generalization. To substantiate the headline claim, the authors should evaluate on architecturally novel encoders not seen in Phase I, or explicitly restrict the claim to within-family adaptation.","section":"Sec. 4.1, Table 2"},{"comment":"Because the channel selection module computes the similarity matrix M using W_q and W_k learned only on Phase I encoders and then reorganizes both the neighbor feature and the specific prompt with the same M (Eqs. (2)-(4)), the specific prompt cannot compensate for a semantically mismatched channel selection on a truly new architecture. The current experiments never expose this risk, since all Phase II LiDAR encoders belong to the three Phase I families. Please provide an experiment with a distinct architecture (e.g., a different 3D backbone family or a camera-only encoder) or an analysis demonstrating that the channel correspondence learned in Phase I transfers.","section":"Sec. 3.2, Eqs. (1)-(4)"},{"comment":"The reported improvements '7.9% and 11.1%' (Abstract) and '9.2%' (Sec. 4.2) do not match the AP values in Table 2. For example, in pp8-pp4* with F-cooper, AP@0.7 is 60.9 vs 51.1 (PnPDA) and 53.1 (MPDA), giving improvements of 9.8 and 7.8 points, not 9.2; the largest AP@0.7 gain in the table is 24.5 points (pp8-sd1* with F-cooper vs MPDA). Please specify exactly how each claimed number is computed (which scenarios, which baseline, absolute or relative), and report variance across runs or seeds.","section":"Abstract and Sec. 4.2"},{"comment":"The comparison set includes only PnPDA and MPDA, although the related work cites other heterogeneous collaborative perception methods (e.g., HEAL [23], Hetecooper [28], V2X-ViT [34]). If these are not directly comparable because they are not immutable-feature interpreters, state this explicitly; otherwise the 'SOTA interpreter' claim is not supported. Adding at least one recent comparable method would strengthen the empirical claim.","section":"Sec. 4.2"}],"minor_comments":[{"comment":"The caption should clarify that '+' rows use the pp8-pp4-vn4 Phase I combination, in which pp4 is a Phase I neighbor, and that '*' rows use pp8-vn4-sd2; this distinction is easy to miss.","section":"Table 2"},{"comment":"In the adversarial loss equation, both expectation terms are written with F_g as the argument; if one term is intended to be evaluated on F_ego, this is a typo and should be corrected.","section":"Eq. (8)"},{"comment":"The statement that 0.16M parameters is '1.4% of the respective interpreters' would be clearer if the total interpreter parameter count were reported in the main text.","section":"Sec. 4.3"},{"comment":"The column header 'V oxel' and the formatting of 'Half Lidar Range (x,y)' appear inconsistent; the table should be cleaned up.","section":"Table 1"},{"comment":"Please clarify whether the parameter counts for PnPDA and MPDA include the fusion module and the standard-semantic-space interpreters, so the comparison is apples-to-apples.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's PnPDA baseline [25] shares authors with this submission; the comparison is implemented in the same lab, so a neutral third-party implementation or public code would increase confidence. I also note that the abstract's improvement numbers are not derived from Table 2, which should be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is a decent architecture proposal for immutable heterogeneous collaborative perception: one shared interpreter network plus per-agent prompts, so new agent types are added by training only a prompt and a resizer. The adaptation of visual prompt tuning and LoRA to this setting is clean and the two-phase training is well motivated. The ablations are thorough and the code link is a plus. The supplementary multi-modal experiment (EfficientNet to ResNet and vice versa) is the strongest evidence that the idea generalizes across genuinely different encoder families, but it is confined to the supplement and omitted from the abstract's headline. The soft spots are real and not minor. First, the headline numbers do not match Table 2. The abstract's claim of up to 11.1% improvement does not line up with the AP values in the table, and Section 4.2's 9.2% is also hard to reproduce from the pp8-pp4* F-cooper row. The authors need to state whether these are absolute or relative, averaged or best-case, and then correct the text. Second, the Phase II generalization experiments are weaker than advertised. The test agents pp4, sd1, and vn6 are all resolution variants of the same three backbone families (PointPillar, SECOND, VoxelNet) used in Phase I, not architecturally new encoders. The '+' rows are particularly problematic: when Phase I used pp8-pp4-vn4, pp4 is already a seen neighbor, so that row measures retention, not generalization. The cross-family evidence exists only in the supplementary, uses two related 2D CNNs, and shows smaller gains. Third, there are no error bars or multiple seeds, so significance is unknown. Fourth, the baseline set is narrow, only PnPDA and MPDA, so the comparison is not current. A mechanism-level concern also deserves attention: the channel selection module is frozen in Phase II, and the specific prompt is added after the similarity matrix M has already reorganized channels. Prompt tuning therefore cannot correct a mis-selection of semantically mismatched channels for a truly unseen encoder. The paper does not address this directly. None of this kills the core idea; the shared-interpreter-plus-prompts design is a useful engineering contribution and I would like to see it tested more rigorously. I would send this to peer review; the reviewers should ask for cross-family LiDAR experiments (e.g., a transformer-based encoder), corrected numerical reporting, and variance estimates. If the authors can show transfer to a genuinely new LiDAR backbone, the claim is much stronger. As written, it is a conditional accept at best.","headline":"A solid prompt-tuning architecture for heterogeneous collaborative perception, but the abstract's numbers don't match the tables and the generalization claim is only tested with resolution variants of the same backbone families.","tokens_in":789,"tokens_out":1030,"would_cite":true,"duration_ms":46976,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single interpreter can map any neighbor's features into the ego vehicle's semantic space.","keywords":["collaborative perception","immutable heterogeneity","feature interpreter","prompt tuning","polymorphism","autonomous driving","3D object detection","OPV2V"],"falsifier":"Take a neighbor encoder whose architecture was not seen in Phase I (for example, a transformer-based LiDAR backbone rather than a resolution variant of PointPillar, VoxelNet, or SECOND), fine-tune only its specific prompt, and measure AP@0.7 on OPV2V against retraining the full interpreter on that agent type; if the prompt-only result falls materially below the retrained interpreter while resolution-variant prompts do not, the extensibility claim is refuted.","tokens_in":16666,"feed_emoji":"🚗","tokens_out":6732,"duration_ms":62502,"temperature":0.7,"pith_summary":"The paper addresses an openness problem in collaborative perception: when vehicles with different, fixed perception networks share intermediate features, those features are not mutually understandable, and prior solutions either retrain a full interpreter for every new agent type or pay a two-stage semantic-loss penalty through a standard semantic space. PolyInter is proposed as a single polymorphic feature interpreter that maps any neighbor's features into the ego vehicle's semantic space in one stage, using one shared interpreter network plus a general prompt and per-agent specific prompts. New agent types are integrated by fine-tuning only their specific prompt and a small resizer, with the interpreter network and general prompt frozen. On the OPV2V dataset the paper reports precision improvements of up to 11.1% AP@0.7 over existing interpreters while training as little as 1.4% of PolyInter's parameters when adapting to a new agent.","feed_headline":"One interpreter serves every agent type in collaborative perception","feed_subtitle":"New agents join by fine-tuning only a small prompt, with precision gains up to 11.1 percent.","key_machinery":"PolyInter's load-bearing object is the polymorphic interpreter: one interpreter network plus a shared general prompt and a stack of agent-specific prompts, all hosted on the ego agent. The interpreter network is built from a Channel Selection Module, which computes a cosine-similarity matrix between ego and neighbor BEV feature channels and reweights the neighbor feature accordingly, and a Spatial Attention Module, which applies fused axial attention to align positions into the ego semantic space. The prompts are what make the design polymorphic: the general prompt G is shared by all agents and is trained to encode cross-agent commonalities through an adversarial loss and a style loss, while each specific prompt S_i is trained with a single-agent detection loss to retain encoder-specific semantics. Phase I trains the interpreter and all prompts jointly with a collaborative detection loss; Phase II freezes everything except the new agent's specific prompt and a resizer, and the specific prompt can be low-rank decomposed to reduce trainable parameters to about 1.4% of the interpreter.","core_discovery":"At its center is the claim that semantic alignment across immutable heterogeneous encoders can be factored into a shared alignment mechanism plus a tiny per-agent identity. The shared mechanism is learned once: a channel selection module matches neighbor feature channels to ego channels by cosine similarity, and a spatial attention module aligns spatial semantics by fused axial attention. The per-agent identity lives in learnable prompts: a general prompt is trained to capture common semantics via adversarial and style losses, while each specific prompt is trained to preserve agent-specific semantics via a single-agent detection loss. In the generalization phase, only the new agent's specific prompt and a resizer are optimized, and the paper shows on OPV2V (with supplementary results on V2XSet and DAIR-V2X) that this prompt-only adaptation reaches or exceeds the accuracy of interpreters that are fully retrained per agent pair.","pith_inferences":["The paper does not test architecturally novel backbones, since Phase II agents (pp4, sd1, vn6) are resolution and layer-count variants of the same PointPillar, SECOND, and VoxelNet families used in Phase I; a fair reader should treat transfer to genuinely new architectures as an open prediction rather than an established result.","If prompts encode a compact agent identity, then interpolating or averaging the prompts of known agents may produce a usable prompt for an unseen encoder without any fine-tuning, which would be a cheap testable extension.","The adversarial training that strips agent-specific statistics from the general feature suggests the same interpreter could double as a privacy filter, removing identifying encoder statistics from shared features; that use is not evaluated here.","The storage model of one interpreter plus many prompts raises a capacity question: at some number of agent types, prompt interference may degrade accuracy, and the paper gives no evidence about that ceiling."],"forward_implications":["A fleet can integrate a new vehicle model by storing one extra prompt per agent type instead of one interpreter per ego-neighbor pair, reducing storage and adaptation cost.","Because interpretation is one-stage, it avoids the cumulative semantic loss that two-stage standard-space methods incur, so the accuracy gap over those methods should widen as the semantic gap between encoders grows.","Prompts for previously integrated agents remain untouched when a new agent is added, so the interpreter's behavior for existing collaboration partners is not disturbed.","The low-rank prompt decomposition offers a direct trade-off between on-board parameter budget and perception precision, making prompt-only adaptation feasible under memory constraints."],"supporting_citations":[{"why":"Supplies the one-stage and two-stage interpreter baselines whose precision PolyInter is compared against; removing it removes the main accuracy comparison.","marker":"[25]"},{"why":"Provides the other interpreter baseline for immutable heterogeneous collaboration that PolyInter must beat.","marker":"[36]"},{"why":"Supplies the OPV2V dataset and evaluation protocol used for the main experiments.","marker":"[35]"},{"why":"Defines the PointPillar encoder family used for the ego agent and several neighbor variants.","marker":"[16]"},{"why":"Defines the VoxelNet encoder family whose resolution variants appear as neighbors in both phases.","marker":"[44]"},{"why":"Defines the SECOND encoder family used as a Phase I neighbor and as the Phase II test agent sd1.","marker":"[38]"},{"why":"Supplies the visual-prompt-tuning idea that motivates learnable prompts as the extension point.","marker":"[15]"},{"why":"Provides the low-rank decomposition used to shrink the specific prompt's trainable parameters.","marker":"[12]"}],"fun_headline_variants":["One interpreter fits all agents in collaborative perception","Single AI interpreter adapts to any agent with tiny prompts","Polymorphic interpreter: one model, many agent types","Prompt-only tuning lets one interpreter serve all agents","11.1% precision gain with one interpreter for all"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The shared interpreter network trained in Phase I generalizes to genuinely new agent types after only prompt fine-tuning; the Phase II tests use resolution variants of the same backbone families seen in Phase I, so the experiments do not yet establish transfer across architecturally novel encoders.","fun_headline_variants_meta":{"raw":{"variants":["One interpreter fits all agents in collaborative perception","Single AI interpreter adapts to any agent with tiny prompts","Polymorphic interpreter: one model, many agent types","Prompt-only tuning lets one interpreter serve all agents","11.1% precision gain with one interpreter for all"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1225,"prompt_tokens":937,"completion_tokens":288,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":212}},"tokens_in":553,"tokens_out":288,"duration_ms":5726,"temperature":1.0,"reasoning_tokens":212,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:16:08.116086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a neighbor encoder whose architecture was not seen in Phase I (for example, a transformer-based LiDAR backbone rather than a resolution variant of PointPillar, VoxelNet, or SECOND), fine-tune only its specific prompt, and measure AP@0.7 on OPV2V against retraining the full interpreter on that agent type; if the prompt-only result falls materially below the retrained interpreter while resolution-variant prompts do not, the extensibility claim is refuted.","supporting_citations":[{"cited_title":"Plug and play: A representation en- hanced domain adapter for collaborative perception","cited_arxiv_id":null,"evidence_quote":"Supplies the one-stage and two-stage interpreter baselines whose precision PolyInter is compared against; removing it removes the main accuracy comparison."},{"cited_title":"Bridging the domain gap for multi-agent percep- tion","cited_arxiv_id":null,"evidence_quote":"Provides the other interpreter baseline for immutable heterogeneous collaboration that PolyInter must beat."},{"cited_title":"OPV2V: an open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communica- tion","cited_arxiv_id":null,"evidence_quote":"Supplies the OPV2V dataset and evaluation protocol used for the main experiments."},{"cited_title":"Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom","cited_arxiv_id":null,"evidence_quote":"Defines the PointPillar encoder family used for the ego agent and several neighbor variants."},{"cited_title":"V oxelnet: End-to-end learning for point cloud based 3d object detection","cited_arxiv_id":null,"evidence_quote":"Defines the VoxelNet encoder family whose resolution variants appear as neighbors in both phases."},{"cited_title":"SECOND: sparsely em- bedded convolutional detection.Sensors, 18(10):3337, 2018","cited_arxiv_id":null,"evidence_quote":"Defines the SECOND encoder family used as a Phase I neighbor and as the Phase II test agent sd1."},{"cited_title":"Belongie, Bharath Hariharan, and Ser-Nam Lim","cited_arxiv_id":null,"evidence_quote":"Supplies the visual-prompt-tuning idea that motivates learnable prompts as the extension point."}],"review_version":1}