{"id":"ac38c3a8-bb14-4bcf-a239-72403439ce78","arxiv_id":"2505.18039","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Distilling CLIP into EfficientNet-B3 with projection heads yields a 24.6 MB model that performs zero-shot labeling on driving scenes in real time on an edge camera.","lead":"Clip4Retrofit shrinks OpenAI's CLIP model into a small EfficientNet-based model that runs on a low-cost car camera and labels images in about 35 milliseconds. The compressed model keeps much of CLIP's zero-shot ability, so it can recognize objects it was never explicitly trained on, which matters for autonomous driving and data collection.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The deployed student embedding is never directly trained against text, and removing the GL projector may discard the only representation that was aligned with CLIP text embeddings, so the zero-shot labeling claim is unsupported.","rationale":"The reader's weakest_assumption already identifies the core concern: the student is never trained against text, and removing the PCA/GL projectors may break the transitive alignment with CLIP text embeddings. My stress-test sharpens this into a more precise mechanistic objection: the GL loss in Eq. 5 aligns only h'_S = FC(h_S), not h_S, and Section 3.3.3 explicitly removes the GL projector before deployment. This makes the problem structural rather than a matter of 'perfect' projector removal. I could not find any evidence in the manuscript that the deployed representation is ever compared with text embeddings directly, nor are the text prompts used for the reported zero-shot ROC-AUC specified. The proposed concrete test would settle the issue by measuring text-based zero-shot accuracy before and after projector removal. Since the reader's verdict is already CONDITIONAL and this concern is exactly the condition that needs to be resolved, I recommend keeping the verdict unchanged rather than escalating to rejection or unverdictable status.","tokens_in":9471,"tokens_out":3761,"duration_ms":33823,"concrete_test":"Run a direct text-image alignment check on the deployed model with PCA/GL projectors removed. Take the same validation images used in Table 1, compute student embeddings, and compute cosine similarity against CLIP text embeddings for the eight class names and simple prompts such as 'a photo of a truck'. Report per-class ROC-AUC for this text-based zero-shot test, and repeat the same test using the student embeddings before projector removal (i.e., with h'_S available during training). If the post-removal text-based AUC drops to near chance while the pre-removal AUC is high, the projector removal destroys the only aligned representation and the central claim fails. Also report the distribution of cosine similarities between student and CLIP text embeddings to verify that they occupy the same space.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the deployed EfficientNet-B3 student produces embeddings that are directly comparable, via cosine similarity, to pre-stored CLIP text embeddings. The training objective does not establish this. In Section 3.3.1, the GL loss (Eq. 5) aligns the projected student feature h'_S = FC(h_S) with the teacher embedding h_T. It does not align h_S itself. Section 3.3.3 states that after training the PCA and GL projectors are removed, leaving the lightweight student model for deployment. If the GL projector is removed, the deployed representation is h_S, which was never constrained to the CLIP text-embedding space by any loss term. If the 3-layer MLP projection head mentioned in Section 3.3 is instead retained, the paper is ambiguous about whether that head is the GL projector, and about which representation is actually deployed. Either way, no experiment demonstrates that the post-removal embeddings yield valid zero-shot text-based classification. The reported ROC-AUC values could in principle arise from image-to-image similarity or from correlated low-level features rather than from genuine alignment with CLIP text embeddings. The conclusion's limitation paragraph does not acknowledge this projector-removal alignment risk, so the paper's own stated limitations do not cover its weakest point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Clip4Retrofit distills a CLIP ViT-L/14 teacher into an EfficientNet-B3 student using PCA and group-wise linear (GL) projectors plus an adversarial multi-view loss, then removes the projectors and deploys the model on the Bosch Retrofit camera for in-car image labeling. The paper reports zero-shot ROC-AUC on an internal Bosch Cars dataset, Cityscapes, and Mapillary Vistas, along with a 35 ms/frame inference time and a 24.6 MB binary model. The central claim is that the distilled student preserves CLIP's zero-shot cross-modal capabilities while running in real time on a 3 eTOPS NPU.","tokens_in":9672,"tokens_out":4322,"duration_ms":35435,"significance":"If the claims are substantiated, the paper offers a practically valuable route to CLIP-style zero-shot labeling on automotive edge hardware: the cross-architecture distillation setup, the concrete deployment pipeline, and the evaluation on public driving benchmarks are genuine strengths. However, the paper does not currently demonstrate the load-bearing property that the deployed student embeddings, after projector removal, remain aligned with CLIP text embeddings. Without that demonstration, the reported AUCs do not establish text-based zero-shot labeling. The experimental section also lacks error bars, full class coverage, and comparisons to other CLIP distillation methods, so the performance claims are not yet conclusive. The paper contains no machine-checked proofs or released code, but the use of public benchmarks and a real hardware deployment is a positive feature.","major_comments":[{"comment":"The central zero-shot claim is not supported by the training objective. Equation (5) aligns h'_S = FC(h_S) with the teacher embedding h_T; it does not align h_S itself. Section 3.3.3 then states that the PCA and GL projectors are removed after training, so the deployed representation is h_S, which was never constrained against CLIP text embeddings. No experiment verifies that post-removal cosine similarity with pre-stored CLIP text vectors yields valid classification, and the conclusion's limitation paragraph does not acknowledge this projector-removal alignment risk. The reported ROC-AUC values could in principle arise from image-to-image similarity or correlated low-level features rather than genuine text alignment. Please either retain the projector and evaluate that deployed configuration, or add a direct text-alignment test on the deployed model, such as zero-shot classification with held-out textual prompts compared against the teacher.","section":"Sections 3.3.1, 3.3.3, 3.4.1"},{"comment":"The quantitative support for 'competitive zero-shot performance' is narrow and partly contradictory. Only 8 of the 34 Cityscapes and 124 Mapillary Vistas classes are reported, no error bars or significance tests are given, and no alternative CLIP distillation baseline is compared. The claim of 'minimal accuracy degradation' is not uniformly supported: on Cityscapes Bicycle, Clip4Retrofit's 0.6692 is 8.2 AUC points below ViT-L/14's 0.7512, while on other classes the student exceeds ViT-B/32. The paper should report all evaluated classes or justify the selection criterion, provide variance estimates, and compare with at least one prior CLIP distillation method to contextualize the trade-off.","section":"Section 4.2 and Table 1"},{"comment":"The evaluation protocol for the public datasets is underspecified. The paper states that semantic mask annotations are treated as labels for the entire image, but it does not define how class-level positives and negatives are sampled, which text prompt templates are used for zero-shot classification, or how images containing multiple classes are handled. Without these details, the reported ROC-AUC values cannot be reproduced and the comparison with the CLIP baselines is not fully interpretable.","section":"Section 4.1.1"},{"comment":"The real-time claim rests on a single runtime figure of 35 ms per frame. No measurement methodology, number of runs, variance, input preprocessing details, or hardware/software versions are reported, and there is no comparison to the teacher model on any feasible hardware baseline. Since real-time operation is one of the two headline contributions, this claim needs more rigorous support.","section":"Sections 3.4.1 and 3.4.5"}],"minor_comments":[{"comment":"The heading '3.2. Distillation Framework' is followed immediately by '3.3. Model Architecture' with no textual content under 3.2; either fill in the section or renumber the subsections.","section":"Section 3.2"},{"comment":"The abstract and conclusion state that the model 'preserves' CLIP's zero-shot recognition capabilities, but this is not yet demonstrated given the projector-removal concern; rephrase to 'reported' or 'intended' pending the required validation.","section":"Abstract and Conclusion"},{"comment":"Figure captions inconsistently refer to 'Clip Custom' and 'Clip Custom@300', while Table 1 uses 'Clip4Retrofit'; the class names also vary ('bike' in Figure 3 vs 'bicycle' in Figures 4-5). Please standardize the naming.","section":"Section 4.2 and Figures 3-5"},{"comment":"There is a typo in 'approximately35 milliseconds' (missing space), and the Intro has 'summarize our main contributions as follows::' with a double colon.","section":"Section 3.4.5"},{"comment":"The text cites [8] as demonstrating distillation of large vision-language models, but reference [8] is the ALIGN paper on noisy text supervision, not a distillation method; please correct the citation or the claim.","section":"Section 2.2"},{"comment":"The paper mentions a '3-layer MLP projection head' in the architecture description and later removes 'PCA and GL projectors' in Section 3.3.3; clarify whether the MLP head is the same as the GL projector and whether it is retained or removed in the deployed model.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper describes a real industrial deployment, which is a strength, but the missing validation of the deployed embedding's text alignment is a substantive gap that should be addressed before publication. The reported AUC improvements over ViT-B/32 on several classes are interesting, yet the ad hoc class selection and lack of baselines make the current evidence inconclusive. I would not reject on novelty grounds; the topic fits an applied CV journal, but the central claim needs experimental support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a plausible engineering integration with a real deployment target, but the central evidence chain has a gap the paper never closes. The specific combination — CLIP ViT teacher distilled into EfficientNet-B3 using PCA/GL projectors plus adversarial multi-view training — is new relative to the cited literature. The deployment numbers are concrete: 24.6 MB binary, about 35 ms per frame on a 3 eTOPS NPU. That alone is useful. I also credit the evaluation on public benchmarks (Cityscapes, Mapillary) that were not used for training, which keeps the main claim from being circular.\n\nSoft spots, in order of severity. The biggest is what happens after training. The GL loss aligns FC(h_S) with the teacher embedding h_T. The paper then says the PCA and GL projectors are removed, leaving the deployed student. If the GL projector is that FC — or the 3-layer MLP head — then the deployed representation h_S was never directly constrained to the CLIP text embedding space. No experiment shows that post-removal embeddings still match text embeddings via cosine similarity. That is exactly the step needed to support \"zero-shot ROC-AUC\" against text queries. The ambiguity about whether the MLP head is retained makes it impossible to verify from the text. The conclusion's limitations mention general degradation but not this alignment risk.\n\nSecond, the evaluation is thin. AUC is reported for 8 classes per dataset, with no error bars, no repeated runs, no comparison to other CLIP distillation methods, and no prompt template given. On Cityscapes bicycle, the student is 8 points below ViT-L/14, which undercuts \"minimal accuracy degradation.\" On other classes it beats ViT-B/32, so the claim is not uniformly wrong, but the selective reporting makes overall assessment hard.\n\nThird, there is no code or data release, and the internal Bosch dataset is not described in enough detail to reproduce. That matters less because public benchmarks exist, but it still limits verification.\n\nWho is this for? Someone building a real edge labeling system who wants a starting recipe. It is not a reliable source for the zero-shot claim as written.\n\nRecommendation: send to peer review, but with a sharp referee who asks for (a) a clear statement of which head is deployed, (b) a text-pairing sanity check (e.g., zero-shot classification on standard CLIP benchmarks), and (c) error bars and a broader class set. The core integration is worth engaging; the evidence is not yet there.","headline":"Solid engineering integration, but the zero-shot claim rests on an unverified projector-removal step and thin evaluation; worth a referee but not yet citable.","tokens_in":10255,"tokens_out":2181,"would_cite":false,"duration_ms":16502,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 24.6 MB distilled CLIP model labels driving images in real time on an edge camera.","keywords":["CLIP distillation","edge AI","zero-shot classification","EfficientNet-B3","cross-architecture distillation","autonomous driving","real-time image labeling"],"falsifier":"On a held-out set of driving images, compute the cosine similarity between the deployed student's image embeddings and CLIP text embeddings for class names such as 'truck' and 'tunnel', and measure zero-shot ranking accuracy; if removing the projectors shifts the embeddings so that text-image similarity drops to near chance across multiple classes, the zero-shot pipeline breaks.","tokens_in":9216,"feed_emoji":"🚗","tokens_out":4659,"duration_ms":36058,"temperature":0.7,"pith_summary":"This paper claims that a full CLIP vision-language model can be compressed into a 24.6 MB convolutional student that runs in about 35 ms per frame on a low-power automotive camera, while keeping zero-shot labeling accuracy close to the original. The approach is cross-architecture distillation: an EfficientNet-B3 backbone learns to mimic the CLIP teacher's image embeddings through projection heads that are removed after training, leaving a small model that still lands in CLIP's shared image-text space. If the claim is right, thousands of deployed vehicles could label their own camera images in real time, avoiding cloud latency and cutting the cost of manual annotation. The paper reports zero-shot ROC-AUC on urban driving classes that is competitive with, and sometimes better than, the original CLIP ViT-B/32 and ViT-L/14 baselines.","feed_headline":"Distilled CLIP labels images in 35 ms on a low-power edge camera","feed_subtitle":"EfficientNet-B3 inherits CLIP's zero-shot text alignment, then runs fully onboard with no cloud round-trip.","key_machinery":"The machinery is a two-stage cross-architecture distillation loop. During training, PCA and GL projectors map EfficientNet's convolutional feature maps onto the teacher ViT's attention and embedding spaces, and a discriminator pushes the student's features toward the teacher's distribution under random image distortions. After training, both projectors are removed; the deployed model is just EfficientNet-B3 plus its MLP head, relying on the claim that the learned mapping is internalised so cosine similarity against pre-stored CLIP text vectors still works.","core_discovery":"The central discovery is that the cross-modal alignment of CLIP can be transferred to a lightweight convolutional student without ever training the student on text. The student is trained only against the teacher's image embeddings, using a Partially Cross-Attention (PCA) projector, a Group-wise Linear (GL) projector, and a small adversarial discriminator, so that after the projectors are discarded the EfficientNet-B3 output still sits in the CLIP embedding space. The paper reports that this distilled model, quantized and compiled for the Retrofit camera, runs at approximately 35 ms per frame on a 3 eTOPS NPU and achieves zero-shot labeling performance close to the original CLIP models on Cityscapes and Mapillary Vistas, with gains on some classes such as 'truck' and 'guard rail'.","pith_inferences":["The paper does not verify post-removal alignment of the student's embeddings with CLIP text embeddings; a direct text-contrastive probe is the natural next experiment, and the paper's own reported weaknesses on small or distant objects suggest where it would fail.","Because the student is never trained against text, the framework inherits CLIP's domain blind spots (e.g., rare traffic signs); a lightweight on-device contrastive fine-tune with a handful of text prompts could repair those spots without blowing the 35 ms budget.","If the transitive alignment truly survives projector removal, the recipe generalizes beyond CLIP to any frozen multi-modal teacher, which would make retrofitted edge cameras a general zero-shot labeling platform rather than a CLIP-specific trick.","The paper's adoption of DINOv2-style self-supervised retrieval for data curation implies the distillation quality is coupled to the curation pipeline; a test that varies the curator while keeping the student fixed would isolate how much performance comes from distillation versus data selection."],"forward_implications":["A 24.6 MB binary model on a 3 eTOPS NPU can label 1080p driving frames at about 28 frames per second, meeting onboard real-time constraints without any cloud round-trip.","Deployed camera fleets could perform continuous self-labeling of newly collected driving data, reducing the cost and delay of manual annotation for downstream fine-tuning.","The same distillation recipe can be applied to other CLIP-scale vision-language teachers, provided the teacher's image embedding space is stable and accessible.","Zero-shot ROC-AUC on classes like 'guard rail' and 'truck' exceeds the ViT-B/32 baseline on the tested datasets, showing the student is not merely a uniform low-cost approximation.","Quantization to INT16 and operator replacement during ONNX conversion preserve enough numeric fidelity to keep zero-shot ranking intact at deployment."],"supporting_citations":[{"why":"Supplies the teacher model whose image-text embedding space the student is trained to match; the zero-shot pipeline's text vectors come from it.","marker":"[18]"},{"why":"Provides the Partially Cross Attention and Group-wise Linear projectors that the paper adapts for cross-architecture distillation.","marker":"[11]"},{"why":"Defines the EfficientNet family; EfficientNet-B3 is the student backbone chosen after comparing it with B4, B5, and Light-ViT.","marker":"[20]"},{"why":"Cityscapes public benchmark, used for zero-shot ROC-AUC evaluation with semantic class labels as whole-image labels.","marker":"[1]"},{"why":"Mapillary Vistas v2.0 public benchmark, used to test generalization of the distilled model beyond the internal driving dataset.","marker":"[14]"},{"why":"Inspires the self-supervised retrieval and clustering pipeline used to curate the internal training dataset for distillation.","marker":"[15]"}],"fun_headline_variants":["CLIP zero-shot distilled to 35ms edge camera labeling","Retrofit distillation brings CLIP to onboard cameras in 35ms","Edge CLIP: EfficientNet-B3 inherits text alignment without text training","35ms per frame: CLIP knowledge distilled onto resource-limited NPU","Distilling CLIP's multimodal brain into a low-power camera chip"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that after the PCA and GL projectors are stripped away, the frozen student's image embeddings remain aligned with CLIP's text embeddings, so zero-shot labeling by cosine similarity keeps working; the paper never trains the student against text or directly verifies this alignment post-removal.","fun_headline_variants_meta":{"raw":{"variants":["CLIP zero-shot distilled to 35ms edge camera labeling","Retrofit distillation brings CLIP to onboard cameras in 35ms","Edge CLIP: EfficientNet-B3 inherits text alignment without text training","35ms per frame: CLIP knowledge distilled onto resource-limited NPU","Distilling CLIP's multimodal brain into a low-power camera chip"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000191,"raw_usage":{"total_tokens":1355,"prompt_tokens":968,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":293}},"tokens_in":584,"tokens_out":387,"duration_ms":3511,"temperature":1.0,"reasoning_tokens":293,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:35:42.979257+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out set of driving images, compute the cosine similarity between the deployed student's image embeddings and CLIP text embeddings for class names such as 'truck' and 'tunnel', and measure zero-shot ranking accuracy; if removing the projectors shifts the embeddings so that text-image similarity drops to near chance across multiple classes, the zero-shot pipeline breaks.","supporting_citations":[{"cited_title":"Cross-architecture knowledge distillation","cited_arxiv_id":null,"evidence_quote":"Provides the Partially Cross Attention and Group-wise Linear projectors that the paper adapts for cross-architecture distillation."},{"cited_title":"Efficientnet: Rethinking model scaling for convolutional neural networks","cited_arxiv_id":null,"evidence_quote":"Defines the EfficientNet family; EfficientNet-B3 is the student backbone chosen after comparing it with B4, B5, and Light-ViT."},{"cited_title":"The mapillary vistas dataset for semantic understanding of street scenes","cited_arxiv_id":null,"evidence_quote":"Mapillary Vistas v2.0 public benchmark, used to test generalization of the distilled model beyond the internal driving dataset."}],"review_version":1}