{"id":"94e0c7d1-62e7-45b7-ac71-559771ab342b","arxiv_id":"2606.19627","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"VCG uses domain-adapted CLIP embeddings for zero-shot multimodal retrieval in e-commerce video feeds, achieving 50% uplift in deep video completion in A/B tests.","lead":"The paper introduces the Video Candidate Generation (VCG) system, a multimodal retrieval engine that uses adapted CLIP models to recommend new videos in e-commerce feeds without relying on user interaction history. This approach could help address the challenges of cold-start problems and biases in dynamic video-based shopping platforms.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No evidence given that domain-adapted CLIP produces a usable shared space for zero-shot cold-start retrieval","rationale":"The reader's weakest assumption is exactly the unverified step required for the central claim; the abstract supplies no counter-evidence, so the concern stands.","tokens_in":1697,"tokens_out":247,"duration_ms":9360,"concrete_test":"On a held-out set of new videos, compute top-50 retrieval precision using only the adapted CLIP embeddings against a random baseline and against a simple visual-feature baseline; if precision stays within 2% of random, the shared-space claim does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline 50% uplift is claimed to result from VCG's multimodal retrieval that works without behavioral history. The abstract states that a domain-adapted CLIP maps users and videos into one space and that generative embeddings collapse, yet supplies zero quantitative support: no recall@K, no embedding alignment statistics, no cold-start ablation, and no comparison of visual-only retrieval against position/duration-biased baselines. Without those numbers the attribution of the A/B result to the claimed mechanism remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper presents the Video Candidate Generation (VCG) system, a scalable multimodal retrieval framework for e-commerce short-form video feeds that addresses extreme cold-start by employing a domain-adapted CLIP-based vision-language model to embed users and videos into a shared semantic space for zero-shot visual-content retrieval. It compares generative LLM embeddings against discriminative CLIP embeddings (noting collapse in the former for retrieval), details the architecture, reports online A/B testing results claiming a 50% uplift in deep video completion due to bias mitigation, and includes an interactive demo for Product-to-Video, Video-to-Product, and Zero-Shot Semantic Search scenarios.","tokens_in":1817,"tokens_out":573,"duration_ms":30028,"significance":"If the A/B results prove robust and the zero-shot multimodal mechanism is shown to outperform behavioral-history baselines while controlling for position/duration biases, the work would be significant for industrial recommendation systems facing cold-start in immersive video feeds; the explicit comparison of generative vs. discriminative embeddings and the bi-directional retrieval demo are also potentially useful contributions.","major_comments":[{"comment":"Abstract and evaluation sections: the central claim of a 50% uplift in deep video completion from online A/B testing is presented without any description of the experimental design, including test/control group sizes, duration of the test, statistical significance testing, baseline system details, or explicit controls for position and duration biases; this renders the attribution of the uplift to the VCG multimodal zero-shot mechanism unverifiable from the manuscript.","section":"Abstract / Evaluation"},{"comment":"Architecture and results sections: the claim that the domain-adapted CLIP model successfully maps users and videos into a shared semantic space enabling effective zero-shot retrieval is unsupported by any quantitative evidence such as recall@K, embedding alignment statistics (e.g., cosine similarity distributions), cold-start ablation studies, or comparisons of visual-only retrieval performance against position-biased baselines.","section":"Architecture / Results"},{"comment":"Evaluation section: the statement that generative models suffer from embedding space collapse in retrieval tasks (while excelling at attribute prediction) is asserted without supporting metrics, visualizations of embedding distributions, or quantitative retrieval performance numbers comparing the two embedding types.","section":"Evaluation"}],"minor_comments":[{"comment":"The manuscript would benefit from a dedicated related-work section contrasting VCG against prior cold-start video recommendation approaches that also use multimodal embeddings.","section":null},{"comment":"Notation for the shared semantic space and any loss functions used in domain adaptation of CLIP should be formalized with equations for reproducibility.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"Thank you for your thorough review and constructive feedback. We address each major comment below and will revise the manuscript to improve verifiability and add supporting evidence where the current version is lacking.","responses":[{"response":"We agree that the A/B test description is insufficiently detailed. The revised manuscript will add a dedicated evaluation subsection covering test and control group sizes, experiment duration, statistical significance testing procedures, baseline system configuration, and explicit methods used to control for and measure position and duration biases. This will allow independent verification of the uplift attribution.","revision_made":"yes","referee_comment":"[Abstract / Evaluation] Abstract and evaluation sections: the central claim of a 50% uplift in deep video completion from online A/B testing is presented without any description of the experimental design, including test/control group sizes, duration of the test, statistical significance testing, baseline system details, or explicit controls for position and duration biases; this renders the attribution of the uplift to the VCG multimodal zero-shot mechanism unverifiable from the manuscript."},{"response":"We accept that quantitative validation of the shared embedding space is required. The results section will be expanded to report recall@K for zero-shot retrieval, cosine similarity distributions between user and video embeddings, cold-start ablation results, and direct comparisons of visual-only performance versus position-biased baselines.","revision_made":"yes","referee_comment":"[Architecture / Results] Architecture and results sections: the claim that the domain-adapted CLIP model successfully maps users and videos into a shared semantic space enabling effective zero-shot retrieval is unsupported by any quantitative evidence such as recall@K, embedding alignment statistics (e.g., cosine similarity distributions), cold-start ablation studies, or comparisons of visual-only retrieval performance against position-biased baselines."},{"response":"We will add the missing quantitative support. The revised evaluation will include retrieval performance metrics for both generative and discriminative embeddings, visualizations of embedding distributions demonstrating collapse, and side-by-side numbers contrasting retrieval versus attribute prediction tasks.","revision_made":"yes","referee_comment":"[Evaluation] Evaluation section: the statement that generative models suffer from embedding space collapse in retrieval tasks (while excelling at attribute prediction) is asserted without supporting metrics, visualizations of embedding distributions, or quantitative retrieval performance numbers comparing the two embedding types."}],"tokens_in":1474,"tokens_out":458,"duration_ms":17467,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core claim is that a domain-adapted CLIP model creates a usable shared space for zero-shot video retrieval when behavioral history is absent, and that this drives a 50% lift in deep video completion in live A/B tests. The paper also reports that generative LLM embeddings collapse in the same retrieval setting while the discriminative CLIP version does not.\n\nWhat is actually new is the explicit framing of extreme cold-start plus position and duration biases for short-form video feeds in e-commerce, plus the side-by-side comparison of generative versus discriminative embeddings on that task. The architecture description and the three bi-directional demo scenarios (product-to-video, video-to-product, zero-shot search) are clear enough to understand how the system is meant to run at scale.\n\nThe soft spot is the evaluation. The abstract states the 50% uplift and the embedding-space collapse but gives no recall@K, no alignment statistics between user and video embeddings, no cold-start ablation, and no description of how the A/B test handled position or duration confounds. Without those numbers the attribution of the lift to the multimodal shared space stays unsupported, exactly as the stress-test note flags. The circularity burden is low because no fitted parameters are shown, but that does not make up for the missing quantitative checks.\n\nThis is for recsys practitioners who need to surface new short videos in shopping feeds. A reader who already works on CLIP adaptation or video candidate generation could extract the practical architecture details. The work shows clear thinking about the problem setting and honest engagement with the generative-versus-discriminative distinction, so it is coherent on its own terms.\n\nI would send it to peer review so the authors can supply the missing metrics and ablations; the production claim is large enough to justify referee time if the evidence is there in the full manuscript.","headline":"The paper describes a CLIP-adapted retrieval system for cold-start e-commerce video feeds and claims a 50% A/B uplift, but the abstract supplies no metrics to tie the result to the claimed mechanism.","tokens_in":2350,"tokens_out":457,"would_cite":false,"duration_ms":17712,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A vision-language model maps new videos and users into one space, enabling retrieval without click history and delivering 50% more full video watches.","keywords":["multimodal retrieval","cold start","e-commerce","video recommendation","vision language model","zero shot retrieval"],"falsifier":"Running the A/B test with the model turned off or replaced by a model that ignores visual content and measuring whether the 50% uplift in deep completions disappears.","tokens_in":2630,"feed_emoji":"🎥","tokens_out":591,"duration_ms":26845,"temperature":0.7,"pith_summary":"This paper introduces VCG to handle the extreme cold-start problem in e-commerce video feeds, where new short videos have almost no user interaction data. It adapts a CLIP model so that videos and users live in the same embedding space, letting the system recommend based on what the video looks like rather than past watches. The approach also counters biases from video position and length that skew normal signals. Online tests with real users showed a 50 percent increase in videos completed to the end. The authors also compare different embedding types and show that models trained to discriminate work better for ranking than those that generate text.","feed_headline":"Model retrieves new videos by sight alone, lifts completions 50%","feed_subtitle":"VCG adapts CLIP to create a shared space for users and videos, solving cold-start in e-commerce feeds.","key_machinery":"Domain-adapted CLIP model for mapping users and videos to a shared semantic space that supports zero-shot visual retrieval.","core_discovery":"VCG creates a shared semantic space using a domain-adapted vision-language model based on CLIP, which allows zero-shot retrieval of videos for users based solely on visual content. This setup mitigates engagement biases in immersive feeds and results in a 50% uplift in deep video completion as measured by online A/B testing.","pith_inferences":["The method could be tested in non-commerce video platforms facing similar new content issues.","Visual alignment might allow recommendations for entirely new user groups without any history.","Further gains may come from adding text or audio features to the shared space."],"forward_implications":["Generative LLM embeddings excel at attribute prediction but suffer from embedding space collapse during retrieval.","Discriminative CLIP embeddings provide better performance in retrieval tasks under cold-start conditions.","The system enables three bi-directional retrieval scenarios including product-to-video and zero-shot semantic search.","Position and duration biases in video feeds are reduced through content-based matching."],"fun_headline_variants":["VCG maps users and videos via adapted CLIP","Zero-shot visual retrieval solves e-commerce cold-start","Domain CLIP adaptation mitigates video feed biases","VCG shows 50% uplift in deep video completions"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The adapted vision-language model places users and videos in a shared space where visual similarity predicts engagement without any past behavior data.","fun_headline_variants_meta":{"raw":{"variants":["VCG maps users and videos via adapted CLIP","Zero-shot visual retrieval solves e-commerce cold-start","Domain CLIP adaptation mitigates video feed biases","VCG shows 50% uplift in deep video completions"]},"model":"grok-4.3","cost_usd":0.006701,"raw_usage":{"total_tokens":3120,"prompt_tokens":664,"num_sources_used":0,"completion_tokens":60,"cost_in_usd_ticks":67012000,"prompt_tokens_details":{"text_tokens":664,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2396,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":664,"tokens_out":60,"duration_ms":18449,"temperature":1.0,"reasoning_tokens":2396,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T18:50:16.058448+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the A/B test with the model turned off or replaced by a model that ignores visual content and measuring whether the 50% uplift in deep completions disappears.","supporting_citations":[],"review_version":1}