{"id":"cd2324d8-d4b8-4ca5-9c28-014381429a66","arxiv_id":"2607.27766","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CoRA uses ridge regression and a low-rank SVD to build a gradient-free, output-conditioned retrieval index for in-context learning, beating static retrievers on ten text and four multimodal benchmarks.","lead":"This paper presents CoRA, a training-free way to choose in-context examples on a phone or small device, by aligning text and image representations with the structure of stored candidate answers. It matters because it makes output-aware retrieval cheap enough for on-device AI without sending data to the cloud.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Output-conditioned retrieval is only tested on homogeneous single-task pools; the on-device setting it claims to serve is heterogeneous, and the paper's own NL2Bash result shows output surface variability can break the conditioning signal.","rationale":"The paper is internally coherent: Proposition 3.1 is a standard Eckart-Young statement, the ridge-regression construction in Eqs. (4)-(7) is algebraically sound, and the streaming derivation in Sec. 4 checks out provided standardization statistics are folded into the sufficient-statistic accumulation. The practical deployment result on Raspberry Pi 5 is a genuine positive. What is missing is not a mathematical error but a mismatch between the motivating scenario and the evaluation protocol. The method's distinguishing ingredient is output-conditioned alignment, and the only way that ingredient can help in a real local-memory setting is if output embeddings from a heterogeneous pool still carry a stable signal for selecting demonstrations. The paper evaluates this only in single-task pools, where the output space is already regularized by the task. The NL2Bash results show that when outputs are many-to-many and surface-variable, the conditioning signal weakens; a mixed-task pool would make this problem structural. This does not invalidate the benchmark claims, but it should keep the verdict conditional until the mixed-task regime is tested. I therefore recommend no change to the reader's CONDITIONAL verdict, with the mixed-pool experiment as the decisive check.","tokens_in":25371,"tokens_out":9437,"duration_ms":109617,"concrete_test":"Run a mixed-task retrieval experiment: pool candidate pairs from at least two classification tasks with different label spaces (e.g., MRPC + QNLI) and two generation tasks (e.g., MTOP + NL2Bash) into one candidate set; evaluate CoRA, w/o Cond, Top-k BERT, BM25, and SBERT on disjoint held-out queries from each constituent task using the same downstream LLM as in Tables 2-3. Compare against the per-dataset results already reported. Include a shuffled-output CoRA control that permutes y_i among candidates while preserving the marginal output distribution. If CoRA's margin over BM25/SBERT is not preserved in the mixed pool, or if the shuffled-output control matches CoRA, then the output-conditioning premise is not robust to heterogeneous local memories.","verdict_should_be":"UNCHANGED","load_bearing_attack":"CoRA's central claim is that it converts a frozen encoder into a task-conditioned retriever for on-device local memories. The empirical support, however, comes exclusively from homogeneous candidate pools where every candidate belongs to one task (Table 1; Sec. 5.2). In the motivating personal-memory setting, a single pool would contain examples from many tasks with different label spaces, vocabularies, output lengths, and surface formats. The conditioning matrix C in Eq. (4) then mixes heterogeneous output embeddings, and the ridge-fitted subspace in Eqs. (5)-(7) may be dominated by task-identity artifacts (e.g., lexical cues that reveal which dataset a sample came from) rather than by output relations that predict demonstration utility for an unseen query. The paper does not test this regime anywhere. Its own NL2Bash analysis (Sec. 5.2, Table 6) is direct evidence that high-variability output surface form can weaken the conditioning signal; a heterogeneous pool amplifies exactly this failure mode. If CoRA's advantage over BM25/SBERT disappears or reverses when the candidate pool mixes tasks, the method does not deliver the task-conditioned on-device retrieval that motivates it, even though it may remain effective for per-dataset benchmark retrieval.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoRA, a gradient-free, output-conditioned exemplar retriever for on-device in-context learning. CoRA selects representative encoder layers via CKA, constructs a conditioning matrix from pooled candidate outputs, fits candidate input representations to that matrix through closed-form ridge regression, and extracts a low-rank retrieval basis by SVD. Query time requires only the query input and a precomputed index. The authors prove that the retained basis optimally preserves the fitted representation (Prop. 3.1), derive an exact two-pass streaming construction (Sec. 4.1), and extend the method to multimodal retrieval (CoRA-M). Experiments cover ten textual datasets and four multimodal benchmarks with several downstream LLMs, plus a Raspberry Pi 5 deployment. The paper claims consistent gains over static retrieval baselines and the prior MLSM method.","tokens_in":25742,"tokens_out":6179,"duration_ms":66577,"significance":"If the empirical claims are supported, CoRA would be a useful contribution: it provides output-informed, task-conditioned retrieval without retriever fine-tuning, backpropagation, or target-model calls, and the streaming index construction with memory independent of n is a genuine practical advantage. Proposition 3.1 is correctly proved, and the exact two-pass derivation in Sec. 4.1 is a clear strength. The Raspberry Pi 5 end-to-end evaluation is also a positive feature. However, several load-bearing validation gaps remain: the default rank is selected on the evaluation datasets themselves, the motivating heterogeneous on-device memory regime is never tested, and the reported error bars are undefined. The significance of the headline claims is therefore not yet fully established.","major_comments":[{"comment":"The default subspace rank r=256 is selected from a grid r∈{32,64,128,256,512} by examining CoRA's performance on the same evaluation datasets (SST-5, QNLI, WebQs, MTOP) that later appear in the headline tables (Tables 2–3). No held-out validation split or nested hyperparameter procedure is described. Since r is a free parameter and is tuned on the test tasks, the comparison against baselines is optimistically biased by test-set model selection. Please report results with validation-based r, or show that the conclusions are stable across r on a separate split.","section":"§5.3, Table 5"},{"comment":"The motivating use case is a single local, heterogeneous memory, yet all retrieval experiments use per-dataset homogeneous pools (Table 1). The paper's own NL2Bash results show that output-surface variability weakens the conditioning signal: CoRA is below BM25 or SBERT on NL2Bash under every backbone (e.g., 0.3248 vs. BM25 0.3346 for Llama-3.2-1B; 0.2871 vs. SBERT 0.3498 for MobileLLM-Pro). A mixed-task pool would combine heterogeneous output spaces and plausibly amplify this failure mode. Without a mixed-pool experiment, or an explicit restriction of the claim to single-task pools, the abstract's assertion that CoRA converts a frozen encoder into a task-conditioned retriever for on-device local memories is not empirically supported.","section":"§5.2, Tables 2–3; §6"},{"comment":"The ± values attached to CoRA results (e.g., 0.6486±0.02, 0.3951±0.02, 39.68±0.01) are never defined: there is no statement of the number of runs, the set of seeds, or whether the intervals are standard deviations or standard errors. Baselines are reported without intervals, so the reader cannot judge whether the headline gains over MLSM or BERT, which are often only 0.01–0.03, are meaningful. Please define the interval and report comparable variability for all methods.","section":"§5.2, Tables 2–4"}],"minor_comments":[{"comment":"The retained-fitted-energy comparison in Table 8 is, by construction, maximized by the SVD basis: Eq. (14)–(18) define V_{1:r} as the optimizer of exactly the quantity reported in the table. This does not invalidate the downstream metric column, but the fitted-energy column is not independent evidence for the method. Please present it as a sanity check rather than as empirical validation.","section":"§3.3, Table 8"},{"comment":"The 'two-pass' streaming description does not state where the Z-score standardization statistics (mean and standard deviation) are computed. If they require a full pass over the pool, this is effectively an extra pre-pass or must be accumulated as sufficient statistics during Pass 1. Please clarify so that the pass count is exact.","section":"§4.1, Eq. (21)"},{"comment":"The table caption says 'Avg. denotes the average score across all five tasks,' but only four multimodal benchmarks are listed. Please correct the caption.","section":"Table 4"},{"comment":"Figure 2 appears corrupted in the manuscript as rendered: the text contains long '/uni000...' placeholder sequences, making the figure unreadable. Please regenerate the figure.","section":"Figure 2"},{"comment":"The downstream metric column in Table 8 does not specify the backbone, prompt configuration, or k. The MRPC value 0.7353 appears to coincide with Table 10 (Raspberry Pi deployment with Qwen3.5-0.8B), which is confusing. Please specify the evaluation setting.","section":"§5.3, Table 8"},{"comment":"No code or data availability statement is provided. To support reproducibility, please indicate whether the implementation and processed datasets will be released.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is in scope for the journal and the mathematical/algorithmic core is solid: the streaming derivation is exact and Prop. 3.1 is correctly proved. My main concern is that the empirical validation is not yet as strong as the abstract's wording. The absence of a heterogeneous-pool experiment is particularly important because the method's motivating scenario is a single on-device memory containing mixed tasks, and the paper's own NL2Bash results show the conditioning signal degrades with output variability. I would also require well-defined error bars and a validation-based choice of r before the headline comparisons can be fully trusted. These issues are addressable with additional experiments and analysis, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick take on arXiv:2607.27766. It’s a solid, honestly presented incremental paper. The genuinely new bit is CoRA: a closed-form, gradient-free retrieval basis built from paired inputs and outputs of a frozen encoder, with an exact two-pass streaming construction and a text-plus-vision variant. No target-model calls at query time, and the Raspberry Pi 5 demo shows it actually runs on edge hardware. The math in Sections 3 and 4 checks out; the streaming derivation is exact; and the ablations mostly line up with the design story. Credit where due: the paper is careful about what it compares, and it admits NL2Bash fails to beat baselines and discusses why.\n\nWhere it’s soft: First, the motivating setting—on-device personal memory—is heterogeneous, but every experiment uses a homogeneous, single-task candidate pool. The stress-test note is right: the conditioning matrix in Eq. (4) is built from outputs that all share one label space, so task-identity artifacts aren’t stressed. NL2Bash is your own evidence that output-surface variability weakens the signal; a mixed pool is exactly that, amplified. Until they test on a mixed-task pool (or retract the on-device framing), the central claim “task-conditioned retriever for local memories” is only partially supported. Second, r=256 is chosen after ablating on the same evaluation datasets, and the ± numbers in Tables 2–4 are undefined—no seeds, no CI specification. No code or data is released either, which makes the empirical claims hard to verify. Third, Proposition 3.1 is Eckart-Young restated; it’s correct but not a new theorem, and the “optimal” language in the abstract overstates the contribution. That’s minor, though—the interesting part is the empirical link between retained fitted energy and retrieval quality.\n\nI disagree with the reader on the circularity burden: the energy comparison in Table 8 is what you’d expect by construction, but it’s not used dishonestly; the paper ties it to downstream performance, which is the meaningful check.\n\nBottom line: worth serious refereeing. The method is plausible and mostly well supported, but the heterogeneous-pool gap and missing release need addressing. I’d send it out.","headline":"Solid, incremental gradient-free retrieval work whose on-device framing outruns its evaluation: homogeneous pools only, and the release is missing.","tokens_in":26208,"tokens_out":2412,"would_cite":true,"duration_ms":28962,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoRA converts a frozen encoder into a task-conditioned retriever using only closed-form linear algebra, so demonstration selection never needs training.","keywords":["in-context learning","exemplar retrieval","on-device retrieval","task-conditioned retrieval","closed-form ridge regression","low-rank projection","frozen encoder","multimodal retrieval"],"falsifier":"Swap or randomly permute the output embeddings used in the conditioning matrix while keeping inputs, index size, and query pipeline fixed; if downstream ICL accuracy does not drop below CoRA's, the output-derived conditioning is not what carries the gains. The paper runs this check only for shuffled visual features in CoRA-M, not for text outputs.","tokens_in":25301,"feed_emoji":"📱","tokens_out":4223,"duration_ms":44806,"temperature":0.7,"pith_summary":"The paper sets out to resolve the tension in on-device in-context learning between task-aware demonstration selection and lightweight, local retrieval. It claims that a frozen text encoder can be turned into a task-conditioned retriever by fitting candidate input representations to a conditioning space built from paired candidate outputs, using ridge regression and a low-rank projection. If correct, output-aware exemplar selection becomes possible without retriever fine-tuning, backpropagation, or calls to the downstream model, and the whole index can be built in two streaming passes. The authors extend the same alignment to multimodal retrieval and report end-to-end feasibility on a Raspberry Pi 5.","feed_headline":"Outputs teach frozen encoders to retrieve, no training needed","feed_subtitle":"A closed-form, two-pass index brings output-aware example selection to phones and tiny devices.","key_machinery":"The conditioning matrix C (a column of ones plus standardized mean-pooled frozen-encoder embeddings of candidate outputs, optionally with visual embeddings in the multimodal variant) and the ridge-fitted projection P_C = C(C^T C + lambda I)^{-1} C^T. CoRA applies P_C to layerwise input representations, concatenates the fitted blocks, and takes the top-r right singular vectors V_{1:r} of the fitted matrix as the retrieval basis. This object carries the argument: it converts output-side regularities into a fixed, compact projection that transfers task information to unseen queries using only their inputs, and its optimality is characterized by Proposition 3.1.","core_discovery":"The central claim is that the components of a frozen encoder's layerwise input representations that are linearly explained by candidate outputs define a useful retrieval subspace. CoRA builds a conditioning matrix from mean-pooled embeddings of candidate outputs, fits the selected-layer input features to it by closed-form ridge regression, and takes the top-r right singular vectors of the fitted matrix as a compact retrieval basis. Proposition 3.1 shows this basis maximizes retained fitted energy among all rank-r orthogonal projections, i.e., it is the optimal low-rank compression of the output-conditioned fit. At query time only the input is encoded and projected through the precomputed bas","pith_inferences":["The optimality result is about the fitted matrix, not about downstream ICL accuracy; a natural extension would test whether the same basis remains optimal when the retrieval objective is the end-to-end language-model score, and if not, what closed-form correction restores it.","The paper's own NL2Bash result suggests the method's load-bearing assumption can be stressed: tasks with many-to-many input-output mappings, where surface-form outputs are weak proxies for demonstration utility, are exactly where output conditioning may need richer or learned output representations.","Because the streaming construction maintains sufficient statistics G and T, an incremental update variant that absorbs new candidate pairs without a full rebuild seems directly within reach, though the paper leaves evolving pools as future work.","The conditioning mechanism is not tied to a particular encoder; applying the same alignment to other frozen encoders, or using task labels instead of outputs as the conditioning signal, would test how general the principle is."],"forward_implications":["Demonstration selection can become output-aware on-device without training a retriever or querying the target LLM, closing the gap between task-agnostic similarity search and learning-based selection.","Index construction is a two-pass streaming computation whose working memory depends on chunk and feature dimensions, not on the number of candidates, so pools can grow without materializing large matrices.","Since query-time retrieval is a single projection plus nearest-neighbor search in r dimensions, standard ANN indexing and CPU/edge hardware apply directly.","The same closed-form alignment extends to multimodal ICL by appending visual features to conditioning and retrieval spaces, giving a unified retrieval formulation across text and vision-language tasks.","Because CoRA is gradient-free, it runs where training-based retrievers run out of memory, including the reported Raspberry Pi end-to-end pipeline."],"fun_headline_variants":["Output-conditioned retrieval, no training, for on-device ICL","Frozen encoder gets task-aware basis via closed-form ridge","Gradient-free: outputs shape retrieval, no backprop","Optimal low-rank compression of output-conditioned fit","Two-pass streaming index for efficient on-device retrieval"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that mean-pooled frozen-encoder embeddings of candidate outputs (plus visual features in the multimodal variant) carry a stable, task-relevant signal about which demonstrations will help an unseen query; if output surface form is only weakly tied to demonstration utility, the fitted subspace encodes irrelevant structure and query projections degrade.","fun_headline_variants_meta":{"raw":{"variants":["Output-conditioned retrieval, no training, for on-device ICL","Frozen encoder gets task-aware basis via closed-form ridge","Gradient-free: outputs shape retrieval, no backprop","Optimal low-rank compression of output-conditioned fit","Two-pass streaming index for efficient on-device retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000393,"raw_usage":{"total_tokens":1919,"prompt_tokens":782,"completion_tokens":1137,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":1055}},"tokens_in":526,"tokens_out":1137,"duration_ms":12989,"temperature":1.0,"reasoning_tokens":1055,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T01:45:15.134757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Swap or randomly permute the output embeddings used in the conditioning matrix while keeping inputs, index size, and query pipeline fixed; if downstream ICL accuracy does not drop below CoRA's, the output-derived conditioning is not what carries the gains. The paper runs this check only for shuffled visual features in CoRA-M, not for text outputs.","supporting_citations":[],"review_version":1}