{"id":"5f64fcbf-aeb1-4068-8b4a-a7cca3f21d04","arxiv_id":"2607.22922","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"iFVS adapts Product Quantization codebooks to individual filter predicates and query vectors, improving QPS-recall tradeoff over fixed-codebook PQ on SIFT1M/SIFT10M.","lead":"This paper proposes iFVS, a system that trains a separate quantized codebook per filtered query so that compressed vectors rank more accurately under specific filter predicates. It matters because filtered vector search is common in AI+DB systems, and this is a novel way to improve the recall-throughput tradeoff without storing full-precision vectors.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Held-out test recall in Table 2 is below the fixed-codebook baseline, so the reported recall gain is driven by training queries.","rationale":"The reader identified the synthetic filter workload as the weakest assumption, which is about external generality. The more immediate and damaging concern is internal: the headline recall numbers are averages over training and test queries, and the paper's own test-only numbers in Table 2 fall below the baseline it claims to beat. This directly undercuts the strongest claim. The reader's concern about workload representativeness remains relevant, but it is secondary to the train/test aggregation issue. Because the reported evidence contains a concrete contradiction with the central claim, the verdict should move from CONDITIONAL to REJECT unless the authors can show test-only curves with a genuine tradeoff improvement.","tokens_in":8541,"tokens_out":9436,"duration_ms":95166,"concrete_test":"Recompute the iFVS operating point in Figure 2 using only the held-out test query split (Table 2's Test column, M=4096) and recompute Pre_PQ_ADC's Recall@100 and QPS on the same test split. If iFVS test recall is 0.6706 or lower while Pre_PQ_ADC is 0.682 or higher at the reported QPS, the claimed recall improvement over fixed-codebook PQ disappears on held-out queries and the central claim is refuted.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing issue is internal to the reported evaluation, not the synthetic workload. Section 3.3.1 states that the figures report the average recall across training and test queries, while Table 2 reports them separately. At 1% selectivity on SIFT10M, Table 2 (M=4096) gives a test recall of 0.6706, and the largest test recall across all memory-bank sizes is 0.6706. Section 3.3.2 compares iFVS's 0.774 Recall@100 against Pre_PQ_ADC's 0.682, but 0.774 is the train/test average; the held-out test recall is below the fixed-codebook PQ baseline. Thus the central claim that query-specific codebook adaptation improves ranking over fixed-codebook PQ for unseen queries is not supported by the paper's own numbers; the improvement appears only on the training split. The conclusion's future-work item (b), improving generalization over unseen queries, further acknowledges this gap. This is load-bearing because the instance-optimization premise requires the learned adjustments to transfer to a representative workload, not just to the queries used for training.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents iFVS, a learned method for filtered vector search over product-quantized databases. For each query-predicate pair, iFVS fingerprints the pair into a shared memory bank, uses the resulting coefficients to perturb the original PQ codebook, and reweights query and database dimensions with a filter-aware weight vector; the stored PQ codes remain fixed. The method is evaluated on synthetic filtered workloads over SIFT1M and SIFT10M with selectivity 0.01, 0.05, and 0.10, against 13 pre-, post-, and in-filtering baselines. The paper reports QPS-recall tradeoffs and claims that iFVS improves ranking quality over fixed-codebook PQ baselines, e.g., Recall@100 0.774 at 334 QPS vs. Pre_PQ_ADC's 0.682 at 114 QPS on SIFT10M at 1% selectivity.","tokens_in":8792,"tokens_out":10367,"duration_ms":105796,"significance":"If validated, the central idea is relevant and timely: instance-optimizing the quantizer to a filtered workload is a natural extension of the instance-optimized database agenda, and the paper correctly notes that prior query-aware quantization has not been studied for FVS. The evaluation design has strengths: many baselines, separate training/test recall reporting in Table 2, and an anchor regularizer that limits drift from the original PQ codebook. However, the current evidence does not establish the main claim for unseen queries, because the headline comparisons use the average of training and test recall; the held-out test recall is below the corresponding PQ baseline. The absence of training details and the reliance on one synthetic workload further limit the contribution. The direction is plausible, but the paper needs a substantial revision of the evaluation and claims.","major_comments":[{"comment":"The headline comparison uses the train/test average, not held-out recall. The reported SIFT10M number 0.774 Recall@100 is the average of train recall 0.8906 and test recall 0.6572 at M=45,000 (Table 2); the SIFT1M number 0.836 is similarly the average at M=45,000. The best held-out test recall in Table 2 is 0.6706 (SIFT10M, M=4,096), which is below Pre_PQ_ADC's 0.682 reported in §3.3.2. Thus, on unseen queries, iFVS's ranking is worse than this fixed-codebook PQ baseline, contradicting the central claim that query-specific codebook adaptation improves ranking for a representative workload. Section 4(b), which lists improving generalization over unseen queries as future work, confirms that the current method does not deliver this.","section":"§3.3.1–3.3.2, Table 2"},{"comment":"The training objective is not specified. No loss function, hard-negative mining details, number of hard negatives, optimizer, learning rate, batch size, number of epochs, or values for hyperparameters r and h are given. The anchor regularizer is mentioned but its weight is not specified. Since iFVS is a learned component and the paper's evidence is purely empirical, this omission is load-bearing for reproducibility and for understanding whether the reported gains come from the architecture or from a particular training schedule.","section":"§2, Learning C_iFVS and w_f"},{"comment":"The 'representative workload' is entirely synthetic: predicates are imposed on one to three fixed SIFT dimensions, with selectivity bins [0.01, 0.05, 0.10]. The paper does not report which dimensions, how many predicates, or how predicate difficulty varies. Because iFVS is explicitly instance-optimized, every conclusion is tied to this single synthetic workload. The conclusion correctly identifies recent filtered-query benchmarks as future work, but the abstract and introduction state the improvement without this caveat. The evaluation should either include a second workload or the claims should be re-scoped to the tested setting.","section":"§3.1, §3.3.1"},{"comment":"The iFVS entries are single operating points, whereas all baselines are plotted as tradeoff curves by sweeping ef/nprobe. A single point that lies above a baseline curve does not by itself demonstrate a superior QPS-recall tradeoff. To support the 'tradeoff' language, iFVS should be evaluated at multiple operating points (e.g., by varying the number of candidates scored or the memory-bank size) and plotted as a curve. If only one point is available, the paper should make dominance explicit rather than claiming an improved tradeoff.","section":"§3.3.1, Figures 1–2"}],"minor_comments":[{"comment":"The sentence 'The figures report the average recall across both sets' should be prominently repeated before every headline recall number; otherwise readers naturally interpret 0.774 and 0.836 as test-set results.","section":"§3.3.1–3.3.2"},{"comment":"Notation is inconsistent: C_eff appears once, while C_iFVS is used elsewhere; w_z and w_f are used interchangeably. Define all symbols in one place and use them consistently.","section":"§2"},{"comment":"The symbol M denotes HNSW graph connections in Table 1 and memory-bank size in Table 2. This is confusing; rename one of them.","section":"Tables 1 and 2"},{"comment":"The captions do not state which memory-bank size M is used for iFVS. The reader must infer it from the averages in Table 2. Add this information to the caption.","section":"Figures 1–2"},{"comment":"There are a few typos/inconsistencies: 'calucation' in §3.2, and 'Pre_FlatL2' in §3.3.7 should be 'Pre_IndexFlatL2'.","section":"§3.2, §3.3.7"}],"recommendation":"major_revision","confidential_remarks":"I see a real idea here, but the submitted evaluation overstates it. The train/test discrepancy in Table 2 is the crux: once the headline numbers are re-reported on the test split, the main comparison may no longer favor iFVS. I recommend asking the authors to re-do all headline comparisons on held-out queries, provide the training objective and hyperparameters, and ideally add a second workload. If, after this, test recall remains below the fixed-codebook PQ baseline, the paper should be reframed honestly as a study of workload specialization rather than a claim of improved filtered vector search."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know one thing going in: the paper's big reported numbers are averages over training and test queries, and when you separate them the test recall is below the fixed-codebook baseline. Section 3.3.1 says the figures report the average; Table 2 gives the split. At 1% selectivity on SIFT10M, the 0.774 Recall@100 matches memory bank M=45,000, average of train 0.8906 and test 0.6572. Pre_PQ_ADC gets 0.682. So on held-out queries, iFVS underperforms the fixed-codebook PQ it claims to beat. The authors' own future-work item (b) \"improve generalization over unseen queries\" confirms they know this. That's not a small caveat; it undercuts the central claim of the paper.\n\nWhat's actually new and worth credit: the mechanism — perturbing a fixed PQ codebook with a query-filter-specific adjustment learned via a memory bank, plus dimension reweighting based on the predicate — is a genuinely new combination for FVS. The training-side gains are real, and the QPS-recall tradeoff on training queries is strong. The paper also honestly surveys a wide set of baselines, even if the comparison is apples-to-oranges: iFVS is a learned, workload-tuned method, and the baselines are not.\n\nThe evaluation has other soft spots. No code, no hyperparameters, no precise training objective. The workload is synthetic predicates on a few SIFT dimensions, so we don't know if the learned adjustments transfer to realistic filter predicates. And the method is never compared against the query-aware quantization baselines it cites (OOD-DiskANN, query-aware quantization for MIPS), so the positioning is incomplete. These are fixable in principle; the generalization gap is not — it's a property of the method as presented, not just the write-up.\n\nFor a workshop paper, the idea is worth one serious look. If I were the editor, I'd send it to a referee who knows FVS, with explicit instructions to check the train/test reporting. But in its current form, the central claim is unsupported, and I would not cite it as evidence that query-specific codebook adaptation improves ranking for unseen queries.","headline":"Novel idea for query-conditioned PQ codebooks in filtered vector search, but the headline recall gain is on training queries; held-out test recall is below the fixed-codebook baseline, so the core claim doesn't hold.","tokens_in":9304,"tokens_out":3269,"would_cite":false,"duration_ms":33893,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"iFVS claims that adapting the PQ codebook to each query-filter pair improves the QPS-recall tradeoff of compressed filtered vector search without extra per-vector storage.","keywords":["filtered vector search","product quantization","instance-optimized","codebook adaptation","query-aware quantization","PQ codebook","approximate nearest neighbor","QPS-recall tradeoff"],"falsifier":"Run the same iFVS training pipeline on a published filtered-search benchmark with real metadata predicates and compare Recall@100 vs QPS at 1% selectivity: if iFVS no longer exceeds Pre_PQ_ADC on both recall and QPS, or if its test recall with a large memory bank falls below the fixed-codebook baseline, the instance-optimization claim would fail on that ground.","tokens_in":8417,"feed_emoji":"⚡","tokens_out":5565,"duration_ms":50673,"temperature":0.7,"pith_summary":"Filtered vector search combines similarity ranking with relational predicates, and quantized indexes like PQ lose recall because a single fixed codebook is blind to how the filter reshapes the relevant vector space. iFVS proposes to generate a query- and filter-specific perturbation of the base PQ codebook, plus a predicate-derived dimension weighting, so distance estimates are conditioned on the actual query-predicate pair. Starting from the standard PQ codebook with zero adjustment, it trains the perturbation through hard-negative mining and an anchor regularizer, keeping the stored PQ codes and base codebook fixed. On SIFT1M and SIFT10M workloads, iFVS reports consistently better Recall@100 at higher or comparable QPS than fixed-codebook PQ baselines across 1%, 5%, and 10% selectivity bins, with compact index sizes as a byproduct. If correct, this shows that quantization for filtered search can be instance-optimized rather than workload-agnostic.","feed_headline":"Per-query codebooks lift filtered vector search recall","feed_subtitle":"On SIFT10M at 1% selectivity, Recall@100 rises from 0.682 to 0.774 at triple the QPS.","key_machinery":"The central mechanism is the query-specific codebook perturbation Delta = alpha * W, where alpha is produced by hashing the query-predicate pair into a shared memory bank M (h independent hash functions summed over B rows) and W is a learned tensor of shape M x r x K x d; the effective codebook is C_iFVS = C_PQ + Delta, with the predicate also encoded into a dimension-weight vector w_f = softplus(w_z) via a lightweight Transformer. This construction turns a single static PQ codebook into a family of filter-aware codebooks, and because alpha is initialized to zero and w_f to one, training starts from the exact PQ baseline. The mechanism's job is to condition distance estimation on the specifi","core_discovery":"The paper's central claim is that the precision penalty of product quantization in filtered vector search can be partially recovered by making the codebook itself a function of the query and the filter predicate. Concretely, iFVS fingerprints each query-predicate pair into a shared memory bank, sums the hash lookups to form an adjustment vector alpha, and combines alpha with a learned tensor W to perturb the base PQ codebook; a lightweight Transformer encodes the predicate into a weight vector that reweights dimensions during scoring. The perturbation is initialized to zero and the weights to one, so the model degenerates to plain PQ before training; training then adapts alpha and w_f using","pith_inferences":["If real FVS workloads distribute filters over many attribute types rather than a few SIFT dimensions, iFVS's learned adjustments may not transfer; the paper's own future-work list of recently proposed filtered-query benchmarks makes this the open question.","The perturbation mechanism is codebook-level, so the same instance-optimization idea could be applied to other quantization schemes (e.g., OPQ, additive quantization) or even to graph traversal scores, not just PQ.","The observed test-recall decline with larger memory banks suggests overfitting to training query-predicate pairs; adding regularization or a small meta-learned prior on alpha could improve generalization to filters outside the training distribution.","The predicate-derived dimension weighting w_f is decoupled from the codebook perturbation and could be tested in full-precision filtered search, where it might provide a cheap dynamic feature-weighting baseline independent of quantization."],"forward_implications":["On SIFT10M at 1% selectivity, iFVS reports Recall@100 of 0.774 at 334 QPS versus 0.682 at 114 QPS for Pre_PQ_ADC, a direct demonstration that codebook adaptation improves the QPS-recall frontier.","iFVS dominates post-filtering baselines on both datasets and all selectivity bins, and beats all PQ-compressed in-filtering baselines; only raw-vector in-filtering (IVFFlat, HNSWFlat) can exceed its recall at larger selectivities, at lower QPS.","Scaling from 1M to 10M vectors, iFVS's QPS drops only 1.33x at 1% selectivity, while exact pre-filtering slows by 20.2x and Pre_PQ_ADC by 13.6x, suggesting favorable scaling for large quantized FVS.","The compact index size (89.9 MB on SIFT1M, 220.5 MB on SIFT10M) comes at an offline construction cost of 57 and 89 minutes, respectively; this is the primary practical tradeoff.","Memory-bank size M controls a specialization-generalization tradeoff: larger M raises training recall (0.909 to 0.971 on SIFT1M) but lowers test recall (0.749 to 0.702), so generalization to unseen queries is sensitive to this hyperparameter."],"fun_headline_variants":["Query-specific codebooks improve FVS recall","iFVS conditions PQ codebook on query and filter","Per-predicate codebooks narrow PQ recall gap","Instance-optimized codebooks for filtered vector search"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the synthetic filtered workload—predicates imposed on one to three fixed SIFT dimensions with selectivities 1%, 5%, 10% and exact ground truth over predicate-passing vectors—faithfully represents real filtered vector search workloads; if real predicates interact with the vector space differently, the learned codebook adjustments and weight reweighting may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Query-specific codebooks improve FVS recall","iFVS conditions PQ codebook on query and filter","Per-predicate codebooks narrow PQ recall gap","Instance-optimized codebooks for filtered vector search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00047,"raw_usage":{"total_tokens":2182,"prompt_tokens":757,"completion_tokens":1425,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":1363}},"tokens_in":501,"tokens_out":1425,"duration_ms":10839,"temperature":1.0,"reasoning_tokens":1363,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:06:45.859696+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same iFVS training pipeline on a published filtered-search benchmark with real metadata predicates and compare Recall@100 vs QPS at 1% selectivity: if iFVS no longer exceeds Pre_PQ_ADC on both recall and QPS, or if its test recall with a large memory bank falls below the fixed-codebook baseline, the instance-optimization claim would fail on that ground.","supporting_citations":[],"review_version":1}