{"id":"92eb49b5-72a2-49f4-b658-639d852d8272","arxiv_id":"2608.12987","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"DrIG reuses one residual-quantized identifier per candidate as both an ordered decoding target and an unordered relevance set, combining constrained beam search with a prefix-independent prior for universal multimodal retrieval.","lead":"This paper introduces DrIG, a retrieval system that assigns every candidate a short discrete code and generates the code of the item that matches a text, image, or multimodal query. It reports that reusing each code as both a sequence and a set reduces decoding errors and beats earlier generative baselines on the M-BEIR benchmark and text-to-image datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (15)'s prefix max is either corpus-dependent or an undocumented approximation; the paper specifies no feasible way to compute it, undermining the core inference mechanism.","rationale":"The paper's central claim is that reusing the same residual-quantized identifier as an ordered decoding target and as an unordered set-based prior improves generative universal multimodal retrieval. The load-bearing condition is that the set-based prior in Equation (16), via phi in Equation (15), can actually be computed during constrained beam search without destroying the claimed scalability. The reader's weakest_assumption focused on the additive token-level scoring assumption and the reliability of the max-over-prefix proxy. That is a valid concern about effectiveness, but my concern is more fundamental: the paper does not specify an algorithm for Equation (15), and the two plausible readings are mutually incompatible with the reported results. If phi is exact, the computation scales with the number of candidates sharing a prefix, making the flat QPS curve in Figure 3a and the corpus-size-independent efficiency claim untenable. If phi is approximated per level, then it may not correspond to any actual candidate and the 'global relevance prior' is not the quantity defined in Equation (15); the 1-2 point gains in Table 5 would then be attributed to an undocumented surrogate. This is not an accusation of misreporting; the experiments may well have used a sensible approximation. But the manuscript as written does not let a reader verify the core mechanism, and the central claim is conditional on resolving this gap. I therefore keep the verdict CONDITIONAL rather than ACCEPT or REJECT: the concern is concrete and testable, and the paper can be repaired by specifying and validating the actual computation of Equation (15). I credit the extensive ablations and consistent comparisons against GENIUS; those empirical results are substantial, but they do not remove the need for an explicit inference algorithm.","tokens_in":34738,"tokens_out":7093,"duration_ms":82622,"concrete_test":"Obtain the released code or request the authors' inference routine, and inspect how phi in Equation (15) is computed. Concretely, on a small corpus (e.g., 5K candidates), implement the exact max over C_{t<=i} and compare it with the per-level max approximation. Report (i) wall-clock QPS as a function of corpus size for both variants, and (ii) the fraction of approximate maxima that correspond to actual candidate identifiers, together with the resulting Recall@5 difference. If exact phi is used, the flat-QPS scalability claim fails; if the per-level max is used, the paper must document it and the effectiveness claims are conditional on that approximation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 4.3.1 defines the set-based prior in Equation (15) as an exact maximum over C_{t<=i}, the set of all candidate identifiers sharing the current prefix. As written, this requires scoring every candidate under a prefix at each beam expansion; for early prefixes such as the modality token or the empty prefix, C contains the entire corpus (up to 5.6M candidates). That would make inference O(|C|*L) per query and contradict both the flat QPS curve in Figure 3a and the claimed corpus-size-independent decoding cost. The only way to keep QPS flat is to approximate the maximum, for example by taking per-level maxima over codebook tokens and summing them. But such a sum may correspond to a token combination that is not an actual candidate identifier, so it is not a bound on any real candidate's relevance score. The paper gives no data structure, pruning rule, or algorithmic detail for computing Equation (15), and this is the central inference mechanism behind the claimed improvement over GENIUS. The concern is therefore about internal consistency: either the described inference is computationally infeasible at the stated scale, or the implemented prior is an unstated approximation whose behavior and validity are not analyzed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DrIG, a generative framework for universal multimodal retrieval. Each candidate is assigned a single residual-quantized identifier whose first token encodes modality and whose remaining tokens encode progressively finer semantics. The same identifier is used in two roles: a sequential role for Trie-constrained autoregressive decoding, and a set-based role in which the tokens are treated as an unordered set to produce a prefix-independent relevance prior. During inference, the two scores are combined in Equation (16), and an optional dense reranker refines the top-k generated candidates. Experiments on M-BEIR, Flickr30K, and MSCOCO compare DrIG against dense baselines, GENIUS, and text-to-image generative baselines, and include ablations over the set-based role, Trie constraint, query augmentation, ranking loss, contrastive loss, modality codebook, codebook configuration, beam size, reranking depth, and decoder backbone.","tokens_in":34980,"tokens_out":5606,"duration_ms":67071,"significance":"If the reported results hold, DrIG is a meaningful advance for generative universal multimodal retrieval. The paper evaluates on a broad and challenging benchmark (M-BEIR), shows consistent improvements over the closest generative baseline GENIUS, and provides unusually thorough ablations and hyperparameter analyses that give practical design guidance. The hybrid generative-plus-dense reranking strategy is a useful contribution, and the paper is honest about the remaining gap to strong dense retrievers. However, the central inference mechanism in Equation (15) is not specified at an implementable level, and the empirical comparison relies on single-seed results without released code or significance tests. These issues are load-bearing for the paper's main claims and need to be addressed before the manuscript is ready for publication.","major_comments":[{"comment":"The exact definition of the global relevance prior phi(t<=i; z_q) = max_{m in C_{t<=i}} s_oi(z_q, m) is not computationally feasible as written. For early prefixes, C_{t<=i} contains a substantial fraction of the candidate corpus, which has up to 5.6M candidates on M-BEIR. Computing an exact maximum over this set for every expanded prefix, for every query, gives an inference cost that scales with |C|, contradicting the corpus-size-independent decoding claim in Section 4.4 and the flat QPS curve for DrIG in Figure 3a. If the implementation instead uses an approximation, such as per-level maxima over codebook tokens, the paper must say so explicitly, because such a sum can select a token combination that is not an actual candidate identifier and therefore is not a valid relevance score for any real candidate. Please provide the exact algorithm or data structure used to compute Equation (15), state whether it is exact or approximate, and analyze the validity and cost of that approximation.","section":"Section 4.3.1, Equation (15)"},{"comment":"The main comparative claim that DrIG 'consistently outperforms' GENIUS is supported only by single-seed runs with no significance tests and no released code. Some of the task-level differences are small; for example, in local-pool retrieval on MSCOCO qi->ct, DrIG-LT reaches R@5 of 90.0 versus GENIUS-C at 89.9, and several DrIG-C versus GENIUS-C differences are around two points. Given the stochasticity of LMM fine-tuning, identifier construction, and decoder training, the absence of multiple seeds or significance testing makes it difficult to know which of the task-level gains are reliable. Please report multiple seeds with means and variances, or provide significance tests for the main comparisons, and make the code available so the baseline numbers can be reproduced.","section":"Section 5.2, Table 3"},{"comment":"The set-based relevance prior assumes that token-level evidence is additive and order-free, and that the maximum score over all candidates sharing a prefix is a reliable global relevance signal. The paper does not test this assumption directly. The ablations show consistent but modest gains from the set-based role, on the order of 0.6 to 1.7 R@5 points across tasks in Table 5. To strengthen the central claim, please add ablations with alternative aggregation functions, such as mean pooling, max pooling, or a learned attention over identifier tokens, and report the relative contribution of the modality token versus the semantic tokens to s_oi. This would clarify whether the order-free additive model is essential or whether a simpler aggregation would perform equally well.","section":"Section 4.2.2, Equations (8) and (15); Table 5"}],"minor_comments":[{"comment":"The sentence 'Inspired these two studies, DrIG designs identifiers with two complementary roles' is missing the word 'by' and should read 'Inspired by these two studies'.","section":"Section 2.2.1"},{"comment":"Reference [50] contains the typo 'PmLR'; the correct publisher abbreviation is 'PMLR'.","section":"References"},{"comment":"The table would be clearer if the zero-shot rows were explicitly marked with the dagger symbol in the method name itself, since the current formatting places dagger symbols inconsistently and the caption requires careful reading to distinguish M-BEIR-trained from in-domain-trained models.","section":"Table 4"},{"comment":"The t-SNE visualization in Figure 2 is qualitative; please state the number of points visualized and whether the same query or candidate embeddings were used for both panels, so that the visual comparison is meaningful.","section":"Section 5.4.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is closely related to the authors' own ComGTIR paper (reference [30]), which also uses dual identifiers and hybrid retrieval. The novelty here is presented as reusing a single identifier in two roles rather than maintaining two identifier spaces; that difference is real but should be verified by the editor as sufficiently incremental. The absence of released code and single-seed results is a reproducibility concern. If the authors can specify the inference algorithm for Equation (15) and provide multi-seed results, the paper may become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The actual novelty here is cleaner than the packaging: one residual-quantized identifier reused as both an ordered decoding target and an unordered set for prefix-independent scoring, in the universal multimodal setting. Earlier dual-identifier designs (ComGTIR, PAG, TSGen) either use separate identifier spaces or stay text-only. Reusing the same tokens is a genuine variation, and the paper says this clearly. The empirical win over GENIUS is large and consistent across M-BEIR tasks, and the ablation story is honest: the set-based role gives modest but systematic gains, while contrastive pre-quantization and the Trie constraint are the load-bearing components. The hybrid reranking result is also useful, showing how generative candidate generation plus dense reranking can narrow the gap to strong dense retrievers. I give credit for that.\n\nThe soft spot is real and central. Equation 15 defines the global prior as an exact max over all candidate identifiers sharing a prefix. At early prefixes, that set is essentially the whole corpus, millions of candidates. No data structure, pruning rule, or approximation is described, and the flat QPS curve in Figure 3a cannot be reconciled with exact computation at that scale. Either the implementation approximates the max, in which case the approximation is unstated and its behavior unanalyzed, or the inference algorithm as written is infeasible. Since this set-based prior is the main claimed improvement over GENIUS, the authors need to specify what is actually computed. This does not automatically kill the empirical result, but it makes the core inference mechanism underspecified.\n\nAlso noteworthy: results are single-seed with no significance tests, code and models are promised but not released, and the efficiency comparison in Figure 3a uses brute-force dense search rather than ANN, so the \"scalability advantage\" over dense retrieval is overstated. The adaptive margin scale gamma in Equation 20 is not given a value in the hyperparameter table. The set-based additive score assumption is not tested against alternative aggregations, though the ablations suggest the gains are real but modest. None of these are fatal on their own; together they mean the paper needs a revision, not a rejection.\n\nCitation pattern is fine. The overlap with the authors' own ComGTIR is disclosed, and the positioning against GENIUS, PAG, and TSGen is accurate. I see no circularity.\n\nI would send this to a serious referee. The central idea is worth engaging, and the empirical comparison is strong enough to merit referee time. The revision should clarify Equation 15, release code or at least detailed pseudocode for the prior, report variance across seeds, and rerun the efficiency comparison against ANN-based dense retrieval. If those are addressed, this becomes a solid IR contribution.","headline":"DrIG's dual-role identifier idea is new and the empirical gains over GENIUS are consistent and substantial, but the set-based prior in Equation 15 is computationally underspecified and the efficiency claim needs an approximation or an algorithm before it stands.","tokens_in":35515,"tokens_out":1886,"would_cite":true,"duration_ms":23507,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reusing one identifier as both a sequence and a set lifts generative retrieval accuracy.","keywords":["generative information retrieval","universal multimodal retrieval","residual quantization","dual-role identifiers","constrained beam search","set-based relevance prior","hybrid retrieval","M-BEIR"],"falsifier":"Train DrIG with the set-based prior computed from the additive token score replaced by a mean or a learned attention pooling, and measure Recall@K on M-BEIR; if performance does not drop, then the additive token-level prior is not the active ingredient behind the reported gains.","tokens_in":34515,"feed_emoji":"🔎","tokens_out":5737,"duration_ms":57472,"temperature":0.7,"pith_summary":"The paper argues that the main weakness of generative information retrieval—that left-to-right decoding can prune a relevant item as soon as an early prefix scores poorly—can be addressed without creating a second identifier space. DrIG gives every candidate a single residual-quantized identifier and reuses its tokens in two roles: an ordered target for autoregressive generation, and an unordered set whose token scores provide a prefix-independent relevance prior during beam search. On the M-BEIR universal multimodal benchmark, DrIG raises average Recall from 29.5 to 38.0 in local-pool retrieval and from 28.6 to 36.4 in global-pool retrieval over the previous generative baseline, and with dense reranking reaches 50.4 and 48.9 averages, competitive with mid-tier dense retrievers while keeping throughput insensitive to corpus size. The sympathetic reading is that the dual-role reuse of one identifier is what lets a generative retriever keep global relevance awareness without losing the efficiency of constrained decoding.","feed_headline":"One identifier, two roles: DrIG lifts generative retrieval accuracy","feed_subtitle":"Re-reading the same quantized code as an unordered set guides beam search and narrows the gap to dense retrievers.","key_machinery":"The central object is the dual-role identifier: a length-$L$ residual-quantized code $m = [m_1,\\dots,m_L]$ whose first codebook has size 3 for modality and whose later codebooks encode increasingly fine semantics. In its sequential role the identifier is decoded under a Trie constraint with a score that sums token-embedding dot products; in its set-based role the same tokens are mapped through a fixed global codebook table to token-level query scores, and a candidate-level order-invariant score is computed as a sum over the identifier's tokens. The mechanism that carries the argument is the combined expansion score $f(t_{\\le i}; z_q) = \\delta(t_{\\le i}) + \\eta(t_{<i}) + E^{dec}_i[t_i]\\cdot h_i + \\lambda \\phi(t_{\\le i})$, where $\\phi(t_{\\le i})$ is the maximum order-invariant score over all candidates sharing the current prefix. This term injects global, prefix-independent relevance into every beam expansion, so a branch that the local decoder underrates can survive if its reachable candidates match the query as a set.","core_discovery":"The central claim is that a single residual-quantized identifier can simultaneously serve as an ordered generative target and as an unordered set-based relevance representation, and that this dual role is what lets constrained beam search avoid prefix-pruning errors. Concretely, DrIG encodes each candidate with a first token that names the modality (image, text, or image-text) followed by residual-quantized semantic tokens; during decoding the same codebook tokens are scored independently against the query via an order-invariant score, and the best score among all candidates sharing a prefix is added, weighted by a hyperparameter, to the autoregressive score. The authors report that this consistently improves over the GENIUS baseline across all M-BEIR task types, with particularly large gains on knowledge-intensive InfoSeek and compositional CIRR, and that the improvement is complementary to dense reranking. When the top-k generated candidates are reranked by continuous cosine similarity, the method moves from generative-only averages of 38.0 and 36.4 to 50.4 and 48.9, surpassing CLIP-SF and BLIP-FF in the global-pool setting while remaining below stronger LMM-based rerankers.","pith_inferences":["If the dual-role reuse is the active ingredient, the same one-identifier-two-views design should transfer to other structured decoding tasks where the output vocabulary is built from quantized embeddings, such as hierarchical product or entity search; the paper leaves this implicit.","The additive token score is the paper's simplest aggregation choice; replacing it with a learned or attention-weighted pooling over the set tokens would be a direct test of whether token independence is a limitation or a feature, and could improve hard cases like NIGHTS.","The flat throughput curve suggests that generative retrieval with dual-role identifiers could support dynamic corpora more gracefully than dense indexes, since inserting a candidate only requires adding its code tokens to the Trie and the global codebook table; the paper lists dynamic corpora as future work, not as a demonstrated capability.","Reading the ablations as a whole, the remaining gap on text-to-text and knowledge tasks points to quantization depth as the likely bottleneck rather than reranker strength; this is an interpretation, not a claim the paper makes."],"forward_implications":["On M-BEIR, DrIG outperforms the previous generative universal multimodal retriever on every task type, raising average Recall from 29.5 to 38.0 in local-pool retrieval and from 28.6 to 36.4 in global-pool retrieval.","Reranking the top-k generated candidates with dense cosine similarity brings the global-pool average to 47.1 and 48.9 for the CLIP-based and LMM-based reranking variants, surpassing the CLIP-SF and BLIP-FF dense baselines while remaining below stronger LMM-based rerankers.","The set-based prior is what carries the prefix-independence claim: removing it costs one to two Recall points across tasks, while removing the Trie constraint costs far more, showing that valid-prefix control and global relevance guidance are complementary.","Throughput stays nearly flat as the candidate pool grows from 5K to 300K, because the online cost is dominated by fixed-length identifier decoding and a small top-k reranking step rather than by scoring the full corpus.","Identifier expressiveness, not decoder size, is the main lever on knowledge-intensive tasks: deeper quantization and larger codebooks improve text-centric Recall@1 substantially, while scaling the decoder from T5-small to T5-large gives only mixed gains and hurts visually fine-grained tasks."],"supporting_citations":[{"why":"Defines GENIUS, the generative universal multimodal retrieval baseline that DrIG compares against and improves on by roughly 27-29% average Recall.","marker":"[23]"},{"why":"Provides the M-BEIR benchmark, task taxonomy, local and global pool settings, and the UniIR dense baselines CLIP-SF and BLIP-FF used for comparison.","marker":"[66]"},{"why":"Supplies the LMM embedding and two-stage fine-tuning recipe that DrIG uses for instruction-aware representations and as the strong LamRA reranker baseline.","marker":"[40]"},{"why":"ComGTIR is the closest prior generative text-image retrieval framework with dual identifiers and hybrid reranking, which DrIG generalizes to universal multimodal retrieval.","marker":"[30]"},{"why":"Residual quantization is the technique that converts continuous embeddings into the level-wise codebooks from which dual-role identifiers are built.","marker":"[25]"},{"why":"PAG motivates combining sequential and set-based identifier views; DrIG adopts the idea but reuses one identifier for both roles.","marker":"[72]"}],"fun_headline_variants":["Dual-role IDs: one code for ordered and set-based retrieval","Rethinking identifiers: DrIG's ordered and unordered roles","DrIG: one ID, two roles, better generative retrieval","Same tokens, two views: DrIG boosts beam search","Prefix-free relevance: DrIG reuses quantized codes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a candidate's relevance can be estimated by adding up independent per-token scores, and that the best-scoring candidate under a prefix reveals the value of keeping that prefix.","fun_headline_variants_meta":{"raw":{"variants":["Dual-role IDs: one code for ordered and set-based retrieval","Rethinking identifiers: DrIG's ordered and unordered roles","DrIG: one ID, two roles, better generative retrieval","Same tokens, two views: DrIG boosts beam search","Prefix-free relevance: DrIG reuses quantized codes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000226,"raw_usage":{"total_tokens":1545,"prompt_tokens":1101,"completion_tokens":444,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":717,"completion_tokens_details":{"reasoning_tokens":359}},"tokens_in":717,"tokens_out":444,"duration_ms":4599,"temperature":1.0,"reasoning_tokens":359,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:09:02.161702+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train DrIG with the set-based prior computed from the additive token score replaced by a mean or a learned attention pooling, and measure Recall@K on M-BEIR; if performance does not drop, then the additive token-level prior is not the active ingredient behind the reported gains.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines GENIUS, the generative universal multimodal retrieval baseline that DrIG compares against and improves on by roughly 27-29% average Recall."}],"review_version":1}