{"id":"e7b7fe22-0cf2-4f97-bbb6-4824385a6ee1","arxiv_id":"2501.14459","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Integrated Gradients with a separate query and document padding baseline produces token-level explanations for dense retrievers; applying it to GPL-adapted models suggests adaptation shifts attention toward domain-specific terms.","lead":"This paper uses Integrated Gradients to show which words matter most when a dense retrieval model ranks documents, and compares a base model with one adapted to finance and biomedical domains. The adapted model appears to rely more on domain terms like 'gold' and 'corona' and on document titles.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"IG attribution comparison conflates attribution magnitude with term frequency; the 'domain-adapted models focus more on in-domain terminology' claim needs a frequency-controlled test.","rationale":"The reader's weakest-assumption diagnosis focuses on sample size and selection, which is real and well-supported by the appendix's limitation statement. However, the deeper risk is that the ranking-based attribution word clouds aggregate summed attributions over a retrieved set, so a token that occurs frequently in those documents will tend to have a larger total attribution whether or not the model's per-token attribution actually changed. This makes the central qualitative claim vulnerable to a frequency confound, which would not be fixed merely by sampling more queries. The proposed method is internally coherent, the IG baseline choice is defensible, and the paper explicitly acknowledges its qualitative nature, so there is no fatal flaw; the appropriate bar is a CONDITIONAL accept with a concrete frequency-controlled evaluation. This partially overlaps with the reader's concern about representativeness, but the specific technical check I propose targets the ranking aggregation rather than the instance-level examples.","tokens_in":8036,"tokens_out":2731,"duration_ms":19365,"concrete_test":"Run the ranking-attribution analysis on a fixed set of queries (e.g., the FIQA and TREC-COVID test queries) for both the base and the GPL-adapted model, and report per-token summed attribution normalized per occurrence: sum of attribution for token t divided by the number of times t appears in the top-25 documents. If the frequency-normalized attribution for in-domain terms such as 'hedge' and 'gold' is not significantly higher for the adapted model (e.g., via a paired bootstrap over queries), the claim that adaptation changes per-token focus is unsupported. As a second check, repeat the title-attribution analysis (Figure 5) with the title length controlled, comparing mean attribution per title token rather than the raw title sum.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that domain-adapted models 'focus more on in-domain terminology' (Abstract, Figures 1-4, Section 4.2) rests on word clouds whose word sizes are summed attributions over the top 25 retrieved documents. This aggregation conflates token importance with token frequency: a term like 'gold' can appear larger simply because it appears in many of the top 25 documents, not because the model assigns it higher per-occurrence attribution. The paper's only quantitative check (Figure 5, title attribution sums) is not frequency-controlled either. The Appendix explicitly concedes 'a deep qualitative analysis of a small sample of queries' and notes there is 'no standard quantitative way of evaluating attributions.' That concession is honest but it means the paper's main positive finding is currently an anecdotal observation, not an established effect. The IG method itself is sound and the baseline choice (PAD replacement, query/document separately) is reasonable; the issue is with the comparison protocol, not with the attribution framework. Also, Figures 1 and 3 use 'random' queries and documents with no seed or selection description, so the two models are not shown to be compared on identical inputs in the ranking word clouds. If the claims are meant to generalize, the comparison must be controlled for the same query set and matched document distributions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an Integrated Gradients (IG) based attribution method for dense retrievers, introducing a PAD-token baseline that yields query-side and document-side token attributions. It applies the method to compare a MS MARCO fine-tuned DistilBERT dense retriever with its GPL domain-adapted variants on TREC-COVID and FIQA, reporting both instance-level and ranking-level (top-25 aggregated) attribution visualizations. The authors reproduce the known GPL effectiveness gains and qualitatively observe that domain-adapted models assign more positive attribution to in-domain terminology such as \"corona,\" \"hedge,\" and \"gold,\" and (for TREC-COVID) to document titles. They conclude that IG is a viable interpretability tool for dense retrievers and that domain adaptation shifts token-level attribution toward domain-specific vocabulary.","tokens_in":8232,"tokens_out":3583,"duration_ms":35125,"significance":"If the central observations were quantitatively established, the paper would provide a useful starting point for understanding how unsupervised domain adaptation changes dense retriever behavior at the token level, and the proposed PAD baseline is a reasonable, axiomatically grounded way to obtain query and document attributions. The paper uses public models and datasets, reproduces known retrieval-effectiveness gains in Table 1, and is transparent about its limitations. The strengths are the simplicity of the attribution setup and the explicit admission of the exploratory nature of the analysis. However, the significance is currently limited because the main empirical claim rests on a small number of qualitative examples and an aggregation procedure that has not been shown to isolate token importance from token frequency.","major_comments":[{"comment":"The ranking-based attribution word clouds are produced by summing token attributions over the top 25 retrieved documents. This conflates per-occurrence attribution with token frequency: a token such as \"gold\" can appear larger because it occurs in many of the top documents rather than because the model assigns it higher importance per occurrence. The paper's central claim that domain-adapted models \"focus more on in-domain terminology\" is therefore not established by the visualized word clouds. A frequency-controlled test is needed, for example average attribution per token occurrence, attribution conditioned on document frequency, or a comparison between the base and adapted models on matched document sets.","section":"Section 3, Ranking Analysis; Figures 1-4"},{"comment":"The qualitative comparison between base and domain-adapted models is based on one \"random\" query-document pair per dataset per model, with no seed, sampling protocol, or query/document identifiers reported. The captions do not state whether Figures 1 and 2 (and Figures 3 and 4) use the same query and document for the two models, although the text implies such a comparison. Without this information, the observed attribution differences cannot be verified or attributed to domain adaptation rather than to input variation and selection bias. A larger, documented sample of queries with quantitative aggregation (e.g., mean attribution for domain terms over many queries, with confidence intervals) is required to support the general claim.","section":"Section 4.1 and 4.2; Figures 1-4"},{"comment":"The paper concedes that there is \"no standard quantitative way of evaluating attributions\" and that the analysis is \"a deep qualitative analysis of a small sample of queries.\" This is honest, but the central claim about attribution shifts after domain adaptation can still be quantified without solving the general faithfulness problem: for instance, by computing aggregate attribution statistics over a random sample of test queries, comparing per-token or per-term attribution between base and adapted models, and reporting significance. Figure 5 is a step in this direction but reports no sample size, variance, or statistical test. Adding such a quantitative layer is essential for the paper's main conclusion.","section":"A Limitations; Section 5"}],"minor_comments":[{"comment":"The column header \"GLP\" should be \"GPL\"; the same typo should be corrected throughout if it appears elsewhere.","section":"Table 1"},{"comment":"The captions say the query and document are \"random\" but do not state the random seed, the sampling procedure, or whether the same query-document pair is used for the base and domain-adapted models; please add this information and ideally make the exact inputs available.","section":"Figure captions 1-4"},{"comment":"The title-attribution plot would benefit from error bars, the number of query-document pairs used, and a statistical comparison (e.g., a paired test) between the base and adapted models.","section":"Figure 5"},{"comment":"The sentence \"we evaluate provided the models\" is ungrammatical and should be revised, and the pooling/sequence-length settings should be stated more precisely with respect to the original GPL configuration.","section":"Section 3, Baseline"},{"comment":"The paper uses placeholder ACM template text (e.g., the header \"Trovato et al.\" and conference name placeholders) that should be removed or replaced before submission.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short exploratory study whose methodological proposal is reasonable but whose main empirical claim is currently supported only by qualitative, non-reproducible examples and by a ranking aggregation that conflates frequency with importance. The issues are fixable within the scope of the paper by adding a quantitative, frequency-controlled analysis over a larger query sample, but they are load-bearing for the central conclusion. I recommend major revision rather than rejection because the underlying IG formulation and the comparison setup are sound in principle."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper applies Integrated Gradients to dense bi-encoder retrieval using a separate [PAD] baseline for query and document tokens. That is a genuine extension of Zhan et al.'s cross-encoder work, and it is the first application to dense retrievers as far as I can tell. The method section is clear, and the retrieval numbers in Table 1 reproduce the known GPL gains, which is a good sanity check. The title attribution analysis in Figure 5 is a nice quantitative addition, even if small in scope.\n\nThe paper is also honest about its limitations. The appendix concedes a small sample and the absence of a standard quantitative evaluation of attributions. That is real intellectual honesty, and it makes the work easier to trust as an exploratory proof of concept.\n\nThe soft spots are real, though. The ranking word clouds sum attributions over the top 25 retrieved documents. That conflates per-occurrence attribution with term frequency: a word like \"gold\" can appear large simply because it shows up in many of the top 25 documents, not because the model gives it high attribution per occurrence. The stress-test note is right that the central claim about domain-adapted models focusing more on in-domain terminology needs a frequency-controlled test. The paper does not provide one. Also, the \"random\" queries and documents are not described with seeds or selection criteria, so I cannot verify that the base and adapted models are being compared on identical inputs. The instance-based figures are anecdotal, as the authors admit.\n\nNone of this sinks the paper. The methodological claim that IG is viable for dense retrieval explanation holds up as a proof of concept. The empirical claim about what domain adaptation does is plausible but not established. I would not cite it as evidence for that claim, but I would cite it as prior work using this baseline.\n\nThis is a workshop-scale exploratory study. A serious referee should engage with it, and the right outcome would be a revision that adds a frequency-controlled analysis and releases code and data. For a full conference I would be skeptical; for a workshop or short-paper track I would lean accept.","headline":"A plausible, honest exploratory study applying Integrated Gradients to dense retrievers, but the main generalization rests on frequency-confounded word clouds and two anecdotal queries.","tokens_in":8763,"tokens_out":1960,"would_cite":false,"duration_ms":20172,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By applying Integrated Gradients to dense retrievers, this paper shows that unsupervised domain adaptation shifts token-level attribution toward domain-specific vocabulary, and argues that Integrated Gradients gives a viable window into…","keywords":["integrated gradients","dense retrieval","domain adaptation","interpretability","input attribution","unsupervised domain adaptation","neural information retrieval"],"falsifier":"Compute token attributions for a larger, randomly drawn set of queries (say 100 per dataset) for both the adapted and the base model, define a fixed list of in-domain terms per dataset, and test whether the mean positive attribution for those terms is statistically significantly higher in the adapted model; if the difference is absent or within noise, the paper's qualitative pattern would not survive in aggregate.","tokens_in":7802,"feed_emoji":"🔍","tokens_out":6041,"duration_ms":51936,"temperature":0.7,"pith_summary":"Dense retrievers map queries and documents into vectors, but what they actually attend to is hard to see. This paper adapts the Integrated Gradients attribution method to dense retrieval, with a new baseline that attributes importance separately to query and document tokens, and applies it to models before and after unsupervised domain adaptation on a financial and a biomedical dataset. The central finding is that the adapted models place more positive attribution on in-domain terminology such as \"hedge,\" \"gold,\" \"corona,\" and \"disease\" than the unadapted base models do. The paper also argues that Integrated Gradients is a viable tool for explaining dense retrievers, including both single-pair (instance) explanations and explanations of an entire ranking. If correct, this gives IR practitioners a way to inspect what domain adaptation changes inside the model.","feed_headline":"Adapted retrievers shift focus to in-domain terms","feed_subtitle":"Token attributions show adapted models weight words like hedge, gold, and corona more heavily than base models.","key_machinery":"The central object is a modified Integrated Gradients baseline for dense retrievers. Integrated Gradients attributes a model's output to input tokens by integrating gradients along a straight path from a baseline input to the actual input; the paper's baseline uses padding tokens, replacing query tokens while keeping the document fixed to get query attributions, and replacing document tokens while keeping the query fixed to get document attributions. This yields instance-based explanations for a single query-document pair. For ranking-based explanations, the method sums token attributions over the top 25 retrieved documents for a query, so tokens that recur in highly ranked documents and contribute positively receive large aggregated scores. This aggregation turns the attribution method into a tool for comparing how base and domain-adapted models weight vocabulary across an entire ranked list.","core_discovery":"The paper's central claim is that Integrated Gradients can be used to produce token-level explanations for dense retrievers, and that applying this method reveals a consistent behavioral shift after domain adaptation: the adapted model concentrates its positive attributions on vocabulary specific to the target domain, while the unadapted model attends to more generic terms. To obtain query and document attributions, the paper replaces query tokens with padding tokens while leaving the document unchanged, and vice versa, running integrated gradients from those baselines. It then aggregates token attributions over the top 25 retrieved documents to produce ranking-level word clouds. On TREC-COVID and FIQA, the attributed tokens shift from generic words toward terms like \"corona,\" \"disease,\" \"hedge,\" and \"gold,\" and for TREC-COVID the adapted model assigns positive attribution to document titles where the base model assigns negative attribution. The paper interprets this as evidence that unsupervised domain adaptation teaches the retriever to rely on domain vocabulary and document structure rather than generic overlap.","pith_inferences":["A direct next step would be to convert the word-cloud comparisons into a quantitative attribution-difference score over a fixed domain vocabulary list; the paper notes that no standard quantitative evaluation of attributions exists, so such a metric would be a natural extension.","If the title-attribution result generalizes, then unsupervised adaptation on title-concatenated documents may teach retrievers to lean on titles; one way to test this is to remove titles at inference time and compare effectiveness drops for adapted and base models.","The same baseline-based attribution procedure could be applied to other unsupervised adaptation objectives, such as query generation or knowledge distillation, to see whether the in-domain vocabulary focus is specific to pseudo-labeling or common to all adaptation methods.","Because the paper's ranking attributions are averages over top documents, a fully global explanation could be built by averaging over many queries and documents; the paper flags global attributions as out of reach for Integrated Gradients, so this would be a different tool rather than a trivial extension."],"forward_implications":["If domain-adapted retrievers focus on in-domain terms, then explanations of retrieved results can be read as evidence about what vocabulary the model learned, not just which documents match.","The proposed baseline gives a reusable way to attribute dense retrieval scores to query and document tokens, applicable to any dot-product bi-encoder.","Ranking-level attributions let practitioners inspect not only a single match but the set of top documents, revealing terms that drive an entire ranking.","For TREC-COVID specifically, the title attribution result suggests that unsupervised adaptation can make a model rely more heavily on document structure (titles), a behavior with consequences for how documents should be formatted for retrieval."],"supporting_citations":[{"why":"Supplies the Integrated Gradients method and the baseline requirements (zero signal, zero score) that the paper adapts to dense retrieval.","marker":"[14]"},{"why":"Provides the GPL unsupervised domain adaptation models and the title-concatenation setup analyzed in the experiments.","marker":"[17]"},{"why":"Provides the MS MARCO fine-tuned DistilBERT dense retriever used as the base model before adaptation.","marker":"[2]"},{"why":"Shows Integrated Gradients with padding-token baselines on a BERT cross-encoder, the starting point this paper extends to dense retrievers.","marker":"[20]"},{"why":"Describes synthetic query generation for domain adaptation, an alternative approach the paper contrasts with GPL.","marker":"[10]"},{"why":"Supplies the sentence-embedding library used to load and run the dense retriever models.","marker":"[11]"}],"fun_headline_variants":["Integrated gradients reveal why adapted retrievers favor domain terms","Adapted retrievers shift attributions to in-domain vocabulary","Explaining domain adaptation in dense retrievers with token attributions","Why domain-adapted retrievers focus on niche terms: a gradient view","Token-level explanations show adapted retrievers emphasize domain words"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central observation rests on a small set of example queries, which the paper itself calls a deep qualitative analysis of a small sample of queries, with one query-document pair per model per dataset and attributions summed over the top 25 retrieved documents, so the claim that domain adaptation shifts attribution to in-domain terms depends on those examples being representative.","fun_headline_variants_meta":{"raw":{"variants":["Integrated gradients reveal why adapted retrievers favor domain terms","Adapted retrievers shift attributions to in-domain vocabulary","Explaining domain adaptation in dense retrievers with token attributions","Why domain-adapted retrievers focus on niche terms: a gradient view","Token-level explanations show adapted retrievers emphasize domain words"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000618,"raw_usage":{"total_tokens":2878,"prompt_tokens":968,"completion_tokens":1910,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":1827}},"tokens_in":584,"tokens_out":1910,"duration_ms":11716,"temperature":1.0,"reasoning_tokens":1827,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:07:54.648906+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute token attributions for a larger, randomly drawn set of queries (say 100 per dataset) for both the adapted and the base model, define a fixed list of in-domain terms per dataset, and test whether the mean positive attribution for those terms is statistically significantly higher in the adapted model; if the difference is absent or within noise, the paper's qualitative pattern would not survive in aggregate.","supporting_citations":[],"review_version":1}