{"id":"2aa02c61-87c1-4284-b264-e8181df08e16","arxiv_id":"2412.03886","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"UDIG is an attribution algorithm that chooses uniform, vocabulary-near interpolation points between a MASK baseline and an input token, improving three faithfulness metrics over Integrated Gradients and Discretized Integrated Gradients on BERT and DistilBERT.","lead":"The paper introduces UDIG, a new method for explaining language model predictions by attributing importance to individual words. It selects anchor words close to the embedding-space path between a MASK baseline and the input, and reports better log-odds, comprehensiveness, and sufficiency scores than two existing attribution methods on sentiment and question-answering tasks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed advantage of UDIG’s uniform and bounded interpolation is confounded by the simultaneous switch from PAD to MASK baseline in §2.3; no ablation separates the two.","rationale":"The reader’s conditional verdict identifies anchor displacement and delta error as the weak point. I see a more basic confound. The method’s novelty is twofold: a uniform and bounded interpolation path and a MASK baseline. The two are never separated. Since the MASK baseline is justified in §2.3 as better than PAD, and all benchmark tables compare UDIG with MASK against DIG with PAD and IG with its usual baseline, the headline improvement cannot be assigned to the interpolation strategy. This is the single most load-bearing issue because if the baseline alone explains the gains, the central conceptual contribution (“uniform and bounded points”) is not supported, even though the package may still be a useful method. The proposed ablation is cheap and decisive. I therefore keep the verdict conditional: accept only if the baseline-controlled comparison confirms the path mechanism, or if the claim is weakened to describe the full UDIG recipe without causal attribution to path uniformity.","tokens_in":7812,"tokens_out":6855,"duration_ms":66592,"concrete_test":"Run a 2×2 baseline/path ablation on the released code for BERT-SST2 and DistilBERT-SST2 (or one of the other datasets): (i) UDIG with PAD baseline, (ii) UDIG with MASK baseline, (iii) DIG with PAD baseline, (iv) DIG with MASK baseline, holding fixed the number of steps, neighborhood size k, greedy/max-count variant, and evaluation procedure. Report log odds, comprehensiveness, sufficiency, and delta percent for each cell. If UDIG-PAD beats DIG-PAD by roughly the same margin as UDIG-MASK beats DIG-MASK, the path-uniformity mechanism is confirmed; if DIG-MASK closes the gap to UDIG-MASK, the MASK baseline is the active ingredient.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper’s central explanatory claim — that choosing uniform and bounded anchor words along the path is what makes UDIG outperform IG and DIG — is not isolated experimentally. Section 2.3 changes the baseline from the PAD token used by DIG to the MASK token, and Table 1 is offered as evidence that MASK gives near-zero model output. The paper even states that this baseline choice “resulted in better performance metrics and word attributions.” Because Tables 2–5 vary both the interpolation algorithm and the baseline simultaneously, any observed improvement could be attributable to the baseline rather than to uniformity or boundedness of the anchors. The conclusion’s assertion that “keeping the interpolation points uniform and bounded yields better results” therefore rests on an untested attribution of cause. This is not an internal mathematical inconsistency; it is a missing control in the empirical argument, and it is directly testable with the authors’ released code.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Uniform Discretized Integrated Gradients (UDIG), a path-based attribution method that modifies Discretized Integrated Gradients (DIG) by placing interpolation points uniformly along the straight line between the target word embedding and a MASK-token baseline, then selecting nearest vocabulary words as anchors and enforcing monotonicity before applying a Riemann-sum approximation. The method is evaluated on binary sentiment classification (SST-2, IMDb, Rotten Tomatoes) using BERT and DistilBERT, and on extractive question answering using SQuAD, with log odds, comprehensiveness, and sufficiency as evaluation metrics. The paper reports that UDIG outperforms IG and DIG on almost all metric/task combinations, also shows lower median delta approximation error, and introduces the MASK token as a baseline recommendation.","tokens_in":8057,"tokens_out":4760,"duration_ms":44312,"significance":"If the reported gains are robust, UDIG is a simple, practical improvement over DIG that offers a useful trade-off between following the straight-line IG path and staying close to actual vocabulary words. The release of code on GitHub is a strength, as is the use of external faithfulness metrics (log odds, comprehensiveness, sufficiency) rather than self-similarity of attributions. The main limitations are the absence of an ablation separating the baseline change from the interpolation change, an unsupported implementation-invariance claim, and insufficient reporting of variance and sampling details. These issues currently prevent the reader from fully attributing the observed improvements to the proposed interpolation strategy.","major_comments":[{"comment":"The comparison between UDIG and DIG varies two factors simultaneously: the interpolation algorithm and the baseline token (MASK in §2.3 versus PAD for DIG). The paper itself notes in §2.3 that the MASK token 'resulted in better performance metrics and word attributions,' and the central conclusion in §5 that 'keeping the interpolation points uniform and bounded yields better results' is therefore not isolated by the experiments. An ablation is needed that runs UDIG with the PAD baseline and DIG with the MASK baseline; without it, the superiority in Tables 2–5 could be due entirely to the baseline change.","section":"§2.3, Tables 2–5, §5"},{"comment":"The claim that UDIG satisfies Implementation Invariance because it is a path-integral method is not valid as stated. The interpolation path in §2.2 is constructed through nearest-neighbor search in the model's word-embedding space and then by perturbing anchor embeddings to enforce monotonicity. Two functionally equivalent models with different embedding parameterizations, for instance after permuting embedding dimensions, would generally select different anchor words and hence different paths, leading to different attribution scores. The paper should either restrict the axiom claim to Sensitivity and Completeness, or define a class of models under which the path is invariant.","section":"§2.4"},{"comment":"The paper claims in §2.4 that UDIG satisfies the Completeness axiom, but the approximation error reported in §4.2 is substantial: median delta error is 25.89% for BERT on SST-2 and 25.95% on Rotten Tomatoes, and several DIG rows exceed 100%. No threshold is given for what counts as acceptable delta error, even though §4.2 says the error 'must be within a threshold.' Since the Riemann-sum approximation in Eq. (2) is the quantity actually used to produce attributions, a high delta error weakens the direct link to the exact path integral and the completeness property. The authors should report the distribution of delta errors, justify why the approximation remains faithful at these error levels, or temper the completeness claim.","section":"§4.2, Tables 6–9"},{"comment":"The empirical support is under-specified. For IMDb, §3.1 states that 'around 650 examples' were sampled; for SQuAD, §3.2 says 950 question-answer pairs were sampled 'based on the length of the context,' but the exact counts, sampling procedure, and random seeds are not given. In addition, Tables 2–5 report single point estimates with no variance, confidence intervals, or significance tests. Without these details, it is impossible to assess whether the reported improvements are stable or within noise, and the experiments are not fully reproducible as described.","section":"§3.1, §3.2, Tables 2–5"}],"minor_comments":[{"comment":"The term 'bounded' in the title and conclusion is never formally defined. The description of selecting anchors 'in the vicinity' of each point lacks a definition of the neighborhood radius or how boundedness is quantified after the monotonicity perturbation.","section":"Title and §2.2"},{"comment":"Table 1 reports model outputs at a single baseline sequence for each length, with no standard errors or repetition. The claim that MASK is 'almost zero' and 'equally unbiased' would be stronger with multiple random sequences and measures of variability.","section":"Table 1"},{"comment":"The typesetting of Eq. (1) is unclear; the integral notation and the definitions of the variables are hard to parse, making it difficult to match Eq. (1) to the Riemann-sum expression in Eq. (2).","section":"Eq. (1)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript makes an incremental but potentially useful contribution over DIG. The main substantive barrier is the confounded baseline change, which a simple ablation can resolve. The implementation-invariance claim should be corrected rather than simply asserted. Given the moderate empirical support and the availability of code, I would support a revised version if the authors provide the ablation, variance reporting, and detailed sampling information."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: UDIG is a real algorithmic variant of DIG, not a rehash. The idea—sample uniform points along the straight line first, then snap to nearby vocabulary anchors and enforce monotonicity—is clean, and the delta-error comparison is a nice diagnostic. But the paper's headline claim, that uniform/bounded interpolation is what drives the improvement, is not supported by the experiments as presented. Section 2.3 switches the baseline from PAD to MASK at the same time and states that MASK gave better metrics. Tables 2–5 therefore vary two things at once. Without an ablation (UDIG with PAD, DIG with MASK), the causal story is untested. The stress-test note is right. What the paper does well: the delta-error tables (6–9) consistently show UDIG below DIG, and these are computed on the completeness gap rather than on the downstream faithfulness metrics, so they are less contaminated by the baseline switch. The time-complexity appendix is candid, and the code is released. Soft spots besides the confound: no error bars or significance tests; the IMDB and SQuAD sampling is described with 'around' and no seed; the delta error is still 15–31% after up-sampling, so completeness is only approximate; and the claim that MASK is a better baseline is plausible but not isolated. The notation in Equation (1) is also a bit garbled, though that is minor. Net: this is a legitimate incremental contribution to token-attribution methods. It does not open a new line of research, but for practitioners comparing IG/DIG/UDIG on a budget it is informative. The central claim should be reworded as 'our combination of uniform anchoring and the MASK baseline gives better results,' pending an ablation. That ablation is straightforward with the released code. Who should read it: people doing explainability for transformer classifiers and QA models, especially those using IG-style path integrals in discrete embedding spaces. It is not for theorists; there is no worst-case guarantee and the delta analysis is empirical. Recommendation: send it to review. The method is plausible, the code exists, and the missing control is fixable. A good referee would ask for the ablation, variance, and detailed sampling. I would not cite it in its current form for the superiority claim, but the delta-error comparison is worth remembering.","headline":"A plausible attribution variant whose main claim is currently confounded by its baseline switch; worth refereeing with an ablation as the condition.","tokens_in":697,"tokens_out":1718,"would_cite":false,"duration_ms":34507,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes UDIG, which computes word attributions by integrating model gradients along a nonlinear path of uniformly spaced, monotonic anchor words, and reports that it outperforms IG and DIG on sentiment classification and…","keywords":["Large Language Models","Explainable AI","Input attribution","Integrated Gradients","Discretized Integrated Gradients","Uniform interpolation","BERT","Question Answering"],"falsifier":"On a model and dataset where the vocabulary is sparse along the baseline-to-target direction, compute UDIG's median delta-percent error and compare attribution rankings with a dense numerical integral; if the delta stays near or above the reported roughly 25 percent even after up-sampling, or if rankings change when the neighborhood size $k$ changes, the claim that the perturbed anchors still track the intended integral would be falsified.","tokens_in":7608,"feed_emoji":"🔤","tokens_out":7577,"duration_ms":71636,"temperature":0.7,"pith_summary":"Integrated Gradients explains a model by accumulating gradients along a straight line from a baseline to the input, but in embedding spaces those intermediate points are not real words, so the gradients can be unrepresentative. This paper proposes Uniform Discretized Integrated Gradients (UDIG), which samples evenly spaced points on the straight line and then replaces each point with a nearby real word, called an anchor, that is made monotonic with respect to the baseline and the target word. UDIG also uses the MASK token as the baseline instead of PAD, because the model output at MASK is closer to zero. On BERT and DistilBERT, across SST2, IMDb, Rotten Tomatoes, and SQuAD, UDIG improves log-odds, comprehensiveness, and sufficiency relative to IG and DIG in almost all comparisons. The paper interprets this as evidence that keeping the interpolation points uniform and bounded yields more accurate attribution scores.","feed_headline":"UDIG path method tops IG and DIG on text attributions","feed_subtitle":"Uniformly spaced anchor words and a MASK baseline yield better log-odds, comprehensiveness, and sufficiency scores.","key_machinery":"The central object is the interpolation path itself. Starting from a target word embedding $w$ and a baseline $w_0$ (the MASK token embedding), UDIG places $K$ uniformly spaced points on the line segment, searches each point's neighborhood for candidate words, and selects an anchor by either the Greedy rule (pick the word closest to its monotonic form) or the Max-Count rule (pick the word with the most monotonic dimensions). The selected anchors are then made monotonic dimension-wise, which lets the Riemann sum $\\sum_{k} \\frac{\\partial F(x_{ij}^k)}{\\partial x_{ij}} (x_{ij}^{k+1}-x_{ij}^k)$ approximate the path integral in equation (1). Because the attribution score is still a path integral, it satisfies sensitivity, implementation invariance, and completeness; the MASK baseline's near-zero model output is what makes the completeness identity numerically meaningful.","core_discovery":"The paper's central claim is that attribution quality for language models improves when a path-integral method is both discretized onto real vocabulary words and kept geometrically disciplined. UDIG fixes $K$ evenly spaced points on the straight segment from a baseline embedding to the target word embedding, replaces each point with a nearby anchor word from the vocabulary, and perturbs non-monotonic dimensions so the selected anchors are monotonic with respect to the baseline and target. This keeps the path nonlinear enough to accumulate meaningful word-level gradients while staying close enough to the straight line to avoid the erratic paths that hurt DIG. The paper's empirical claim is that, on BERT and DistilBERT across SST2, IMDb, Rotten Tomatoes, and SQuAD, UDIG yields better log-odds, comprehensiveness, and sufficiency scores than IG and DIG in almost all comparisons, and that its median delta-percent error is consistently lower, supporting the view that uniform, bounded interpolation points make attribution computation more faithful.","pith_inferences":["A natural extension, not tested in the paper, is to replace the Greedy and Max-Count anchor rules with an anchor choice that explicitly minimizes post-perturbation distance to the straight line; the residual delta error suggests such a rule could push completeness error below the reported medians.","The delta-percent error could serve as a practical screening test for any path-based attribution method, since it quantifies how far a discretized path is from satisfying the completeness identity.","UDIG's largest delta reductions occur on cases where DIG's delta is very high, such as the Rotten Tomatoes BERT setting, which suggests the method has the most to gain in embedding spaces where real words lie sparse and far from linear interpolations."],"forward_implications":["If UDIG's uniform, bounded path is the reason for the improvement, then up-sampling the path should reduce the delta-percent error further; the paper reports lower median delta than DIG on every tested model and dataset.","For models that contain a MASK token, the paper recommends using MASK rather than PAD as the attribution baseline because its model output is closer to zero and it behaves less like a learned class-specific token.","For models without a MASK token, such as GPT-style models, the paper says a zero embedding vector or the PAD token is a suitable fallback baseline.","At a fixed computation budget, UDIG needs fewer interpolation steps than IG or DIG to reach the same runtime, and on the Rotten Tomatoes set those fewer steps still improve log-odds and comprehensiveness.","The method is intended to apply to any predictive language model with token embeddings, not only to classification and question answering."],"supporting_citations":[{"why":"Supplies the Integrated Gradients path-integral definition and the axioms of sensitivity, implementation invariance, and completeness that UDIG inherits.","marker":"[4]"},{"why":"Defines Discretized Integrated Gradients, including the anchor-selection, monotonicity, and up-sampling procedures, and provides the log-odds, comprehensiveness, and sufficiency metrics used for comparison.","marker":"[5]"},{"why":"Provides the BERT architecture that UDIG explains and on which sentiment and question-answering attributions are evaluated.","marker":"[1]"},{"why":"Provides DistilBERT, the second sentiment-classification model benchmarked in the paper.","marker":"[6]"},{"why":"Supplies the SST2 sentiment dataset used for attribution benchmarking.","marker":"[7]"},{"why":"Supplies the IMDb sentiment dataset used for attribution benchmarking.","marker":"[12]"},{"why":"Supplies the Rotten Tomatoes sentiment dataset used for attribution benchmarking.","marker":"[13]"},{"why":"Supplies the SQuAD question-answering dataset used to benchmark start-token and end-token attributions.","marker":"[14]"}],"fun_headline_variants":["UDIG: word-anchored paths boost LLM attribution scores","Uniform discretized gradients beat IG and DIG on text","Path method with real word anchors improves LLM explainability","UDIG picks word anchors for better attribution in LLMs","New attribution method UDIG outperforms on sentiment and QA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the anchor words, after their non-monotonic dimensions are perturbed to enforce monotonicity, stay close enough to the straight line from the baseline to the target that gradients at those anchors approximate the true integral, and the paper offers no worst-case bound on that distance, only median delta percentages.","fun_headline_variants_meta":{"raw":{"variants":["UDIG: word-anchored paths boost LLM attribution scores","Uniform discretized gradients beat IG and DIG on text","Path method with real word anchors improves LLM explainability","UDIG picks word anchors for better attribution in LLMs","New attribution method UDIG outperforms on sentiment and QA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2968,"prompt_tokens":958,"completion_tokens":2010,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":1941}},"tokens_in":574,"tokens_out":2010,"duration_ms":26317,"temperature":1.0,"reasoning_tokens":1941,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:58:04.419145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a model and dataset where the vocabulary is sparse along the baseline-to-target direction, compute UDIG's median delta-percent error and compare attribution rankings with a dense numerical integral; if the delta stays near or above the reported roughly 25 percent even after up-sampling, or if rankings change when the neighborhood size $k$ changes, the claim that the perturbed anchors still track the intended integral would be falsified.","supporting_citations":[{"cited_title":"Axiomatic attribution for deep networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the Integrated Gradients path-integral definition and the axioms of sensitivity, implementation invariance, and completeness that UDIG inherits."},{"cited_title":"Discretized Integrated Gradients for Explaining Language Models,","cited_arxiv_id":null,"evidence_quote":"Defines Discretized Integrated Gradients, including the anchor-selection, monotonicity, and up-sampling procedures, and provides the log-odds, comprehensiveness, and sufficiency metrics used for comparison."},{"cited_title":"BERT: Pre-training of deep bidirectional transformers for language understanding.,","cited_arxiv_id":null,"evidence_quote":"Provides the BERT architecture that UDIG explains and on which sentiment and question-answering attributions are evaluated."},{"cited_title":"Distilbert, a distilled version of bert: smaller, faster, cheaper, and lighter,","cited_arxiv_id":null,"evidence_quote":"Provides DistilBERT, the second sentiment-classification model benchmarked in the paper."},{"cited_title":"Recursive deep models for semantic compositionality over a sentiment treebank,","cited_arxiv_id":null,"evidence_quote":"Supplies the SST2 sentiment dataset used for attribution benchmarking."},{"cited_title":"Learning word vectors for sentiment analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the IMDb sentiment dataset used for attribution benchmarking."},{"cited_title":"Seeing stars: Exploiting class relationships for sentiment categorization,","cited_arxiv_id":null,"evidence_quote":"Supplies the Rotten Tomatoes sentiment dataset used for attribution benchmarking."},{"cited_title":"SQuAD: 100,000+ Questions for Machine Comprehension of Text,","cited_arxiv_id":null,"evidence_quote":"Supplies the SQuAD question-answering dataset used to benchmark start-token and end-token attributions."}],"review_version":1}