{"id":"aaa3a9ea-6ced-4926-b2e3-16e05566a050","arxiv_id":"2607.04270","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":1,"one_line_summary":"LBR removes length bias in LLM recommenders via length-aware attention offsets and Trie-branching information-length normalization, improving accuracy and fairness with negligible cost.","lead":"LLM recommenders systematically favor items by title length: longer texts grab more attention, while raw decoding scores punish long titles and naive length penalties reverse the bias. LBR fixes both sides with a length-offset in attention and an information-length normalizer based on Trie branching, lifting NDCG@5 by ~17% on Amazon data.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified","rationale":"The paper cleanly diagnoses a dual length bias that is real under Trie-constrained LLM recommendation and supplies a lightweight, model-agnostic fix whose empirical gains are consistent (avg. +16.82% NDCG@5), ablated, and accompanied by released code. The reader correctly flags that Lemma 1 rests on mild, unproven assumptions (A1 concentration and A2 multiplicative length trend) and that linear g(l) plus Hartley entropy are modeling choices. Those points are accurate but do not threaten the central systems claim: LAAC parameters a,b are learned jointly with the recommender, Fig. 6 already shows the desired length-invariance after calibration, and non-linear g(l) is reported to add little. EILN’s information-theoretic length is a reasonable, Trie-aware alternative to token-count normalization whose failure modes are already illustrated by the α-sensitivity study. No circular derivation, missing control, or contradictory result appears. Therefore the ACCEPT verdict with high confidence remains appropriate; the concrete test above is only a useful diagnostic, not a required gate. Agreement with the reader is full on both the identified soft spot and its limited practical weight.","tokens_in":20596,"tokens_out":543,"duration_ms":6663,"concrete_test":"Re-train LAAC on Toy/Office while freezing g(l) to the empirical linear fit from Fig. 3 (no joint optimization) and re-measure NDCG@5 plus the post-calibration attention–length correlation of Fig. 6; if accuracy drops >5% relative or residual length–attention slope remains large, the end-to-end adaptation is doing more work than the paper’s theoretical story suggests.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (consistent accuracy/fairness gains from dual length-bias mitigation) is supported by consistent results across three datasets and two backbones, ablations, and released code. The reader’s weakest assumption (A2 of Lemma 1) is real but non-load-bearing: LAAC is jointly optimized end-to-end under the recommendation loss rather than relying on a fixed pre-estimated g(l), and the paper already reports that non-linear alternatives give negligible gains. Empirically, Fig. 6 shows post-calibration attention becomes nearly length-invariant and Table 3 shows both components contribute. The lemma is only an approximate justification under mild concentration assumptions; the practical method does not stand or fall with exact multiplicativity. No internal inconsistency or critical experimental gap undermines the systems claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper identifies length bias in LLM-based sequential recommenders that represent items by variable-length text: longer items accumulate more attention mass on the input side, while summed autoregressive log-likelihood on the output side (under Trie-constrained beam search) systematically favors short items; standard length normalization can reverse the bias and hurt accuracy. It proposes LBR, a lightweight model-agnostic fix consisting of (i) Length-Aware Attention Calibration (LAAC), which adds a learned length-dependent offset δ(l_v)=−log(g(l_v)) with linear g(l)=al+b into attention logits, and (ii) Effective Information Length Normalization (EILN), which replaces raw token count by a Hartley-entropy surrogate U_y=∑log2|V_k| derived from Trie branching factors and reweights token log-probabilities accordingly. Experiments on three Amazon datasets (Toy, Office, Book) with BIGRec and LLaRA backbones report average NDCG@5 gains of ~16.8% over strong baselines, ablations isolating both components, attention-weight and length-group exposure plots, and negligible overhead; code is released.","tokens_in":20861,"tokens_out":1077,"duration_ms":15959,"significance":"If the dual-bias diagnosis and the reported gains hold, the work is a useful systems contribution for the growing class of generative LLM recommenders that decode under catalog Tries. The problem is concrete and previously under-addressed; the remedies are simple, model-agnostic, and cheap (two extra scalars plus O(N) bookkeeping). Strengths include consistent gains across three datasets and two backbones, component ablations (Table 3), direct evidence that post-calibration attention becomes nearly length-invariant (Fig. 6) and that exposure variance across length groups shrinks (Fig. 7), statistical-significance claims, and public code. The information-theoretic length surrogate is a clean, falsifiable design choice that improves on both unnormalized scoring and classical length penalty. The contribution is primarily empirical/systems rather than theoretical, but that is appropriate for the venue and claim.","major_comments":[{"comment":"§4.1 and Appendix F, Lemma 1 / Assumption (A2): the theoretical claim that calibrated expected attention is length-invariant rests on a multiplicative pre-calibration model E[Z_u(i)|l_i=l]=κ_u g(l) and a concentration approximation for the softmax normalizer. The paper already notes that g is learned end-to-end under the recommendation loss (not fitted to attention) and that non-linear alternatives give negligible gains; Fig. 6 further shows the practical effect. The lemma is therefore only an approximate justification. Either weaken the claim to “motivating analysis under mild assumptions” or add a short empirical check that the learned g recovers a roughly multiplicative trend on held-out attention mass, so the theory and practice stay aligned.","section":null},{"comment":"§5.2 / Table 1 and §5.4 / Fig. 7: fairness is asserted via reduced variance of recommendation proportions across length groups that are equalized by cumulative item frequency. This is a reasonable proxy, but it is not a standard item-side fairness metric (e.g., exposure parity conditioned on popularity or provider-level metrics). A brief additional analysis that the accuracy gains are not driven by a reallocation that harms already under-exposed long/short items (or a simple popularity-controlled length-group breakdown) would make the fairness claim more robust without changing the experimental design.","section":null}],"minor_comments":[{"comment":"Eq. (9) is labeled s_EIF while the surrounding text uses EIL / EILN; unify the acronym (EILN vs EIF) for consistency.","section":null},{"comment":"Fig. 4 caption and §3.2: “number of valid tokens” is clear, but a short note that |V_k| is the Trie branching factor at step k would help readers who skip the figure.","section":null},{"comment":"Appendix D (equal-length truncation/padding and RQ-VAE semantic IDs) is useful negative evidence; a one-sentence pointer in the main §3.1 would make the “naive remedies fail” claim easier to verify.","section":null},{"comment":"Table 1 improvement percentages are relative to the best baseline per column; stating this explicitly in the caption would avoid ambiguity when the best baseline changes across metrics.","section":null},{"comment":"Minor typography: “Length-A ware” (space) appears in the abstract and §4.1; “seemingly integrated” in §5.2 should be “seamlessly”.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The central systems claim is well supported; the two major points are polish rather than load-bearing flaws. Fit for a solid IR/RecSys venue is good. No novelty or citation-pattern concerns stood out."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean, useful systems paper. The real contribution is the joint diagnosis: longer item texts pull more aggregate attention on the input side, while plain log-likelihood (and even classical length normalization) misbehaves under Trie-constrained decoding because tokens at low-branch nodes are easy “ghost” tokens. Their Effective Information Length—Hartley entropy of the valid-token set at each Trie step—is a domain-specific, sensible surrogate that classical NMT length penalties never needed. LAAC is just a learned linear offset in the attention logits; both pieces are lightweight and model-agnostic.\n\nWhat they do well: consistent NDCG/Hit lifts on three Amazon sets and two backbones (BIGRec, LLaRA), average ~17% NDCG@5 over the best baseline, ablations that isolate both modules, attention and exposure-distribution plots that actually show the bias shrinking, and released code. They also show why naive equal-length padding, RQ-VAE IDs, and D3-style α=0 do not solve the problem. The lemma in the appendix is only mild justification under concentration and multiplicative-bias assumptions; it is not load-bearing because a and b are trained end-to-end with the rec loss and non-linear g(l) gave them nothing extra. Fig. 6 and the ablations carry more weight than the proof.\n\nSoft spots are minor and proportionate. The linear g(l) and Hartley choice are modeling decisions, not derived necessities; evaluation stays inside the usual Amazon sequential-rec setting with LLaMA-3.2-3B. No circular metrics, no hidden free parameters that dominate the claim. Citation pattern is normal for the subfield.\n\nThis is for people building or auditing LLM sequential recommenders and anyone working on constrained generation. It deserves a serious referee; I would accept it for peer review and would cite the EIL idea if I touch Trie decoding or length fairness. Bring it to reading group if the group cares about practical recsys bias.","headline":"Solid systems paper that cleanly diagnoses dual length bias in LLM recommenders and ships a cheap, effective fix with real gains and code.","tokens_in":21420,"tokens_out":505,"would_cite":true,"duration_ms":5395,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"LLM recommenders favor long or short titles by design; LBR removes that length bias on both attention and decoding.","keywords":["Sequential Recommendation","Large Language Models","Length Bias","Attention Calibration","Constrained Decoding","Trie","Fairness"],"falsifier":"Re-run the attention-mass versus length plots (Figure 3 / Figure 6) after calibration; if a clear positive slope remains, or if NDCG@5 gains disappear once items are forced to equal token length, the central claim fails.","tokens_in":21513,"feed_emoji":"📏","tokens_out":836,"duration_ms":9337,"temperature":0.7,"pith_summary":"When large language models recommend products from text titles, item length quietly warps the answer. Longer titles grab more attention mass in the model’s context, so they influence preference modeling more than they deserve; when the model scores candidates by summing log-probabilities, short titles win simply because fewer negative terms are added. Ordinary length normalization flips the bias the other way, because under a Trie of valid catalog tokens many positions are nearly deterministic and therefore cheap to predict. The paper’s claim is that both distortions can be cancelled with a lightweight, model-agnostic fix called LBR: a length-dependent offset inside the attention logits, plus a new score that normalizes by the total branching uncertainty along each item’s path rather than by raw token count. On three Amazon catalogs and two LLM recommenders the method raises ranking metrics by roughly seventeen percent on average while making exposure across length groups far more even, at almost no extra training or inference cost.","feed_headline":"LLM recommenders bias by title length; LBR cancels both sides","feed_subtitle":"Attention offset plus Trie-branching scores lift NDCG@5 ~17% with almost no extra cost","key_machinery":"LBR: Length-Aware Attention Calibration injects a learned offset −log(g(l)) into attention logits so cumulative attention becomes roughly length-invariant; Effective Information Length Normalization replaces token count by the sum of Hartley entropies log2|V_k| along the Trie path, then weights each log-probability by that same informativeness before averaging.","core_discovery":"Length bias is a structural artifact of token-level LLM recommendation: longer item texts accumulate excess attention on the input side, while summed log-likelihood (and naive length normalization) systematically favor short or long items on the output side. LBR removes both effects by Length-Aware Attention Calibration and Effective Information Length Normalization, producing higher accuracy and fairer length exposure with negligible overhead.","pith_inferences":["The same branching-entropy normalizer could improve constrained generation outside recommendation (e.g., product-attribute completion or catalog-constrained dialogue).","If the length–attention relationship turns out non-linear on other domains, a richer parametric g(l) could be swapped in without altering the rest of LBR.","Jointly modeling length bias with known popularity and position biases may yield further gains once the two axes are no longer confounded."],"forward_implications":["Any Transformer recommender that concatenates free-text item titles can plug in the same two corrections without changing architecture.","Catalog exposure becomes closer to the true length distribution in the data, reducing pressure on sellers to invent unnaturally short or padded titles.","Conventional length-penalty hyperparameters become unnecessary once branching entropy is used as the normalizer.","Fairness audits of LLM recommenders should report length-group variance alongside popularity and position metrics."],"fun_headline_variants":["LBR cuts dual length bias in LLM recommenders via attention offset and trie norms","Length bias skews LLM recs on input and output; LBR calibrates both sides","LLM item texts bias attention and scores; LBR neutralizes with length-aware fixes","LBR: length-aware attention plus effective length norm for unbiased LLM recs","Token-length artifacts in LLM recommenders fixed by LBR calibration and norm"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The expected attention mass an item receives before calibration is assumed to be a simple multiplicative function of its length, so subtracting the log of that function cancels the bias.","fun_headline_variants_meta":{"raw":{"variants":["LBR cuts dual length bias in LLM recommenders via attention offset and trie norms","Length bias skews LLM recs on input and output; LBR calibrates both sides","LLM item texts bias attention and scores; LBR neutralizes with length-aware fixes","LBR: length-aware attention plus effective length norm for unbiased LLM recs","Token-length artifacts in LLM recommenders fixed by LBR calibration and norm"]},"model":"grok-4.5","effort":"low","cost_usd":0.005928,"raw_usage":{"total_tokens":1592,"prompt_tokens":858,"num_sources_used":0,"completion_tokens":108,"cost_in_usd_ticks":59280000,"prompt_tokens_details":{"text_tokens":858,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":626,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":858,"tokens_out":108,"duration_ms":6689,"temperature":1.0,"reasoning_tokens":626,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T20:28:42.063325+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the attention-mass versus length plots (Figure 3 / Figure 6) after calibration; if a clear positive slope remains, or if NDCG@5 gains disappear once items are forced to equal token length, the central claim fails.","supporting_citations":[],"review_version":1}