Pith. sign in

REVIEW 1 major objections 2 minor 60 cited by

A 7-billion-parameter model outperforms Llama 2 13B on every benchmark and Llama 1 34B on reasoning, mathematics, and code generation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-24 06:09 UTC pith:UYV4HIS6

load-bearing objection Mistral 7B is a practical open release whose main value is the weights and the benchmark numbers, not new methods. the 1 major comments →

arxiv 2310.06825 v1 pith:UYV4HIS6 submitted 2023-10-10 cs.CL cs.AIcs.LG

Mistral 7B

Albert Q. Jiang , Alexandre Sablayrolles , Arthur Mensch , Chris Bamford , Devendra Singh Chaplot , Diego de las Casas , Florian Bressand , Gianna Lengyel
show 10 more authors
Guillaume Lample Lucile Saulnier L\'elio Renard Lavaud Marie-Anne Lachaux Pierre Stock Teven Le Scao Thibaut Lavril Thomas Wang Timoth\'ee Lacroix William El Sayed
This is my paper
classification cs.CL cs.AIcs.LG
keywords Mistral 7Blanguage modelgrouped-query attentionsliding window attentionbenchmarksLlama comparisoninstruction tuningmodel release
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents Mistral 7B, a 7B-parameter language model that beats substantially larger models on standard evaluations. It attributes the gains to two attention changes: grouped-query attention for quicker inference and sliding window attention for long sequences at lower cost. An instruction-tuned variant also exceeds Llama 2 13B Chat on both human and automatic checks. The models are released under Apache 2.0. The central point is that targeted architectural choices can deliver higher performance than simply increasing parameter count.

Core claim

Mistral 7B v0.1 is a 7-billion-parameter model that uses grouped-query attention and sliding window attention to outperform Llama 2 13B across all evaluated benchmarks and Llama 1 34B in reasoning, mathematics, and code generation. The instruction-tuned Mistral 7B Instruct version surpasses Llama 2 13B Chat on both human and automated benchmarks. The models are released under the Apache 2.0 license.

What carries the argument

Grouped-query attention paired with sliding window attention, which together reduce inference cost while supporting long sequences.

Load-bearing premise

The chosen benchmarks and test sets measure genuine downstream usefulness and contain no overlap with the training data.

What would settle it

A new benchmark suite free of training-data overlap on which Mistral 7B scores below Llama 2 13B, or direct evidence that the original test sets leaked into training.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Models of 7B parameters can exceed 13B-parameter models on common tasks when attention mechanisms are adjusted.
  • Inference speed improves without sacrificing sequence length handling.
  • Instruction tuning applied to the base model produces a chat version stronger than the corresponding Llama 2 variant.
  • Open release under Apache 2.0 allows direct use and further fine-tuning by others.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Efficiency-focused designs may shift emphasis from raw scale toward architecture in future model development.
  • Smaller models become more practical for on-device or low-resource deployment.
  • Continued benchmark saturation could prompt creation of harder, more leakage-resistant evaluation sets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 2 minor

Summary. The manuscript introduces Mistral 7B, a 7B-parameter language model that outperforms Llama 2 13B across all evaluated benchmarks and Llama 1 34B on reasoning, mathematics, and code generation. It employs grouped-query attention (GQA) and sliding window attention (SWA) for inference efficiency and long-sequence handling. An instruction-tuned variant (Mistral 7B -- Instruct) is also presented and shown to surpass Llama 2 13B -- Chat on human and automated benchmarks. The models are released under Apache 2.0.

Significance. If the empirical results hold, the work demonstrates that targeted architectural choices can enable smaller models to match or exceed larger ones on standard tasks, with direct implications for efficient deployment. The open release of weights supports independent verification and extension, which strengthens the contribution.

major comments (1)
  1. [Evaluation] Evaluation section: the headline claim that Mistral 7B outperforms Llama 2 13B on every reported benchmark rests entirely on the numerical scores, yet the manuscript provides no description of n-gram decontamination, membership-inference checks, or confirmation that the test sets (MMLU, HumanEval, GSM8K, etc.) were held out from training data. This directly affects the reliability of the reported margins.
minor comments (2)
  1. No training corpus composition, token count, or hyperparameter details are supplied, limiting reproducibility and contextualization of the efficiency claims.
  2. Benchmark tables lack error bars or multiple-run statistics, making it impossible to assess whether the observed differences are statistically meaningful.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the evaluation section. We address the concern below and will incorporate clarifications in the revised manuscript.

read point-by-point responses

  1. Referee: [Evaluation] Evaluation section: the headline claim that Mistral 7B outperforms Llama 2 13B on every reported benchmark rests entirely on the numerical scores, yet the manuscript provides no description of n-gram decontamination, membership-inference checks, or confirmation that the test sets (MMLU, HumanEval, GSM8K, etc.) were held out from training data. This directly affects the reliability of the reported margins.

    Authors: We agree that the manuscript lacks explicit details on these points, which is a valid concern for transparency. In the revised version we will add a dedicated paragraph in the Evaluation section describing our internal data curation process, including n-gram overlap checks performed to reduce contamination with the listed benchmarks and confirmation that the test sets were excluded from training. We did not run membership-inference attacks, as they are not standard practice in the majority of contemporaneous LLM papers and would require substantial additional compute; we will note this limitation explicitly. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical model release with benchmark comparisons

full rationale

The manuscript introduces Mistral 7B, describes architectural choices (GQA, SWA), and reports empirical benchmark results against prior models. No derivations, equations, fitted parameters presented as predictions, or load-bearing self-citations appear. The central claim is a direct empirical comparison of released weights; it is self-contained against external benchmarks and contains no steps that reduce to inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical model release rather than a derivation. No free parameters are fitted inside a mathematical claim; the only implicit assumptions are standard supervised language-model training and the validity of the chosen benchmarks.

pith-pipeline@v0.9.0 · 5728 in / 1048 out tokens · 22675 ms · 2026-05-24T06:09:42.492805+00:00 · methodology

0 comments
read the original abstract

We introduce Mistral 7B v0.1, a 7-billion-parameter language model engineered for superior performance and efficiency. Mistral 7B outperforms Llama 2 13B across all evaluated benchmarks, and Llama 1 34B in reasoning, mathematics, and code generation. Our model leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. We also provide a model fine-tuned to follow instructions, Mistral 7B -- Instruct, that surpasses the Llama 2 13B -- Chat model both on human and automated benchmarks. Our models are released under the Apache 2.0 license.

Figures

Figures reproduced from arXiv: 2310.06825 by Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, L\'elio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timoth\'ee Lacroix, William El Sayed.

Figure 1
Figure 1. Figure 1: Sliding Window Attention. The number of operations in vanilla attention is quadratic in the sequence length, and the memory increases linearly with the number of tokens. At inference time, this incurs higher latency and smaller throughput due to reduced cache availability. To alleviate this issue, we use sliding window attention: each token can attend to at most W tokens from the previous layer (here, W = … view at source ↗
Figure 2
Figure 2. Figure 2: Rolling buffer cache. The cache has a fixed size of W = 4. Keys and values for position i are stored in position i mod W of the cache. When the position i is larger than W, past values in the cache are overwritten. The hidden state corresponding to the latest generated tokens are colored in orange. Pre-fill and Chunking. When generating a sequence, we need to predict tokens one-by-one, as each token is con… view at source ↗
Figure 3
Figure 3. Figure 3: Pre-fill and chunking. During pre-fill of the cache, long sequences are chunked to limit memory usage. We process a sequence in three chunks, “The cat sat on”, “the mat and saw”, “the dog go to”. The figure shows what happens for the third chunk (“the dog go to”): it attends itself using a causal mask (rightmost block), attends the cache using a sliding window (center block), and does not attend to past to… view at source ↗
Figure 4
Figure 4. Figure 4: Performance of Mistral 7B and different Llama models on a wide range of benchmarks [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Results on MMLU, commonsense reasoning, world knowledge and reading comprehension for [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Human evaluation of Mistral 7B – Instruct vs Llama 2 13B – Chat Example. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CATCH-ME if you RAG: a dataset of Contextually Annotated multi-Turn Counterspeech against Hate and Misinformation Exchanges

    cs.CL 2026-06 unverdicted novelty 8.0

    Presents a new expert-curated dataset of multi-turn counterspeech dialogues in five languages targeting hate against seven groups, with span annotations linking to verified external knowledge for RAG applications.

  2. TW-LegalBench: Measuring Taiwanese Legal Understanding

    cs.CL 2026-06 unverdicted novelty 8.0

    TW-LegalBench evaluates 13 LLMs on over 30,000 Taiwanese legal tasks from exams and judgments, showing top models pass lawyer thresholds but struggle with exact statute citations.

  3. Entropy-Gated Latent Recursion

    cs.LG 2026-06 unverdicted novelty 8.0

    EGLR adds a deterministic layer-recursion axis gated by entropy that is complementary to temperature sampling, raising joint oracle accuracy on MATH-500 from 83.4% to 91.6% for a 3B model.

  4. Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generation

    cs.AI 2026-06 accept novelty 8.0

    Across 30 LLMs and 205 TLA+ tasks, syntactic correctness reaches at most 26.6% and semantic correctness 8.6%, with all successes limited to progressive prompting and no advantage from larger models.

  5. Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generation

    cs.CV 2026-06 unverdicted novelty 8.0

    A safety direction estimated in a source LLM is transported to a target generator through lightweight alignment on benign data alone, matching native safety performance without any target-side unsafe data.

  6. Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truth

    cs.CL 2026-05 unverdicted novelty 8.0

    Introduces BonaFide benchmark of 3,066 ground-truth labeled CoTs showing most faithfulness metrics perform near chance with biases and poor scaling to longer chains.

  7. RTI-Bench: A Structured Dataset for Indian Right-to-Information Decision Analysis

    cs.CL 2026-05 accept novelty 8.0

    RTI-Bench is the first publicly released structured dataset of CIC administrative decisions with outcome labels, exemption citations, IRAC reasoning, and timelines, built from 1,218 corpus cases and 298 PDFs, achievin...

  8. Privacy Auditing with Zero (0) Training Run

    cs.CR 2026-05 unverdicted novelty 8.0

    Zero-Run auditing supplies valid lower bounds on differential privacy parameters from fixed member and non-member datasets by modeling and correcting distribution-shift confounding via causal-inference techniques.

  9. Crafting Reversible SFT Behaviors in Large Language Models

    cs.LG 2026-05 unverdicted novelty 8.0

    LCDD creates sparse carriers for SFT behaviors that SFT-Eraser can reverse, with ablations showing the sparse structure enables causal control.

  10. HeadQ: Model-Visible Distortion and Score-Space Correction for KV-Cache Quantization

    cs.LG 2026-05 conditional novelty 8.0

    HeadQ removes 84-94% of excess perplexity from 2-bit key quantization by storing low-rank residuals in a calibration-learned query basis for score-space correction and using A²-weighted distortion for values.

  11. DurableUn: Quantization-Induced Recovery Attacks in Machine Unlearning

    cs.LG 2026-05 conditional novelty 8.0

    INT4 quantization recovers up to 22 times more forgotten training data in unlearned LLMs, and the proposed DURABLEUN-SAF method is the first to maintain forgetting across BF16, INT8, and INT4 precisions.

  12. Architecture Determines Observability of Transformers

    cs.LG 2026-04 unverdicted novelty 8.0

    Certain transformer architectures lose internal linear signals for decision quality during training, making observability an architecture-dependent property rather than a universal one.

  13. Backdoor Attacks on Decentralised Post-Training

    cs.CR 2026-03 conditional novelty 8.0

    An adversary controlling an intermediate pipeline stage in decentralized LLM post-training can inject a backdoor that reduces alignment from 80% to 6%, with the backdoor persisting in 60% of cases even after subsequen...

  14. CacheTrap: Unveiling a Stealthier Gray-Box Trojan against LLMs

    cs.CR 2025-11 conditional novelty 8.0

    CacheTrap achieves 100% targeted attack success on five open-source LLMs by using an efficient search to locate and flip a single bit in the KV cache as a transient trigger, while preserving normal accuracy without th...

  15. MediQAl: A French Medical Question Answering Dataset for Knowledge and Reasoning Evaluation

    cs.CL 2025-07 accept novelty 8.0

    MediQAl is a new French medical QA benchmark with 32k exam-sourced questions in three formats and cognitive labels, evaluated on 14 LLMs to reveal gaps between factual recall and reasoning performance.

  16. Large Language Diffusion Models

    cs.CL 2025-02 unverdicted novelty 8.0

    LLaDA is a scalable diffusion-based language model that matches autoregressive LLMs like LLaMA3 8B on tasks and surpasses GPT-4o on reversal poem completion.

  17. MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?

    cs.CV 2024-08 conditional novelty 8.0

    MME-RealWorld is the largest manually annotated high-resolution benchmark for MLLMs, where even the best models achieve less than 60% accuracy on challenging real-world tasks.

  18. LiveBench: A Challenging, Contamination-Limited LLM Benchmark

    cs.CL 2024-06 unverdicted novelty 8.0

    LiveBench is a contamination-limited LLM benchmark with auto-scored challenging tasks from recent sources across math, coding, reasoning and more, where top models score below 70%.

  19. ORPO: Monolithic Preference Optimization without Reference Model

    cs.CL 2024-03 conditional novelty 8.0

    ORPO performs preference alignment during supervised fine-tuning via a monolithic odds ratio penalty, allowing 7B models to outperform larger state-of-the-art models on alignment benchmarks.

  20. Evaluating Very Long-Term Conversational Memory of LLM Agents

    cs.CL 2024-02 unverdicted novelty 8.0

    Creates LoCoMo benchmark dataset for very long-term LLM conversational memory and shows current models struggle with lengthy dialogues and long-range temporal dynamics.

  21. Conditional Optimal Bridge for Riemannian Activation Steering

    cs.LG 2026-07 accept novelty 7.0

    Casting activation steering as a Schrödinger Bridge on the residual hypersphere derives the log-density-ratio objective and yields query-adaptive directions that beat fixed baselines without OOD collapse.

  22. Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data

    cs.CV 2026-07 accept novelty 7.0

    A camera-trap TVR benchmark of 135 ethology queries plus an interpretable SALMA-to-JSON plus constrained-LLM-parser pipeline yields 34% set F1, beating zero-shot VLMs at 18%.

  23. A Mechanistic Lens on Semantic Conflicts: Using Activation Patching to Understand LLM Behavior

    cs.SE 2026-07 conditional novelty 7.0

    Semantic conflicts between code and cues cut execution-grounded correctness; residual-stream patching localizes the causal signal to the edit site, sparse carriers, and readout.

  24. Most LLM Conformity Needs No Speaker: Measuring the Speaker-Free Floor in Peer-Pressure Benchmarks

    cs.CL 2026-07 accept novelty 7.0

    Across six open-weight LLMs and seven datasets, a speaker-free wrong-answer assertion alone flips 66.5% of initially correct answers, versus 10.3% for a plain re-ask; source labels mainly add a modest increment above ...

  25. Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models

    cs.CL 2026-07 conditional novelty 7.0

    Aligned LMs transiently commit to wrong mid-layer preferences that late layers rescue; this wrong-dip predicts structural compression flips, is recipe-specific and trainable, and is distinct from interface failure.

  26. They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On It

    cs.CL 2026-07 conditional novelty 7.0

    Models linearly represent Gricean communicative intent (recognize vs evaluate) from pretraining, but default readout often discards it; a late-layer discriminative direction recovers honoring as well as an explicit in...

  27. HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment

    cs.AI 2026-07 accept novelty 7.0

    HARC couples harmfulness and refusal directions at prompt and response positions, yielding the best robustness-capability-usability trade-off among major safety methods.

  28. Information Dynamics of Language Communication

    cs.CL 2026-06 unverdicted novelty 7.0

    The paper defines STE and SPID, two information-theoretic measures of semantic flow and decomposition in language exchanges, and applies them to four dialogue datasets.

  29. Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

    cs.CL 2026-06 conditional novelty 7.0

    Anisotropy, quantified by dominant-dimension variance fraction, determines the best parameter-free similarity metric for text embeddings, with rank-based metrics gaining ~20% relative where cosine is weakest.

  30. MultiHashFormer: Hash-based Generative Language Models

    cs.CL 2026-06 unverdicted novelty 7.0

    MultiHashFormer enables hash-based autoregression in LMs by encoding tokens as multi-hash signatures, outperforming standard Transformers at 100M-3B scales while keeping parameter count constant for multilingual expansion.

  31. Structure Before Collapse: Transient semantic geometry in next-token prediction

    cs.LG 2026-06 unverdicted novelty 7.0

    Semantic geometry emerges transiently early in next-token prediction training before collapsing to Neural Collapse symmetry in synthetic settings with latent semantic factors.

  32. SSM Adapters via Hankel Reduced-order Modeling: Injection Site Determines Task Suitability in Long-Context Fine-Tuning

    cs.LG 2026-06 unverdicted novelty 7.0

    HRM adapters via Hankel reduced-order modeling outperform LoRA on long-context tasks in Mistral-7B when used as SSM residual modules with FFT-based parallel scan.

  33. The Topology of Ill-Posed Questions: Persistent Homology for Detection and Steering in LLMs

    cs.AI 2026-06 unverdicted novelty 7.0

    Zero-dimensional persistent homology on transformer layer hidden states yields three descriptors per layer whose concatenation improves ill-posedness classification and enables topology-conditioned activation steering...

  34. 4DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Tracking

    cs.CV 2026-06 conditional novelty 7.0

    The paper defines the 4DVLT task for worldline-centered 4D scene understanding, releases Instruct-4D with 129.4K QA pairs, and presents 4DTrack achieving 62.68 TGA_Top1, outperforming adapted baselines by 19.62 points.

  35. GitReq: A Gold Standard Dataset for Software Quality Requirements

    cs.SE 2026-06 unverdicted novelty 7.0

    GitReq is a released dataset of 6,302 expert-labeled GitHub requirements across eight ISO 25010 categories with LLM classification baselines.

  36. User as Engram: Internalizing Per-User Memory as Local Parametric Edits

    cs.AI 2026-06 unverdicted novelty 7.0

    User facts are internalized as surgical local edits to a hash-keyed Engram memory table with reasoning skill held in a shared adapter, claimed to match LoRA recall, improve indirect reasoning 5.6x on average, and comp...

  37. Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation

    cs.CL 2026-06 unverdicted novelty 7.0

    DICE aggregates independently encoded document chunks into a single vector to reduce evidence dilution in long-document dense retrieval, reporting gains on LongEmbed especially beyond 4k tokens.

  38. Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose

    cs.CL 2026-06 unverdicted novelty 7.0

    SkillWeaver formalizes compositional skill routing for LLM agents and introduces SAD, which raises step-level decomposition accuracy from 51% to 67.7% on a new 300-query benchmark over 2209 real MCP skills.

  39. Models Take Notes at Prefill: KV Cache Can Be Editable and Composable

    cs.LG 2026-06 unverdicted novelty 7.0

    KV caches function as notebooks of prefilled conclusions, enabling field-level edits that recover decisions (especially with CoT) and position-portable skill composition with near-identical outputs at O(L) cost.

  40. Polar: A Benchmark for Evaluating Political Bias in LLMs

    cs.CL 2026-06 unverdicted novelty 7.0

    Polar is a new cross-context benchmark showing LLM political bias measurements are not fixed but vary with country, issue, model, and language.

  41. Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization

    cs.CL 2026-06 unverdicted novelty 7.0

    Fine-tuned Mistral-7B via QLoRA achieves up to 12% higher F1 than GPT-4o on biomedical claim verification with 1008 examples, identifies a structural shortcut in SciFact, and shows robust cross-domain transfer from so...

  42. Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data

    cs.LG 2026-06 unverdicted novelty 7.0

    ICL in LLMs shows a sharp ceiling on categorical distributions for high-cardinality tabular data, failing to reproduce rare classes despite examples, while numerical fidelity improves.

  43. INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

    cs.AI 2026-06 unverdicted novelty 7.0

    INFRAMIND is an infrastructure-aware multi-agent orchestration framework that uses RL on a hierarchical constrained MDP to jointly optimize topology, model selection, and scheduling under dynamic load.

  44. STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control

    cs.LG 2026-06 unverdicted novelty 7.0

    STAR-KV applies differentiable soft thresholding for per-head and per-block adaptive low-rank KV cache compression, combined with hybrid decomposition and low-rank-aware quantization, achieving up to 75% compression a...

  45. SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models

    cs.CL 2026-06 unverdicted novelty 7.0

    SurgiQ is a new 13k-question surgical benchmark showing general-purpose LLMs reach 68.1% accuracy while most biomedical models lag and smaller models stay near random baseline.

  46. DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordination

    cs.LG 2026-06 unverdicted novelty 7.0

    DICE formalizes multi-agent LLM coordination as discounted incomplete-information Markov games and introduces Heterogeneous Quantal Response Equilibrium (HQRE) to achieve unique stable equilibria with bounded regret, ...

  47. Tangram: Unlocking Non-Uniform KV Cache for Efficient Multi-turn LLM Serving

    cs.LG 2026-06 unverdicted novelty 7.0

    Tangram makes non-uniform KV cache compression practical for LLM serving with deterministic budget allocation, head group paging, and ahead-of-time load balancing, achieving up to 2.6x throughput gains.

  48. QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

    cs.AI 2026-06 unverdicted novelty 7.0

    QCFuse achieves full-prefill quality in RAG with 1.7x average prefill speedup over full prefill and 1.5x over ProphetKV via compressed query-aware cache fusion.

  49. Text-to-Image Models Need Less from Text Encoders Than You Think

    cs.CV 2026-06 unverdicted novelty 7.0

    A bag-of-position-tagged-words embedding guides text-to-image diffusion models as effectively as full contextual text embeddings from standard encoders.

  50. ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models

    cs.AI 2026-06 unverdicted novelty 7.0

    ClinicalMC is a benchmark of 1,275 Chinese and 5,804 English multi-course clinical samples across four stages, evaluated via a multi-agent framework on closed-source, open-source, and medical LLMs in static and dynami...

  51. EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction

    cs.CL 2026-06 unverdicted novelty 7.0

    Introduces EURO-5K dataset from 136 EU acts and benchmarks full fine-tuning vs QLoRA for BERT and LLM models on reporting obligation extraction, reporting 0.89 F1 with limited gains from legal pretraining except under...

  52. Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models

    cs.LG 2026-06 unverdicted novelty 7.0

    Defines representational capacity as the upper bound on distinguishable near-orthogonal directions in transformer latent spaces, derived from embedding similarity distributions and an adjusted Johnson-Lindenstrauss fo...

  53. Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inference

    cs.DC 2026-06 unverdicted novelty 7.0

    A new fault-injection framework enables a systematic empirical study that produces 17 takeaways on error propagation in LLM inference and four software-only mitigation directions.

  54. Locality Does Not Imply Reachability: Boundary Repair in Block-Sparse Causal Attention

    cs.LG 2026-06 conditional novelty 7.0

    Fixed block causal masks create reachability boundaries where representations depend only on block prefixes, formalized via dependency sets and phase-conditioned coverage functions, with a parameter-free boundary brid...

  55. CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs

    cs.CL 2026-06 unverdicted novelty 7.0

    CultureForest benchmark shows top LLMs degrade sharply on open-ended cultural reasoning tasks, exhibit regional disparities, and are limited more by effective use of knowledge than by lack of knowledge itself.

  56. Can we trust LLM Self-Explanations for Entity Resolution?

    cs.DB 2026-05 unverdicted novelty 7.0

    LLM self-explanations for entity resolution are unstable and weakly faithful to causal evidence, but a hybrid framework using them as priors matches post-hoc quality at up to 10x lower cost.

  57. MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

    cs.CL 2026-05 unverdicted novelty 7.0

    MENTIS applies layerwise covariance torsion (T1), spectral torsion (T2), and ERA localization to paired IT/PA 7-8B models, finding selective larger shifts for normative concepts, negative correlation with entropy, and...

  58. Next-Billion AI Index: The compass for AI utility and adoption in the global majority

    cs.CY 2026-05 unverdicted novelty 7.0

    Introduces nexbax, a diagnostic framework with three themes and 10 dimensions for evaluating AI economic viability, operational practicality, and societal integrity in next-billion-user contexts.

  59. Vector Linking via Cross-Model Local Isometric Consistency

    cs.AI 2026-05 unverdicted novelty 7.0

    A reference-based geometric hashing method recovers cross-model vector correspondences by exploiting local isometric consistency in contrastive embeddings and iteratively bootstrapping from a seed of paired anchors.

  60. What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness

    cs.CV 2026-05 unverdicted novelty 7.0

    The study links three LVLM architectural dimensions to three hallucination types via a new benchmark, finding that language foundation quality reduces co-occurrence errors, visual encoder strength reduces similarity e...

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · cited by 704 Pith papers · 21 internal anchors

  1. [1]

    GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint arXiv:2305.13245, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  2. [2]

    Program Synthesis with Large Language Models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  3. [3]

    Longformer: The Long-Document Transformer

    Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2004

  4. [4]

    Piqa: Reasoning about phys- ical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about phys- ical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, 2020

    work page 2020

  5. [5]

    Evaluating Large Language Models Trained on Code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  6. [6]

    Generating Long Sequences with Sparse Transformers

    Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  7. [7]

    QuAC : Question Answering in Context

    Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. Quac: Question answering in context. arXiv preprint arXiv:1808.07036, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  8. [8]

    BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1905

  9. [9]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  10. [10]

    Training Verifiers to Solve Math Word Problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  11. [11]

    Fu, Stefano Ermon, Atri Rudra, and Christopher Ré

    Tri Dao, Daniel Y . Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems, 2022

    work page 2022

  12. [12]

    Measuring Massive Multitask Language Understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  13. [13]

    Measuring Mathematical Problem Solving With the MATH Dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  14. [14]

    An empirical analysis of compute-optimal large language model training

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simonyan, Erich Elsen, Oriol Vinyals, Jack Rae, and Laurent S...

    work page 2022

  15. [15]

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  16. [16]

    Natural questions: a benchmark for question answering research

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466, 2019. 8

    work page 2019

  17. [17]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    work page 2023

  18. [18]

    xformers: A modular and hackable transformer modelling library

    Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, and Daniel Haziza. xformers: A modular and hackable transformer modelling library. https://github.com/ facebookresearch/xformers, 2022

    work page 2022

  19. [19]

    Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  20. [20]

    Code Llama: Open Foundation Models for Code

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021

    work page 2021

  22. [22]

    SocialIQA: Commonsense Reasoning about Social Interactions

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Com- monsense reasoning about social interactions. arXiv preprint arXiv:1904.09728, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  23. [23]

    Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  24. [24]

    CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A ques- tion answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  25. [25]

    LLaMA: Open and Efficient Foundation Language Models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  26. [26]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  27. [27]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    work page 2017

  28. [28]

    HellaSwag: Can a Machine Really Finish Your Sentence?

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1905

  29. [29]

    AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

    Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models. arXiv preprint arXiv:2304.06364, 2023. 9

    work page internal anchor Pith review Pith/arXiv arXiv 2023