Pith. sign in

REVIEW 2 major objections 5 minor 42 cited by

Gemma 4 Technical Report

T0 review · 2 major / 5 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Gemma 4 open multimodal models leap on STEM, vision, audio and long-context tasks while the 31B dense version ranks as the leading dense open model on human Arena evaluations.

desk verdict Solid open multimodal release with real efficiency engineering and competitive Arena numbers; attribution of the leap to specific design choices is under-supported by missing ablations. read the letter →

arxiv 2607.02770 v1 pith:R2OUDRBF submitted 2026-07-02 cs.CL cs.AI

Gemma Team: Sherif El Abd , Vaibhav Aggarwal , Robin Algayres , Alek Andreev , Olivier Bachem , Ian Ballantyne , Cormac Brick , Victor Cărbune
show 292 more authors
Michelle Casbon Mayank Chaturvedi Victor Cotruta Alice Coucke Phil Culliton Robert Dadashi Lucas Dixon Mohamed Elhawaty Utku Evci Clément Farabet Johan Ferret Filippo Galgani Sertan Girgin Jean-Bastien Grill Maarten Grootendorst Jiaxian Guo Cassidy Hardin Yanzhang He Steven M. Hernandez Omri Homburger Léonard Hussenot Juyeong Ji Armand Joulin Aishwarya Kamath Parnian Kassraie Olivier Lacombe Preethi Lahoti Gaël Liu Gus Martins Luciano Martins Tatiana Matejovicova Ramona Merhej Nikola Momchev Sneha Mondal Ryan Mullins Sindhu Raghuram Panyam Shreya Pathak Sarah Perrin André Susano Pinto Etienne Pot Angéline Pouget Alexandre Ramé Sabela Ramos Douglas Reid David Rim Morgane Rivière Karsten Roth Louis Rouillard Omar Sanseviero Pier Giuseppe Sessa Shane Settle Danila Sinopalnikov Sara Smoot Piotr Stanczyk Andreas Steiner Lawrence Stewart Ilya Tolstikhin Michael Tschannen Anton Tsitsulin Nino Vieillard Renjie Wu Pingmei Xu Haichuan Yang Edouard Yvinec Li Zhang Joe Zou Nicolas Aagnes Abdelrahman Abdelhamed Shivani Agrawal Shubham Agrawal Ibrahim Alabdulmohsin Jean Baptiste Alayrac Uri Alon Chandramouli Amarnath Ankesh Anand Chrysovalantis Anastasiou Setareh Ariafar François-Xavier Aubet Kyriakos Axiotis Federico Barbero Joelle Barral Alexei Bendebury Urs Bergmann Stanley Bileschi Kat Black Mathieu Blondel Sebastian Borgeaud Arthur Bražinskas Ryan Burnell Robert Busa-Fekete Mu Cai Glenn Cameron Charlotte Caucheteux Garima Chadha Jetha Chan Aditya Chawla Blake Jianhang Chen Jesse Chen Lin Chen Xu Chen Derek Cheng Tzu-hsiang Chien Nikolai Chinaev Yi Chou Zhaohui Chu Benjamin Coleman Pooja Consul Sam Conway-Rahman Scott Crowell Dylan Cutler Vivek Dani Samira Daruki Anil Das Daniel Deutsch Nishanth Dikkala Li Ding Qiuhan Ding Shenil Dodhia Konstantin Donhauser Tulsee Doshi Anca Dragan Alex Druinsky Sahil Dua Zoltan Egyed Danielle Eisenbud Daniel Eppens Cindy Fan Bahare Fatemi Yassir Fathullah Vlad Feinberg Milen Ferev Takumi Fujimoto Isaac Galatzer-Levy João Gante Simon Geisler Soham Ghosal Antonious M. Girgis Alec Go Alhaad Gokhale Alex Grills Yiming Gu Pramod Gupta Guru Guruganesh Raia Hadsell Hamza Harkous Jitendra Harlalka Demis Hassabis Anja Hauth Joe Heyward Arian Hosseini Chih-Yang Hsia I-Hung Hsu Xiaopeng Huang Yangsibo Huang Kevin Hui Adrian Hutter Te I Fotis Iliopoulos Advait Jain Ganesh Jawahar Ziwei Ji Qilin Jin Melvin Johnson Kandarp Joshi Arun Kandoor Wang-Cheng Kang Koray Kavukcuoglu Mehran Kazemi Kathleen Kenealy Amr Khalifa Phoebe Kirk Suraj Kothawade Vitaly Kovalev Neel Kovelamudi Adam Kraft Ravin Kumar Harish Kuppam Justin Lannin Chen-Yu Lee Seungji Lee Dmitry Lepikhin Dongdong Li Qiujia Li Valentin Liévin Ethan Lin Ziqian Lin Casper Liu Tianlin Liu Tianqi Liu Xin Liu Mayank Lunayach Min Ma Gagan Madan Andrii Maksai Eric Malmi Michal Matuszak Daniel McDuff Gaurav Menghani Daniil Mirylenka Karolis Misiunas Vedant Misra Andreea Mitran Kareem Mohamed Maksim Mukha Eric Noland James O'Donnell Kate Olszewska Bernett Orlando Wanqiong Pan Rina Panigrahy Unnati Parekh Chunjong Park Eric Paskie Liqian Peng Bryce Petrini Slav Petrov Jonas Pfeiffer Bilal Piot Martyna Plomecka Siim Poder Octavio Ponce Arijit Pramanik David Racz Anish Rajan Michelle Ramanovich Anand Rao Marvin Ritter Vitor Rodrigues Evan Rosen Miko{l}aj Rybiński Noveen Sachdeva Michaël E. Sander Rohit Sathyanarayana Sagar Savla Samuel Schmidgall Tal Schuster Benoit Seguin Andrew Sellergren Aliaksei Severyn Izhak Shafran Dhruv Shah Yuan Shangguan Ashish Shenoy Pradeep Shenoy Rakesh Shivanna Pauline Sho Lucas Spangher Wojciech Stokowiec Tim Strother Yao Su Yinghao Sun Mukund Sundararajan Andrea Tacchetti Mor Hazan Taege Pouya Tafti Chetan Tekur Rahul Thapa Madeleine Traverse Lenart Treven Tao Tu Chien Te Tung Petar Veličković Malini Pooni Venkat Sagar Gubbi Venkatesh Vidya Venkiteswaran Francesco Visin Alex Vitvitskyi Kiran Vodrahalli Weiyi Wang Xin Wang Tris Warkentin Jan Wassenberg John Wieting Lechao Xiao Hao Xu Yuhui Xu Fuzhao Xue Arun Yadav Jun Yan Antoine Yang Lin Yang Ming-Hsuan Yang Ziyu Ying Jae Hyeon Yoo Sajjad Zafar Fred Zhang Jiageng Zhang Jianyi Zhang Xiaofan Zhang Chao Zhao David Zhou Chen Zou
This is my paper · ORCID
classification cs.CLcs.AI
keywords multimodallanguagemodelsopen-weightmixture-of-expertsthinkingmodelong-contextefficiencyquantization-awaretrainingencoder-freearchitecturespeculativedecoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Gemma 4 is a family of open-weight, natively multimodal language models ranging from roughly 2B to 31B parameters, offered in both dense and mixture-of-experts forms. The paper establishes that these models, released under Apache 2.0, deliver a clear performance jump over the prior Gemma generation across STEM reasoning, vision, audio, and long-context benchmarks, and that the largest members rival much larger frontier open models in blind human preference tests. The suite adds a thinking mode that first emits reasoning traces, an encoder-free 12B architecture that ingests raw image patches and audio chunks, and a set of attention, caching, drafting and quantization choices that shrink memory and raise decoding speed. A sympathetic reader cares because the result puts competitive multimodal reasoning and long-context ability into freely available, on-device-friendly packages.

What carries the argument

Thinking mode (the model emits an explicit reasoning trace before the final answer) together with the local/global attention ratio, p-RoPE positional encoding, and KV-cache sharing that together cut the global KV footprint by up to 37.5 percent; these are the mechanisms the paper credits for the reasoning and efficiency gains.

What would settle it

An independent re-run of the same Arena blind comparisons and of the public STEM/multimodal/long-context suites under identical protocols and without the new design choices (or with thinking mode ablated) that fails to reproduce the claimed ranking and numerical leaps.

Watch

Extended reading notes

Core claim

The central claim is that the combination of thinking-mode generation, 5-to-1 local-to-global attention with p-RoPE and key-value reuse, multi-token-prediction drafting, quantization-aware training, and a unified encoder-free backbone for the 12B model produces open multimodal models that substantially outperform Gemma 3 counterparts of similar or larger size and place the 31B dense model at the top of the dense open category on Arena while smaller variants match earlier 27B-class results with far fewer parameters.

Load-bearing premise

The reported gains are produced by the listed architectural and training choices rather than by undisclosed differences in data mixture, filtering, or evaluation protocol.

Editorial extensions

If this is right

  • Open 2–4 B class models become practical substitutes for earlier 27 B models on many STEM and multimodal tasks.
  • Dense open models at 31 B can occupy the same human-preference tier as much larger mixture-of-experts systems.
  • Encoder-free ingestion of raw patches and audio chunks reduces memory fragmentation and simplifies on-device multimodal stacks.
  • Thinking traces become a standard, controllable feature of open instruction-tuned models rather than a closed-model exclusive.
  • Long-context workloads at 128 k tokens become feasible under tighter KV-cache budgets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The encoder-free 12 B design may lower the barrier for community fine-tuning of multimodal adapters because there is no frozen external encoder to keep in sync.
  • If thinking mode transfers cleanly to the smallest models, on-device agents could gain chain-of-thought reliability without cloud round-trips.
  • The same local/global and p-RoPE recipe could be ported to other open dense families to obtain similar cache reductions with modest re-training.
  • Public ablations that isolate thinking mode from data scale would clarify how much of the STEM leap is algorithmic versus corpus-driven.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents Gemma 4, a family of open-weight natively multimodal decoder-only Transformers (dense E2B/E4B/12B/31B and MoE 26B-A4B) that process text, images and audio. Key design choices include a thinking mode that emits reasoning traces, 4:1/5:1 local-to-global attention with p-RoPE (p=0.25), key-as-value reuse and KV-cache sharing for long-context memory reduction (up to 37.5 %), an encoder-free 12B architecture that linearly projects raw 48 imes48 image patches and 40 ms audio chunks, an autoregressive multi-token-prediction drafter for speculative decoding, and quantization-aware training. High-level pre-training (Jan 2025 cutoff corpus, decontamination) and post-training recipes are given. Extensive automatic and human evaluations (Arena Elo with 95 % CIs, MMLU-Pro, AIME, LiveCodeBench, GPQA, vision suites, CoVoST/FLEURS audio, RULER/LOFT/MTOB long-context) claim large gains over Gemma 3 and parity with far larger open models, with the 31B dense model leading the dense open category on Arena.

Significance. If the reported numbers hold under independent scrutiny, the work supplies a valuable open multimodal baseline family that advances practical efficiency and reasoning at edge-relevant sizes. Concrete engineering contributions include the quantified KV-cache and QAT memory reductions (Table 3), the encoder-free 12B projection design, the MTP drafter, and the thinking-mode integration. Arena leadership of the 31B dense model (Elo 1451) and the observation that E2B roughly matches Gemma-3 27B at ~10 imes fewer parameters are high-utility empirical facts for the community. Release under Apache 2.0 together with quantized checkpoints further increases impact. The absence of full causal ablations is a limitation common to industrial technical reports and does not erase the utility of the released artifacts.

major comments (2)
  1. [Table 5, §4.2] The central claim of a performance leap over Gemma 3 (abstract, §4.2) rests on Tables 5–9, yet evaluation protocols are not matched: Gemma-4 models run in thinking mode while Gemma-3 27B is non-thinking (explicit header note in Table 5); vision results use different maximum token budgets and resizing (Table 6 vs. Pan & Scan). Without non-thinking Gemma-4 numbers, matched-resolution ablations, or data-mixture controls, the contribution of the architectural innovations listed in §2 cannot be isolated from the thinking protocol or later data. This is load-bearing for the attribution language used throughout the abstract and introduction.
  2. [§2.4] §2.4 describes the pre-training corpus only as a “large-scale, diverse collection o cutoff January 2025” with high-level decontamination and safety filtering. Because the leap claim is presented as arising from the design choices of §2, the lack of mixture proportions, decontamination procedure details, or any ablation that holds data fixed while toggling architecture leaves the causal story under-supported. A short caveat or additional controlled experiment would strengthen the manuscript.
minor comments (5)
  1. [Table 1] Table 1 and the surrounding text should more prominently flag that E2B/E4B “effective” parameter counts exclude the large per-layer embeddings; the distinction is easy to miss when comparing against other open models.
  2. [Figure 1] Figure 1 (MTP drafter) would benefit from explicit dimension annotations on the cross-attention and embedder blocks so that the claimed decoding-speed advantage can be verified at a glance.
  3. [Tables 5–9] Most automatic metrics lack error bars or multiple-run statistics (Arena is the exception). Adding even simple standard deviations or bootstrap intervals for the key STEM and long-context numbers would improve interpretability.
  4. [§2.1, Algorithm 1] Algorithm 1 and Figure 2 are clear, yet the precise mapping from N_max values (70 o 1120) to the final soft-token counts after 3 imes3 pooling is left implicit; a short formula or extra column would help re-implementers.
  5. [References] A few concurrent 2026 citations (e.g., Kayyam et al.) appear before their public availability; a note on arXiv versioning would avoid confusion for readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical model-release report whose performance claims rest on external community benchmarks, not on self-defined or fitted quantities.

full rationale

Gemma 4 is an architectural and training technical report. Its central claims (leap on STEM/multimodal/long-context suites; Arena Elo leadership for the 31B dense model; E2B roughly matching Gemma-3 27B at ~10 imes fewer parameters) are supported by tabulated scores on external, community-defined suites (Arena, MMLU-Pro, AIME, LiveCodeBench, GPQA, RULER, CoVoST, FLEURS, MMMU-Pro, etc.). These metrics are not redefined in terms of any parameter fitted inside the paper, nor are they obtained by construction from the listed design choices (thinking mode, local-global attention ratios, p-RoPE, key-as-value reuse, encoder-free projections, MTP drafter, QAT). Self-citations appear only for architectural lineage (prior Gemma reports) and are not load-bearing for the new numbers. There is no uniqueness theorem, no ansatz smuggled via self-citation, no renaming of a known empirical pattern as a derived result, and no equation that reduces a claimed prediction to its own input. Attribution of gains to specific innovations is under-supported by missing ablations (a correctness/attribution concern, not circularity). The derivation chain is therefore self-contained against external benchmarks; circularity score is zero.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

As an empirical systems paper the load-bearing premises are standard Transformer design choices, external benchmark definitions, and the unstated but conventional assumption that the (undisclosed) training mixture plus the listed architectural modifications produce the observed gains. No new physical entities or free parameters fitted to a scientific constant are introduced; free parameters are the usual architectural hyper-parameters and the decision to freeze encoders.

free parameters (4)
  • local-to-global attention ratio = 4:1 / 5:1
    Fixed at 4:1 (E2B) or 5:1 (others); chosen by design rather than derived, directly affects KV-cache size claims.
  • p-RoPE fraction p = 0.25
    Set to 0.25 on global layers; an engineering choice that reduces global KV footprint by the claimed 37.5 %.
  • max vision tokens N_max = up to 1120
    Discrete set {70,140,280,560,1120}; controls resolution and compute, selected rather than derived.
  • MTP drafter depth and width = 4 layers, dim 256/1024
    4-layer Transformer with model dim 256/1024; sized for acceptance-rate vs. overhead trade-off.
assumptions (4)
  • domain assumption Decoder-only Transformer with RMSNorm, QKNorm, and the listed attention patterns is a sufficient backbone for multimodal next-token prediction.
    Standard in modern LLMs; invoked throughout Section 2 without re-derivation.
  • domain assumption Frozen vision/audio encoders (or their lightweight projections) plus continuous embeddings preserve enough information for the LLM to solve the reported multimodal tasks.
    Stated in Sections 2.1–2.3; no proof that freezing is optimal, only empirical results.
  • domain assumption Public benchmarks (Arena Elo, MMLU-Pro, AIME, CoVoST, RULER, etc.) are valid proxies for the capabilities claimed.
    Used as the sole quantitative evidence in Sections 4.1–4.2.
  • ad hoc to paper Decontamination and safety filtering of the January 2025 cutoff corpus remove benchmark leakage and harmful content sufficiently for the reported scores to be meaningful.
    Asserted in Sections 2.4 and 5; no public verification set or leakage statistics provided.
invented entities (2)
  • Unified encoder-free 12B architecture independent evidence
    purpose: Replace separate 550 M vision + 305 M audio encoders with direct patch/chunk projections into the LLM embedding space to reduce memory fragmentation.
    Introduced in Section 2.3; performance tables show it remains competitive, but the entity itself is a new design choice without independent theoretical derivation.
  • Autoregressive multi-token prediction (MTP) drafter head with cross-attention to main-model KVs independent evidence
    purpose: Enable speculative decoding of arbitrary draft length without separate prefill.
    Described in Section 2.6 and Figure 1; acceptance rates are claimed but not tabulated in detail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gemma 4 Technical Report." pith.science (2026). https://pith.science/paper/R2OUDRBF

@misc{pith2026260702770,
  author       = {Pith},
  title        = {Pith review of: Gemma 4 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R2OUDRBF}},
  note         = {Machine review of arXiv:2607.02770}
}
read the original abstract

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 42 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Edit2TikZ: A Comprehensive and Challenging Benchmark for Scientific Figure Editing with TikZ

    cs.CV 2026-08 conditional novelty 7.0 of 10

    Edit2TikZ is a 1,548-sample benchmark for instruction-guided scientific figure editing via TikZ code, with eight atomic operations, multi-step edits, visual localization, and human-aligned metrics.

  2. Do LLMs Beat Nash? Testing Decentralized Coordination in Self-Play Multi-Agent Games

    cs.MA 2026-08 conditional novelty 7.0 of 10

    In one-shot self-play without communication, two frontier LLMs beat the worst symmetric Nash baseline in several two-player game archetypes, while open-weight models vary and team coordination collapses for most model...

  3. Verifiably grounded machine interpretation of lunar geology

    cs.CL 2026-08 accept novelty 7.0 of 10

    SelenoVLM generates raster-verifiable lunar geological descriptions, yet its numeric ages default to the global mare prior unless the published age is injected through retrieval.

  4. Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination

    cs.CL 2026-08 conditional novelty 7.0 of 10

    A new stratified per-question probability-gap metric shows that existing contamination mitigation strategies overstate restoration, and a step-wise decoding cap (RailCap) achieves the lowest measured gap.

  5. Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    Entropy-based CoT pruning does not beat random selection; apparent math-task gains come from numeric tokens, not entropy.

  6. Beyond Geometric Complementarity: Coherent Overlap in Sparse Mixture-of-Experts Routing

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Across six MoE models, expert subspaces overlap substantially, routes stay coherent, the actual prefix narrows the selected expert's geometric advantage in all 39 tested cells, yet later experts still improve next-tok...

  7. Self-supervision drives representational convergence in medical foundation models more than clinical supervision

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Representational convergence among medical image encoders is modest, driven mainly by the self-supervised pretraining objective rather than clinical supervision or scale, yet still sufficient for cross-encoder and cro...

  8. COBS: Cumulant Order Block Sparse Attention

    cs.LG 2026-07 conditional novelty 7.0 of 10

    A compressed within-block key covariance lets block-sparse attention recover most of dense long-context retrieval quality at roughly first-order selector traffic.

  9. Metis: Memory Foundation Model

    cs.CL 2026-07 conditional novelty 6.5 of 10

    Metis puts a trainable fixed-size memory matrix inside a frozen LLM backbone and learns to remember, update, forget, and reflect across turns without replaying original context.

  10. HiEviDR-Bench: A Benchmark for Hierarchical Evidence Aggregation in Deep Research

    cs.IR 2026-07 conditional novelty 6.5 of 10

    A hierarchical evidence-graph benchmark reveals that multimodal deep-research models write fluent reports while failing citation, claim, and answer grounding.

  11. Reasoning Jury: Multi-Model Consensus for Evaluating Reasoning Traces

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Reasoning Jury aggregates independent defect judgments from a jury of LLMs through consolidation or moderated deliberation, beating frontier single-model judges on Hard2Verify and DeltaBench at a fraction of the cost.

  12. Making Your LLMs More Objective: Stabilizing LLM Safety Behavior Across Traits with Trait-Invariant Safety Tuning

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Trait-Subspace Neutralization reduces trait-induced instability in LLM refusal behavior by anchoring trait-conditioned representations to the no-trait baseline within a low-dimensional subspace.

  13. Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences

    cs.CV 2026-08 conditional novelty 6.0 of 10

    LVLM judges are near chance when asked to pick the correctly ordered version of an image sequence, and this temporal blindness persists after fine-tuning and at larger scale.

  14. ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation

    cs.LG 2026-08 conditional novelty 6.0 of 10

    ReOrder-OPD sorts OPD prompts by a proxy for teacher continuation reliability and consistently beats vanilla ordering on matched math and code experiments.

  15. MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory

    cs.AI 2026-08 conditional novelty 6.0 of 10

    MESA learns a query-adaptive subset of five memory structures for long-horizon agents, beating AMA-Agent by 8.5 points on AMA-Bench while using 41% fewer evidence tokens than reading all structures.

  16. MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection

    cs.SD 2026-08 conditional novelty 6.0 of 10

    MADBench introduces a component-aware audio-visual deepfake benchmark with independently manipulated speech and environmental audio, and shows environmental manipulation is easier to detect than synthetic speech.

  17. Evo-Bench: Can Language Models Improve Agent Harness?

    cs.CL 2026-08 conditional novelty 6.0 of 10

    Evo-Bench selects tasks that respond to harness changes and shows frontier models can evolve executable agent harnesses with gains up to 16.6 points, though Office tasks remain difficult.

  18. Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Adversarial text inside Android accessibility data can redirect mobile AI agents to unauthorized actions, with measured attack success up to 0.822 in the most vulnerable setup.

  19. HLSmith: An Expert-Guided Agentic Framework for C/C++-to-HLS Translation

    cs.AR 2026-08 conditional novelty 6.0 of 10

    An expert-guided LLM framework translates C/C++ kernels into HLS FPGA designs, outperforming ChatHLS on PolyBench with a 4.24x geometric-mean speedup and valid RTL designs for all 14 attempted kernels.

  20. Visual Grounding in Zero-Shot Vision-Language Control

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Input-ablation tests show most current VLMs are not visually grounded controllers, though a small symmetry-consensus ensemble works as a hazard monitor.

  21. Skills Know Their Neighbors: Cluster-Contrastive Capability Pages for Skill Retrieval

    cs.IR 2026-08 conditional novelty 6.0 of 10

    Cluster-contrastive Capability Pages that add positive triggers, negative boundaries, and discriminative bodies to skill documents improve first-stage Recall@10 by 2.94 points on average and end-to-end task success by...

  22. Reachability Is Not Realization: Tracing the Sources of LLM Benchmark Gains

    cs.AI 2026-08 conditional novelty 6.0 of 10

    LLM benchmark gains often come from producing already-reachable answers more reliably, not from making new answers reachable under a matched probe.

  23. A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning

    cs.LG 2026-08 conditional novelty 6.0 of 10

    As context grows, LLM internal representations of numerical sequences become more clearly separated by input complexity: simple inputs yield smoother, more globally connected token graphs, while chaotic inputs yield m...

  24. SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Training agents on 27,164 verified synthetic trajectories generated from 2,000 public skills improves their ability to use external skills across four benchmarks and two harnesses.

  25. Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation

    cs.CL 2026-08 conditional novelty 6.0 of 10

    GRPB ranks same-task video edits, converts the ranking into zero-sum advantages, and spreads them over editing segments through a lagged, capped allocator, producing a stronger editing agent.

  26. Fighting Fire with Fire: On the Feasibility of Protecting Exercises Against AI Cheating

    cs.AI 2026-08 conditional novelty 6.0 of 10

    Subtle adversarial image perturbations can steer commercial AI assistants toward designated wrong answers, and the resulting answer patterns can statistically flag students who blindly copy those assistants.

  27. Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks

    cs.SE 2026-08 conditional novelty 6.0 of 10

    Distilled advisory prompts improve non-thinking pass rates by a mean of 3.1 points across 20 small-model software-engineering comparisons and cut output tokens by 58.6% versus full thinking.

  28. TRACE-TS: Attribution-Grounded and Traceable Sensor-Language Reasoning for Human Activity Understanding

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Attribution-guided DAG reasoning traces distilled into a compact sensor-conditioned LM give 84.43% average accuracy and 81.24% F1 across seven HAR datasets, with SNM as an LLM-judge diagnostic for trace fidelity.

  29. Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokens

    eess.AS 2026-07 conditional novelty 6.0 of 10

    Autoregressive TTS from 8-Hz, 768-dimensional continuous tokens works when the tokenizer shapes its latent space with a low-dimensional core and an energy hierarchy, and the generator separates guidance into local, se...

  30. MORFES: A Benchmark for Productive Inflectional Competence in Modern Greek

    cs.CL 2026-07 conditional novelty 6.0 of 10

    MORFES is the first expert-verified Modern Greek productive-inflection benchmark; Sophea-Genesis-1 leads it at 84% per-item production without losing general capability.

  31. AgenticASR: Refining Speech Recognition in Real-World Scenarios via an Agentic Approach

    cs.AI 2026-07 conditional novelty 6.0 of 10

    An ASR–Refiner system emits and revises clean transcripts online over a bounded sliding context, outperforming offline spoken-to-written baselines on a new bilingual rubric benchmark.

  32. Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    For coarse visual attributes like color, count, and size, MLLMs encode counterfactual image evidence in their final layers yet fail to follow instructions about whether to trust it; a learned steering vector improves ...

  33. CodeNib: A Multi-View Data System for Serving Repository Context to Coding Agents

    cs.SE 2026-07 conditional novelty 6.0 of 10

    A per-commit multi-view repository-index system with incremental updates that match independent rebuilds at 8.7x/25.4x median speedups, and context policies that cut agent trajectory tokens 50-87% while holding locali...

  34. Unfit for stranding assessment: a panel-scale multimodal-LLM audit of building-decarbonisation disclosure (BeDA)

    cs.CY 2026-07 conditional novelty 6.0 of 10

    Only ~21.5% of built-environment firm-reports (45.5% of real-estate reports) disclose operational carbon intensity per m²/yr, the metric CRREM-style stranding assessment requires.

  35. Automating structural reliability analysis with a multi-agent large language model framework

    stat.AP 2026-07 conditional novelty 6.0 of 10

    A multi-agent LLM framework automates structural reliability analysis from natural-language input to interpreted β and Pf, with a QLoRA-tuned method-planning agent.

  36. On-Policy Delta Distillation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Using the difference between a reasoning-tuned teacher and its base model as the distillation reward consistently improves on-policy post-training of small reasoning LLMs.

  37. Max Out GRPO Signal: Adaptive Trace Prefix Control for Hard Reasoning Problems

    cs.LG 2026-07 conditional novelty 6.0 of 10

    AdaPrefix-GRPO treats solution-prefix length as a feedback controller targeting 50% rollout success rate during GRPO training, then anneals to zero prefix, yielding 1.6–2.1× accuracy gains over vanilla GRPO at matched...

  38. Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation

    cs.CL 2026-08 conditional novelty 5.0 of 10

    Reference-free RL with language-gated QE rewards plus checkpoint interpolation improves XCOMET and COMETKiwi scores across 46 languages, but the headline metrics are exactly the functions optimized.

  39. Mind the Gaps: Mixture-of-Minds for Human Simulation

    cs.AI 2026-08 reject novelty 5.0 of 10

    Anacreon claims state-of-the-art individual-level survey prediction by clustering public text and training a separate adapter per cluster, but its evaluation is on self-mined questions and lacks an external benchmark.

  40. DiffusionGemma Technical Report

    cs.CL 2026-07 conditional novelty 5.0 of 10

    DiffusionGemma turns a 25B-parameter open-weight autoregressive MoE into a discrete diffusion model that produces ~20 tokens per forward pass and ~1,500 tokens/sec on one H100, establishing a speed-quality tradeoff po...

  41. Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model

    cs.AI 2026-07 conditional novelty 5.0 of 10

    A 3B-parameter model using layer-reused Transformers and multi-stage RL reports higher scores than 9B and 12B rivals on agentic code, office, and tool-use benchmarks.

  42. StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A co-designed 0.9B UI vision-language model tops small-model benchmarks on screen Q&A and Chinese OCR and runs on a Snapdragon 8 Gen5 at 98 tokens/second.

Reference graph

Works this paper leans on

23 extracted references · 20 linked inside Pith · cited by 42 Pith papers

  1. [1]

    J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer nor- malization.arXiv preprint arXiv:1607.06450,

    arXiv

  2. [2]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabili- ties.arXiv preprint arXiv:2507.06261,

    Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabili- ties.arXiv preprint arXiv:2507.06261,

    arXiv

  3. [3]

    Gemma: Open models based on gemini research and technology, 2024a

    Gemma Team. Gemma: Open models based on gemini research and technology, 2024a. Gemma Team. Gemma 2: Improving open lan- guage models at a practical size.arXiv preprint arXiv:2408.00118, 2024b. Gemma Team. Gemma 3: Technical report.arXiv preprint arXiv:2503.19786, 2025a. Gemma Team. Gemma 3n. https://deepmi nd.google/models/gemma/gemma-3n/ , 2025b. Google ...

    arXiv

  4. [4]

    Gulati, J

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, et al. Conformer: Convolution-augmented transformer for speech recognition.arXiv preprint arXiv:2005.08100,

    arXiv 2005

  5. [5]

    Henry, P

    A. Henry, P. R. Dachapally, S. S. Pawar, and Y. Chen. Query-key normalization for trans- formers. InFindings of the Association for Computational Linguistics: EMNLP 2020, pages 4246–4253,

    2020

  6. [6]

    Hsieh, S

    C.-P. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y. Zhang, and B. Ginsburg. Ruler: What’s the real context size of your 9 Gemma 4 Technical Report long-context language models?arXiv preprint arXiv:2404.06654,

    arXiv

  7. [7]

    N. Jain, A. Gu, W.-D. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. Live- codebench: Holistic and contamination free evaluation of large language models for code. InInternational Conference on Learning Repre- sentations, volume 2025, pages 58791–58831,

    2025

  8. [8]

    Kayyam, A

    A. Kayyam, A. M. Gopal, and M. A. Lewis. Do transformers need three projections? system- atic study of qkv variants.arXiv preprint arXiv:2606.04032,

    arXiv

Show all 23 references
  1. [9]

    Kazemi, B

    M. Kazemi, B. Fatemi, H. Bansal, J. Palowitch, C.Anastasiou, S.V.Mehta, L.K.Jain, V.Aglietti, D. Jindal, P. Chen, et al. Big-bench extra hard. arXiv preprint arXiv:2502.19187,

    arXiv

  2. [10]

    M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, J. Y. Shin, T. Wal- she, E. K. Buchanan, et al. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces.arXiv preprint arXiv:2601.11868,

    arXiv

  3. [11]

    Openai o1 system card.arXiv preprint arXiv:2412.16720,

    OpenAI. Openai o1 system card.arXiv preprint arXiv:2412.16720,

    arXiv

  4. [12]

    L. Phan, A. Gatti, Z. Han, N. Li, J. Hu, H. Zhang, C. B. C. Zhang, M. Shaaban, J. Ling, S. Shi, et al. Humanity’s last exam.arXiv preprint arXiv:2501.14249,

    arXiv

  5. [13]

    D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bow- man. Gpqa: A graduate-level google-proof q&a benchmark.ArXiv, abs/2311.12022,

    arXiv

  6. [14]

    N. Shazeer. Fast transformer decoding: One write- head is all you need.CoRR, abs/1911.02150,

    1911 arXiv

  7. [15]

    K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chen, et al. Kimi k2. 5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276,

    arXiv

  8. [16]

    Q. Team. Qwen3. 5-omni technical report.arXiv preprint arXiv:2604.15804,

    arXiv

  9. [17]

    Vodrahalli, S

    K. Vodrahalli, S. Ontanon, N. Tripuraneni, K. Xu, S. Jain, R. Shivanna, J. Hui, N. Dikkala, M. Kazemi, B. Fatemi, et al. Michelangelo: Long context evaluations beyond haystacks via latent structure queries.arXiv preprint arXiv:2409.12640,

    arXiv

  10. [18]

    B. Xiao, B. Xia, B. Yang, B. Gao, B. Shen, C. Zhang, C. He, C. Lou, F. Luo, G. Wang, et al. Mimo-v2-flash technical report.arXiv preprint arXiv:2601.02780,

    arXiv

  11. [19]

    A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, et al. Deepseek-v4: Towards highly efficient million- token context intelligence.arXiv preprint arXiv:2606.19348,

  12. [20]

    A. Zeng, X. Lv, Z. Hou, Z. Du, Q. Zheng, B. Chen, D. Yin, C. Ge, C. Huang, C. Xie, et al. Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,

    arXiv

  13. [21]

    Zhang, W

    Y. Zhang, W. Han, J. Qin, Y. Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V. Axelrod, G. Wang, et al. Google usm: Scaling automatic speech recognition beyond 100 languages.arXiv preprint arXiv:2303.01037,

    arXiv

  14. [22]

    J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou. Instruction- following evaluation for large language models. arXiv preprint arXiv:2311.07911,

    arXiv

  15. [23]

    11 Gemma 4 Technical Report Y. Zuo, S. Qu, Y. Li, Z. Chen, X. Zhu, E. Hua, K. Zhang, N. Ding, and B. Zhou. Medx- pertqa: Benchmarking expert-level medical reasoning and understanding.arXiv preprint arXiv:2501.18362,

    arXiv

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.