pith. sign in

arxiv: 1907.02127 · v1 · pith:IOIMHEXOnew · submitted 2019-07-03 · 💻 cs.SE

NullAway: Practical Type-Based Null Safety for Java

Subarno Banerjee , Lazaro Clapp , Manu Sridharan This is my paper

Pith reviewed 2026-05-25 09:37 UTC · model grok-4.3

classification 💻 cs.SE
keywords null safetyJavatype checkingNullPointerExceptionstatic analysisbuild overheadAndroid apps
0
0 comments X

The pith

NullAway is a Java null safety checker with 1.15X build overhead that avoids false negatives from its assumptions on checked code.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

NullAway prevents NullPointerExceptions at compile time in Java using code annotations and pluggable type checking. It overcomes deployment barriers in prior systems by running with low enough overhead to integrate into every build and by applying targeted unsound assumptions that cut annotation effort while aiming to produce no false negatives on checked code. Measurements show 1.15X build-time overhead versus 2.8-5.1X for comparable tools. Production crash data from Android apps built with NullAway found that all remaining NPEs traced to third-party libraries, deliberate suppressions, or reflection rather than the checker's assumptions.

Core claim

NullAway achieves practical null safety for large-scale Java projects through low-overhead engineering and targeted unsound assumptions that, in practice on checked code, introduce no false negatives, as confirmed by analysis of production crash corpora.

What carries the argument

Targeted unsound assumptions that reduce annotation burden while preserving soundness on checked code.

If this is right

  • NullAway integrates into continuous builds without prohibitive slowdown.
  • Remaining NPEs in deployed apps arise from external libraries or intentional suppressions.
  • Annotation effort can focus on internal code while the checker protects against internal null errors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same pattern of controlled unsoundness for practicality could extend to other Java static analyses.
  • Teams could prioritize suppressing known-safe third-party call sites to further reduce crash sources.
  • Collecting more crash corpora could refine which assumptions remain safe over time.

Load-bearing premise

The targeted unsound assumptions produce no false negatives in practice on checked code.

What would settle it

Discovery of a NullPointerException in NullAway-checked code whose root cause is one of the unsound assumptions rather than an unchecked library, suppression, or reflection.

Figures

Figures reproduced from arXiv: 1907.02127 by Lazaro Clapp, Manu Sridharan, Subarno Banerjee.

Figure 1
Figure 1. Figure 1: An example (with errors) to illustrate initialization [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An example to illustrate NullAway’s purity han￾dling. (see §3.1). In contrast, Eradicate considers initialization performed in all (private or final) constructor callees, even those invoked conditionally, which is less sound. E.g., if line 25 were written as if (cond()) helper();, Eradicate would still treat fields assigned in helper as initialized. Second, Eradicate does not have any checking for use befo… view at source ↗
Figure 3
Figure 3. Figure 3: Flowchart for NullAway’s treatment of unanno￾tated code. did not investigate this approach due to potential runtime overhead and the riskiness of shipping modified code. NullAway has a highly-configurable system for specifying which code is unannotated and how optimistically it is handled. At the highest level, annotated and unannotated code is partitioned based on its Java package, not whether the code is… view at source ↗
Figure 4
Figure 4. Figure 4: Build-time overheads of NullAway, CFNullness, and Eradicate. Compile times are normalized to the ‘Base’ compile times without nullness checking. Absolute times for Base compiles are labeled above the bars (in seconds). third-party libraries, and 10% involved Uber libraries outside the Android monorepo that do not build with NullAway. The next broad category (17%) were manual suppressions of NullAway warnin… view at source ↗
read the original abstract

NullPointerExceptions (NPEs) are a key source of crashes in modern Java programs. Previous work has shown how such errors can be prevented at compile time via code annotations and pluggable type checking. However, such systems have been difficult to deploy on large-scale software projects, due to significant build-time overhead and / or a high annotation burden. This paper presents NullAway, a new type-based null safety checker for Java that overcomes these issues. NullAway has been carefully engineered for low overhead, so it can run as part of every build. Further, NullAway reduces annotation burden through targeted unsound assumptions, aiming for no false negatives in practice on checked code. Our evaluation shows that NullAway has significantly lower build-time overhead (1.15X) than comparable tools (2.8-5.1X). Further, on a corpus of production crash data for widely-used Android apps built with NullAway, remaining NPEs were due to unchecked third-party libraries (64%), deliberate error suppressions (17%), or reflection and other forms of post-checking code modification (17%), never due to NullAway's unsound assumptions for checked code.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper presents NullAway, a pluggable type-based null safety checker for Java. It claims to achieve low build-time overhead (1.15X) suitable for running on every build, in contrast to prior tools (2.8-5.1X), while using targeted unsound assumptions to reduce annotation burden. Evaluation on production crash data from widely-used Android apps shows that remaining NPEs arise only from unchecked third-party libraries (64%), deliberate suppressions (17%), or reflection/post-checking modifications (17%), with zero attributed to the unsound assumptions on checked code.

Significance. If the empirical results hold, the work is significant for enabling practical deployment of null safety in large Java codebases. The concrete overhead measurements and crash-source breakdown directly support the practicality claim and bound the impact of acknowledged unsoundness, providing evidence that the system can reduce NPE crashes without introducing new ones in checked code.

minor comments (2)
  1. §4 (Evaluation): clarify whether the 1.15X overhead figure is a geometric mean across all benchmarks or includes variance; a table with per-project numbers would strengthen the comparison to the 2.8-5.1X range for other tools.
  2. The description of the unsound assumptions (e.g., handling of generics or initialization) would benefit from a short enumerated list in §3 to make the targeted nature of the unsoundness easier to compare against the crash breakdown in §5.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive review and recommendation to accept the paper. We appreciate the recognition of the significance of the overhead measurements and the crash-source breakdown in bounding the impact of NullAway's unsound assumptions.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper presents an engineering artifact (NullAway) whose key claims are empirical measurements of build overhead (1.15X) and a breakdown of production NPE sources from external crash data (64% third-party, 17% suppressions, 17% reflection). These results are obtained by direct instrumentation and corpus analysis rather than any derivation, fitting, or self-citation chain. No equations, ansatzes, uniqueness theorems, or renamings appear in the provided text; the unsound assumptions are stated explicitly and then tested against independent data, leaving the central claims self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim depends on the practical safety of the unsound assumptions, which is treated as a domain assumption validated by crash data rather than derived from first principles.

axioms (1)
  • domain assumption Targeted unsound assumptions produce no false negatives in practice on checked code
    Invoked to support the claim that remaining NPEs are never due to NullAway's assumptions.

pith-pipeline@v0.9.0 · 5736 in / 1192 out tokens · 45059 ms · 2026-05-25T09:37:04.880972+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Checker Framework Manual

    2019. Checker Framework Manual. https://checkerframework.org/manual/. Accessed: 2019-01-29

    work page 2019

  2. [2]

    Error Prone

    2019. Error Prone. http://errorprone.info/ Accessed: 2019-02-07

    work page 2019

  3. [3]

    Error Prone NullableDereference check

    2019. Error Prone NullableDereference check. https://git.io/fhQkO. Accessed: 2019-01-29

    work page 2019

  4. [4]

    Error Prone ReturnMissingNullable check

    2019. Error Prone ReturnMissingNullable check. https://git.io/fhQk3. Accessed: 2019-01-29

    work page 2019

  5. [5]

    Google Core Libraries for Java (Guava)

    2019. Google Core Libraries for Java (Guava). https://github.com/google/guava Accessed: 2019-02-10

    work page 2019

  6. [6]

    Infer : Eradicate

    2019. Infer : Eradicate. https://fbinfer.com/docs/eradicate.html Accessed: 2019-01-29

    work page 2019

  7. [7]

    IntelliJ IDEA @Contract

    2019. IntelliJ IDEA @Contract. https://www.jetbrains.com/help/idea/contract- annotations.html Accessed: 2019-02-07

    work page 2019

  8. [8]

    Kotlin Programming Language

    2019. Kotlin Programming Language. https://kotlinlang.org/. Accessed: 2019-01- 29

    work page 2019

  9. [9]

    NullAway

    2019. NullAway. https://github.com/uber/NullAway Accessed: 2019-07-01

    work page 2019

  10. [10]

    Performance Benchmarking of Java Null Safety Tools

    2019. Performance Benchmarking of Java Null Safety Tools. https://github.com/ subarnob/nullaway-eval. Accessed: 2019-07-01

    work page 2019

  11. [11]

    ReactiveX/RxJava

    2019. ReactiveX/RxJava. https://github.com/ReactiveX/RxJava Accessed: 2019- 02-10

    work page 2019

  12. [12]

    Supplementary Data

    2019. Supplementary Data. https://figshare.com/s/a212932795a43c377a3f. Ac- cessed: 2019-02-20

    work page 2019

  13. [13]

    Swift Programming Language

    2019. Swift Programming Language. https://swift.org/. Accessed: 2019-01-29

    work page 2019

  14. [14]

    The Checker Framework

    2019. The Checker Framework. https://github.com/typetools/checker- framework Accessed: 2019-01-29

    work page 2019

  15. [15]

    The Java Language Specification

    2019. The Java Language Specification. https://docs.oracle.com/javase/specs/jls/ se9/html/. Accessed: 2019-01-29

    work page 2019

  16. [16]

    Understand the Activity Lifecycle

    2019. Understand the Activity Lifecycle. https://developer.android.com/guide/ components/activities/activity-lifecycle. Accessed: 2019-01-29

    work page 2019

  17. [17]

    Using and Avoiding Null Explained

    2019. Using and Avoiding Null Explained. https://github.com/google/guava/wiki/ UsingAndAvoidingNullExplained. Accessed: 2019-01-29

    work page 2019

  18. [18]

    Edward Aftandilian, Raluca Sauciuc, Siddharth Priya, and Sundaresan Krishnan

    work page

  19. [19]

    In 12th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2012, Riva del Garda, Italy, September 23-24, 2012

    Building Useful Program Analysis Tools Using an Extensible Java Com- piler. In 12th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2012, Riva del Garda, Italy, September 23-24, 2012 . 14–23. https://doi.org/10.1109/SCAM.2012.28

    work page doi:10.1109/scam.2012.28 2012

  20. [20]

    O’Hearn, and Ilya Sergey

    Sam Blackshear, Nikos Gorogiannis, Peter W. O’Hearn, and Ilya Sergey. 2018. RacerD: compositional static race detection. PACMPL 2, OOPSLA (2018), 144:1– 144:28. https://doi.org/10.1145/3276514

    work page doi:10.1145/3276514 2018

  21. [21]

    Dan Brotherston, Werner Dietl, and Ondrej Lhoták. 2017. Granullar: gradual nullable types for Java. In Proceedings of the 26th International Conference on Compiler Construction, Austin, TX, USA, February 5-6, 2017 . 87–97. https://doi. org/10.1145/3033019.3033032

    work page doi:10.1145/3033019.3033032 2017

  22. [22]

    O’Hearn, and Hongseok Yang

    Cristiano Calcagno, Dino Distefano, Peter W. O’Hearn, and Hongseok Yang. 2011. Compositional Shape Analysis by Means of Bi-Abduction. J. ACM 58, 6 (2011), 26:1–26:66. https://doi.org/10.1145/2049697.2049700

    work page doi:10.1145/2049697.2049700 2011

  23. [23]

    Alain Deutsch. 1994. Interprocedural May-Alias Analysis for Pointers: Beyond k-limiting. In Proceedings of the ACM SIGPLAN’94 Conference on Programming Language Design and Implementation (PLDI), Orlando, Florida, USA, June 20-24,

    work page 1994

  24. [24]

    https://doi.org/10.1145/178243.178263

    230–241. https://doi.org/10.1145/178243.178263

    work page doi:10.1145/178243.178263

  25. [25]

    Ernst, Kıvanç Muşlu, and Todd Schiller

    Werner Dietl, Stephanie Dietzel, Michael D. Ernst, Kıvanç Muşlu, and Todd Schiller. 2011. Building and using pluggable type-checkers. In ICSE 2011, Pro- ceedings of the 33rd International Conference on Software Engineering . Waikiki, Hawaii, USA, 681–690. https://doi.org/10.1145/1985793.1985889

    work page doi:10.1145/1985793.1985889 2011

  26. [26]

    Rustan M

    Manuel Fähndrich and K. Rustan M. Leino. 2003. Declaring and checking non-null types in an object-oriented language. In Proceedings of the 2003 ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications, OOPSLA 2003, October 26-30, 2003, Anaheim, CA, USA . 302–312. https://doi.org/ 10.1145/949305.949332

    work page doi:10.1145/949305.949332 2003

  27. [27]

    Manuel Fähndrich and Songtao Xia. 2007. Establishing object invariants with delayed types. In Proceedings of the 22nd Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2007, October 21-25, 2007, Montreal, Quebec, Canada . 337–350. https://doi.org/10. 1145/1297027.1297052

    work page arXiv 2007

  28. [28]

    Matthew Finifter, Adrian Mettler, Naveen Sastry, and David A. Wagner. 2008. Verifiable functional purity in Java. In Proceedings of the 2008 ACM Conference on Computer and Communications Security, CCS 2008, Alexandria, Virginia, USA, October 27-31, 2008. 161–174. https://doi.org/10.1145/1455770.1455793

    work page doi:10.1145/1455770.1455793 2008

  29. [29]

    Stefan Heule and Charlie Garrett. 2019. A Dataflow Framework for Java. https: //checkerframework.org/manual/checker-framework-dataflow-manual.pdf Ac- cessed: 2019-02-07

    work page 2019

  30. [30]

    Wei Huang, Ana Milanova, Werner Dietl, and Michael D. Ernst. 2012. Reim & ReImInfer: checking and inference of reference immutability and method purity. In Proceedings of the 27th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2012, part of SPLASH 2012, Tucson, AZ, USA, October 21-25, 2012 . 879–...

    work page arXiv 2012

  31. [31]

    Alexey Loginov, Eran Yahav, Satish Chandra, Stephen Fink, Noam Rinetzky, and Mangala Gowri Nanda. 2008. Verifying dereference safety via expanding- scope analysis. In Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2008, Seattle, W A, USA, July 20-24, 2008 . 213–224. https://doi.org/10.1145/1390630.1390657

    work page doi:10.1145/1390630.1390657 2008

  32. [32]

    Ravichandhran Madhavan and Raghavan Komondoor. 2011. Null dereference ver- ification via over-approximated weakest pre-conditions analysis. In Proceedings of the 26th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2011, part of SPLASH 2011, Port- land, OR, USA, October 22 - 27, 2011 . 1033–1052. ...

    work page doi:10.1145/2048066 2011

  33. [33]

    Mangala Gowri Nanda and Saurabh Sinha. 2009. Accurate Interprocedural Null- Dereference Analysis for Java. In 31st International Conference on Software En- gineering, ICSE 2009, May 16-24, 2009, Vancouver, Canada, Proceedings . 133–143. https://doi.org/10.1109/ICSE.2009.5070515

    work page doi:10.1109/icse.2009.5070515 2009

  34. [34]

    Papi, Mahmood Ali, Telmo Luis Correa Jr., Jeff H

    Matthew M. Papi, Mahmood Ali, Telmo Luis Correa Jr., Jeff H. Perkins, and Michael D. Ernst. 2008. Practical pluggable types for Java. In ISSTA 2008, Proceed- ings of the 2008 International Symposium on Software Testing and Analysis . Seattle, WA, USA, 201–212. https://doi.org/10.1145/1390630.1390656

    work page doi:10.1145/1390630.1390656 2008

  35. [35]

    David J. Pearce. 2011. JPure: A Modular Purity System for Java. In Compiler Construction - 20th International Conference, CC 2011, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2011, Saarbrücken, Germany, March 26-April 3, 2011. Proceedings. 104–123. https://doi.org/10.1007/ 978-3-642-19861-8_7

    work page 2011

  36. [36]

    Benjamin C. Pierce. 2002. Types and Programming Languages (1st ed.). The MIT Press

    work page 2002

  37. [37]

    Xin Qi and Andrew C. Myers. 2009. Masked types for sound object initialization. In Proceedings of the 36th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2009, Savannah, GA, USA, January 21-23, 2009 . 53–65. https://doi.org/10.1145/1480881.1480890

    work page doi:10.1145/1480881.1480890 2009

  38. [38]

    Benno Stein, Lazaro Clapp, Manu Sridharan, and Bor-Yuh Evan Chang. 2018. Safe Stream-Based Programming with Refinement Types. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering . https://doi.org/10.1145/3238147.3238174

    work page doi:10.1145/3238147.3238174 2018

  39. [39]

    Summers and Peter Müller

    Alexander J. Summers and Peter Müller. 2011. Freedom before commitment: a lightweight type system for object initialisation. InProceedings of the 26th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2011, part of SPLASH 2011, Portland, OR, USA, October 22 - 27, 2011 . 1013–1032. https://doi.org/10...

    work page doi:10.1145/2048066.2048142 2011

  40. [40]

    Yoav Zibin, Alex Potanin, Paley Li, Mahmood Ali, and Michael D. Ernst. 2010. Ownership and immutability in generic Java. In Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno/Tahoe, Nevada, USA . 598–617. https://doi.org/10.1145/1869459.1869509

    work page doi:10.1145/1869459.1869509 2010