REVIEW 4 cited by
TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Machine learning models are notoriously difficult to interpret and debug. This is particularly true of neural networks. In this work, we introduce automated software testing techniques for neural networks that are well-suited to discovering errors which occur only for rare inputs. Specifically, we develop coverage-guided fuzzing (CGF) methods for neural networks. In CGF, random mutations of inputs to a neural network are guided by a coverage metric toward the goal of satisfying user-specified constraints. We describe how fast approximate nearest neighbor algorithms can provide this coverage metric. We then discuss the application of CGF to the following goals: finding numerical errors in trained neural networks, generating disagreements between neural networks and quantized versions of those networks, and surfacing undesirable behavior in character level language models. Finally, we release an open source library called TensorFuzz that implements the described techniques.
Forward citations
Cited by 4 Pith papers
-
Toward Reproducible Cross-Backend Compatibility for Deep Learning: A Configuration-First Framework with Three-Tier Verification
A configuration-first, three-tier verification framework quantifies cross-backend drift in deep learning, reporting a 72% pass rate across 672 checks and offering deterministic adapters as mitigations.
-
DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks
DeepEvolution uses population-based metaheuristics to evolve image transformations guided by neuron coverage, increasing coverage and detecting more DNN defects than TensorFuzz in small experiments.
-
Detecting Deep Neural Network Defects with Data Flow Analysis
DeepMorph claims to locate the root cause of poor deep network performance by comparing internal layer outputs, called data flow footprints, against patterns learned from training data.
-
Data Sanity Check for Deep Learning Systems via Learnt Assertions
SaneDL detects invalid inputs by using autoencoder reconstruction losses of intermediate activations as assertions, reporting high detection rates on two image dataset pairs.
Discussion (0). Continue with ORCID to comment.