Data Sanity Check for Deep Learning Systems via Learnt Assertions

Haochuan Lu; Huanlin Xu; Nana Liu; Xin Wang; Yangfan Zhou

REVIEW 3 major objections 5 minor 33 references

Data Sanity Check for Deep Learning Systems via Learnt Assertions

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SaneDL inserts autoencoder-based assertions into pretrained neural networks to flag inputs the model was never trained on.

desk verdict This paper re-frames autoencoder anomaly detection as 'learnt assertions' for DL systems; the framing is plausible but the evaluation is too thin and the exact thresholds behind its headline numbers are never reported. read the letter →

arxiv 1909.03835 v3 pith:KDYYQEFM submitted 2019-09-06 cs.LG stat.ML

Haochuan Lu , Huanlin Xu , Nana Liu , Yangfan Zhou , Xin Wang This is my paper · ORCID

classification cs.LGstat.ML

keywords datasanitycheckdeeplearningreliabilityinvalidinputdetectionautoencoderassertionout-of-distributionneuralnetworktestingruntimevalidationflowfootprint

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SaneDL, a tool that checks whether an input is valid for a deep learning system before the system fully processes it. The central idea is to insert autoencoder-based assertions between the layers of a pretrained network, train those autoencoders only on intermediate activations produced by valid inputs, and then reject any input whose reconstruction loss at any layer exceeds a threshold. The paper argues that valid inputs are reconstructed well because their activations match the training distribution, while invalid inputs produce abnormal activations and therefore high losses. If this claim holds, SaneDL offers a lightweight, non-intrusive runtime guard for existing deep learning systems, catching out-of-distribution inputs that would otherwise cause unpredictable behavior.

What carries the argument

The central object is the autoencoder-based assertion: a small autoencoder placed between two layers of a pretrained network, trained to reconstruct that layer's intermediate activations from valid inputs. Its reconstruction loss serves as an anomaly score for the input's behavior at that layer, and the threshold is set as a multiple of the mean training loss. The mechanism works by cascading these assertions across layers so that an input must satisfy every layer-specific reconstruction constraint to be accepted.

What would settle it

Take a set of inputs that are close to the valid data distribution, such as valid images with small added noise, mild blur, or a different handwriting style, and measure their per-layer autoencoder losses; if a substantial fraction have losses below $\delta \times$ mean training loss, then the pass-all-assertions rule would wrongly accept them as valid.

Watch

Extended reading notes

Core claim

The paper's central claim is that data sanity can be enforced for deep learning systems by treating each intermediate layer's activations as a data-flow signal and checking it with a learned assertion. Concretely, for a pretrained network N, autoencoders are inserted between layers and trained on the intermediate results of valid training inputs; at runtime, an input is verified only if it passes all assertions, meaning every per-layer reconstruction loss stays below the threshold $\delta \cdot \frac{1}{m}\sum_{j=1}^{m}\text{Loss}_{AE}(IR_j)$, where $\delta$ is a scale coefficient typically chosen between 2 and 4. In the two tested scenarios, the method reports true positive rates of 0.9268 to 0.9975 and false positive rates of 0.0156 to 0.1002 across LeNet, AlexNet, and VGG16, using real-world invalid input cases from MNIST and FlickrLogos rather than artificially perturbed samples. The paper thus establishes that behavior deviation, measured through data-flow reconstruction losses, is a workable symptom for identifying inputs a deep learning system cannot properly handle.

Load-bearing premise

The method assumes that every invalid input produces intermediate activations whose autoencoder reconstruction loss clearly exceeds the training-time mean, so that a single global threshold per layer can cleanly separate valid from invalid inputs.

Editorial extensions

If this is right

Existing pretrained models can be guarded without retraining or architectural changes, since the autoencoders are inserted as add-on monitors.
The same pipeline applies across different network architectures and domains, as demonstrated with LeNet, AlexNet, and VGG16 in both digit recognition and traffic-sign recognition.
The trade-off between catching invalid inputs and accepting valid ones is controlled by the single scale coefficient $\delta$, giving operators a direct knob for system sensitivity.
Real-world invalid inputs, not just crafted adversarial samples, can be detected at runtime with the reported true and false positive rates.
Because the check is per-layer, it can localize which stage of the network first deviates from expected behavior, aiding diagnosis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The global threshold $\delta \times$ mean loss treats every layer identically; a natural extension is to calibrate $\delta$ per layer against a validation set to optimize the trade-off between true and false positives.
The method essentially performs out-of-distribution detection on intermediate activations, so it could be tested on subtler distribution shifts, such as corrupted or lightly perturbed inputs, where the loss separation may be less clean.
Since the autoencoders are trained only on valid inputs, they could be updated online as the production data distribution drifts, keeping the sanity check aligned with evolving input patterns.
The assertion cascade could be combined with prediction confidence to catch inputs whose activations look normal but still produce incorrect outputs, though the paper does not explore this combination.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper reframes a standard technique—autoencoder reconstruction error as an anomaly score—as a runtime 'assertion' mechanism for deep learning systems. The reframing is fine as a packaging exercise, but the evidence is thin and the paper omits the one number that would make its claims reproducible: the δ values used to produce Table I.

What's actually new is the software-engineering angle: inserting AE-based checks between layers, scaling the threshold from the mean training loss, and presenting it as a non-intrusive plugin for deployed models. That's a reasonable thing to try, and I'll credit the authors for testing on real-world distribution shifts (MNIST vs Fashion-MNIST, GTSRB vs FlickrLogos) rather than adversarial perturbations. The AUC scores in Table I are threshold-independent and quite high (0.97–0.99), so the core idea probably has something to it.

But the evaluation is not enough. There are no baseline comparisons—no ODIN, no Mahalanobis-distance OOD detection, no even simple softmax-confidence baseline. No error bars or significance tests. And the stress-test concern is real: the paper never states which δ gives TPR 0.9919 and FPR 0.0156, only that δ in [2,4] 'results in good performance.' Since TPR and FPR trade off monotonically with δ, the reported operating point is not reproducible from the paper alone. That's not a stylistic complaint; for a method whose only knob is δ, it's a load-bearing gap. Missing AE hyperparameters, insertion-layer choices, and the number of assertions per model compound the problem. The writing is also sloppy, but that's minor.

I don't find the method circular—the AEs are trained on valid inputs and thresholds are calibrated from training data, which is standard practice. The AUC numbers are plausible. What bothers me more is the novelty claim: calling this the first assertion-based tool for DL systems ignores a substantial OOD-detection literature that already uses intermediate-layer features with learned thresholds. The authors don't cite any of it.

Who is this paper for? Software-engineering researchers who want a simple runtime monitor for DL systems and haven't followed the OOD detection literature. For an ML audience, it's a rehash. A serious referee at a SE-focused venue could extract a usable paper by enforcing baselines, variance reporting, and the exact δ values; as it stands, it's a workshop-level draft. I'd send it back for major revision rather than reject, but only if the authors can show the threshold choice is fixed before deployment and the reported numbers hold under sensitivity analysis.

Referee Report

3 major / 5 minor

Summary. The paper proposes SaneDL, a runtime input-validation mechanism for deep learning systems. Autoencoders are inserted between layers of a pre-trained model and trained to reconstruct intermediate activations of valid inputs. At inference time, each assertion computes a reconstruction loss, and the input is rejected if any loss exceeds a threshold defined as a constant delta multiplied by the mean training loss. The method is evaluated on two scenarios: distinguishing MNIST digits from Fashion-MNIST inputs when the model was trained on Fashion-MNIST, and distinguishing Flickr logos from GTSRB traffic signs when the model was trained on GTSRB. The reported results show high true-positive rates, false-positive rates between 0.0156 and 0.1002, and high ROC-AUC scores across LeNet, AlexNet, and VGG16.

Significance. If the results are reproducible, SaneDL offers a simple, non-intrusive mechanism for detecting out-of-distribution inputs at runtime, which is practically valuable for deployed deep learning systems. The use of real-world invalid input cases rather than adversarially crafted perturbations is a strength, and the idea of inserting trainable assertions into existing networks is intuitively appealing. However, the evaluation is too thin to establish the stated general claim: there are no comparisons with existing out-of-distribution detection baselines, no error bars or significance tests, and no public implementation or detailed hyperparameter report. The method's single tunable parameter, delta, is never reported exactly, which makes the headline numbers in Table I unreproducible. The significance is therefore conditional on a substantially strengthened evaluation.

major comments (3)

[Section II, threshold equation; Table I] The operating point behind Table I is not specified. The threshold is defined as Thres = delta times the mean reconstruction loss on training data, and the text states only that 'selecting this scale coefficient in an interval from 2 to 4 results in good performance.' The exact delta used for each model, any selection procedure, and the sensitivity of TPR and FPR to delta are omitted. Because TPR and FPR trade off monotonically with the threshold scale, the reported values such as FPR 0.0156 and TPR 0.9919 cannot be reproduced or interpreted from the paper. This is load-bearing: the entire detection mechanism depends on this threshold, and if delta was chosen after observing the test set, the reported numbers could be optimistically biased.
[Section II and Section III-A] The core assumption that valid and invalid intermediate activations yield cleanly separable reconstruction-loss distributions is not demonstrated. The paper asserts that invalid inputs cause 'a huge deviation' but provides no loss distributions, no per-assertion thresholds, and no per-assertion AUC or false-positive rates. Since the final decision requires passing 'all the assertions,' per-assertion false-positive rates compound, and without per-assertion data it is unclear how the aggregated FPR is achieved. This is central to the effectiveness claim, and the omission prevents the reader from validating the mechanism on subtle or adversarial distribution shifts, which may not exhibit clean separation.
[Section III-A, Table I] The empirical support for the general claim is insufficient. There is no comparison with even simple baselines such as softmax confidence thresholds, input-space reconstruction error, or established out-of-distribution detectors, so the relative benefit of SaneDL is unknown. The two scenarios are easy dataset shifts (MNIST versus Fashion-MNIST, traffic signs versus logos), and the test-set sizes, replacement proportions, and random-seed details are not reported. No error bars or significance tests are given, so the differences between models (for example, FPR 0.0801 versus 0.1002) may be within noise. These details are necessary to assess whether the proposed method is reliably effective.

minor comments (5)

[Throughout] The manuscript contains several typos and grammatical errors: 'evluate' in Section III-A, 'dont' and 'constrains' in Section II, and 'the intermediate results of invalid puts' in Section II. These should be corrected.
[Section III-A, Reference [11]] The text refers to 'FlickrLogos-273' while the reference is titled 'FlickrLogos-27'; the dataset name is inconsistent and should be unified.
[Section II, Figure 1] The AE architectures, training epochs, and the exact layer insertion points are not described. Figure 1 is referenced but not visible in the manuscript text; the authors should ensure the figure clearly shows where assertions are placed and how the thresholds are computed per layer.
[Related Work] The related work section cites software testing literature but does not discuss the large body of work on out-of-distribution detection in machine learning (for example, softmax-based confidence methods and density-based detectors). This makes the novelty claim of an 'assertion-based tool' difficult to evaluate and should be addressed.
[Section III-A, Table I] The text says ROC-AUC is used to evaluate performance under different delta, but Table I reports only a single point per model with no ROC curves or AUC intervals. Clarify how the tabulated TPR/FPR values relate to the reported AUC.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the assertion thresholds are fitted to valid training data and the detection claim is tested on held-out invalid inputs.

full rationale

The derivation chain is self-contained and empirically grounded. SaneDL trains autoencoders on intermediate activations of a pre-trained network using only valid training inputs; the assertion threshold is defined in Section II as δ times the mean reconstruction loss over that same training set (Thres = δ · mean loss). The stated claim is that inputs whose layer-wise reconstruction losses exceed these thresholds are invalid. This is a standard anomaly-detection setup: the thresholds are fitted to the valid-input distribution, and the evaluation in Section III uses held-out mixed test sets (Fashion-MNIST/MNIST and GTSRB/FlickrLogos) with ground-truth labels, reporting TPR, FPR, and AUC. No predicted quantity is defined in terms of the result it is supposed to confirm, and no load-bearing claim rests on self-citations. The statement that δ∈[2,4] gives good performance is an empirical observation, not a derivation from the target result. Concerns about the exact δ values being unreported are reproducibility issues, not circularity, because the reported AUC is threshold-independent and the TPR/FPR tradeoff does not make the core detection claim definitionally true. Hence no circular step can be exhibited from the paper's equations or citations.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The main free parameter is the threshold scale δ, chosen by hand, plus unspecified AE design choices. The method rests on a domain assumption that autoencoder reconstruction loss cleanly separates valid from invalid internal activations. No new physical or conceptual entities are introduced.

free parameters (2)

Threshold scale coefficient δ = claimed good in [2,4]
Manual hyperparameter in the threshold formula Thres = δ · mean training loss; the paper states practice shows 2 to 4 works but gives no sensitivity analysis or per-scenario tuning details.
Autoencoder architecture and insertion layers = not specified
The paper does not report the AE structure, feature dimensions, or which layers receive assertions. These choices directly affect detection performance and are unstated free parameters.

assumptions (2)

domain assumption Autoencoders trained on valid inputs reconstruct valid intermediate activations with low error and invalid ones with high error.
This is the load-bearing premise of the method, introduced in Section II and relied on throughout. It is only tested on two simple dataset pairs and is not guaranteed for broader classes of invalid inputs.
domain assumption Behavior deviation in data flow footprints is a reliable symptom of invalid inputs.
Stated in Section I, citing the 'bugs as deviant behavior' idea from software engineering. The paper assumes this transfers to DL data flow without formal justification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Data Sanity Check for Deep Learning Systems via Learnt Assertions." pith.science (2026). https://pith.science/paper/KDYYQEFM

@misc{pith2026190903835,
  author       = {Pith},
  title        = {Pith review of: Data Sanity Check for Deep Learning Systems via Learnt Assertions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDYYQEFM}},
  note         = {Machine review of arXiv:1909.03835}
}

read the original abstract

Reliability is a critical consideration to DL-based systems. But the statistical nature of DL makes it quite vulnerable to invalid inputs, i.e., those cases that are not considered in the training phase of a DL model. This paper proposes to perform data sanity check to identify invalid inputs, so as to enhance the reliability of DL-based systems. We design and implement a tool to detect behavior deviation of a DL model when processing an input case. This tool extracts the data flow footprints and conducts an assertion-based validation mechanism. The assertions are built automatically, which are specifically-tailored for DL model data flow analysis. Our experiments conducted with real-world scenarios demonstrate that such an assertion-based data sanity check mechanism is effective in identifying invalid input cases.

Figures

Figures reproduced from arXiv: 1909.03835 by the authors.

**Figure 1.** shows the framework of SaneDL. Generally, for a pre-trained DNN model, the execution process of SaneDL follows the following workflow. First, a series of AutoEncoder (AE) [9] based assertions are inserted into the structure of a pre-trained DNN. Specifically, given a pre-trained neural network N, we insert AE networks between its layers. These AEs are trained using the intermediate result generated by each layer of… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 28 canonical work pages

[1]

Real time road edges detection and road signs recognition,

J. Duan and V . Malichenko, “Real time road edges detection and road signs recognition,” in International Conference on Control, Automation and Information Sciences, ICCAIS , 2015, pp. 107–112

work page 2015
[2]

Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Trans. Med. Imaging , vol. 35, no. 5, pp. 1153–1159, 2016

work page 2016
[3]

Testing advanced driver assistance systems using multi-objective search and neural networks,

R. B. Abdessalem, S. Nejati, L. C. Briand, and T. Stifter, “Testing advanced driver assistance systems using multi-objective search and neural networks,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE , 2016, pp. 63–74

work page 2016
[4]

A domain strategy for computer program testing,

L. J. White and E. I. Cohen, “A domain strategy for computer program testing,” IEEE Transactions on Software Engineering , no. 3, pp. 247– 257, 1980

work page 1980
[5]

Mitv: multiple- implementation testing of user-input validators for web applications,

K. Taneja, N. Li, M. R. Marri, T. Xie, and N. Tillmann, “Mitv: multiple- implementation testing of user-input validators for web applications,” in 25th IEEE/ACM International Conference on Automated Software Engineering, ASE , 2010, pp. 131–134

work page 2010
[6]

Perturbation-based user-input- validation testing of web applications,

N. Li, T. Xie, M. Jin, and C. Liu, “Perturbation-based user-input- validation testing of web applications,” Journal of Systems and Software , vol. 83, no. 11, pp. 2263–2274, 2010

work page 2010
[7]

Semi-valid input coverage for fuzz testing,

P. Tsankov, M. T. Dashti, and D. A. Basin, “Semi-valid input coverage for fuzz testing,” in International Symposium on Software Testing and Analysis, ISSTA, 2013, pp. 56–66

work page 2013
[8]

Bugs as deviant behavior: A general approach to inferring errors in systems code,

D. Engler, D. Y . Chen, S. Hallem, A. Chou, and B. Chelf, “Bugs as deviant behavior: A general approach to inferring errors in systems code,” in ACM SIGOPS Operating Systems Review , vol. 35, no. 5, 2001, pp. 57–72

work page 2001

Show all 33 references

[9]

Reducing the dimensionality of data with neural networks,

G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” science, vol. 313, no. 5786, pp. 504–507, 2006

2006
[10]

Detection of trafﬁc signs in real-world images: The German Trafﬁc Sign Detection Benchmark,

S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel, “Detection of trafﬁc signs in real-world images: The German Trafﬁc Sign Detection Benchmark,” inInternational Joint Conference on Neural Networks, no. 1288, 2013

2013
[11]

Scalable triangulation-based logo recognition,

Y . Kalantidis, L. G. Pueyo, M. Trevisiol, R. van Zwol, and Y . Avrithis, “Scalable triangulation-based logo recognition,” in Proceedings of the 1st International Conference on Multimedia Retrieval, ICMR , 2011, p. 20

2011
[12]

Gradient-based learning applied to document recognition,

Y . LeCun, L. Bottou, Y . Bengio, P. Haffner et al. , “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998

1998
[13]

Imagenet classiﬁcation with deep convolutional neural networks,

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classiﬁcation with deep convolutional neural networks,” in Advances in neural infor- mation processing systems , 2012, pp. 1097–1105

2012
[14]

Very deep convolutional networks for large-scale image recognition,

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

2014 arXiv
[15]

Increased software reliability through input validation analysis and testing,

J. H. Hayes and A. J. Offutt, “Increased software reliability through input validation analysis and testing,” in Proceedings 10th International Symposium on Software Reliability Engineering (Cat. No. PR00443) . IEEE, 1999, pp. 199–209

1999
[16]

Security testing of web applications: A research plan,

A. Avancini, “Security testing of web applications: A research plan,” in 34th International Conference on Software Engineering, ICSE , 2012, pp. 1491–1494

2012
[17]

Semantic differential repair for input validation and sanitization,

M. Alkhalaf, A. Aydin, and T. Bultan, “Semantic differential repair for input validation and sanitization,” in International Symposium on Software Testing and Analysis, ISSTA , 2014, pp. 225–236

2014
[18]

Using parse tree validation to prevent SQL injection attacks,

G. Buehrer, B. W. Weide, and P. A. G. Sivilotti, “Using parse tree validation to prevent SQL injection attacks,” in Proceedings of the 5th International Workshop on Software Engineering and Middleware, SEM , 2005, pp. 106–113

2005
[19]

Saner: Composing static and dynamic anal- ysis to validate sanitization in web applications,

D. Balzarotti, M. Cova, V . Felmetsger, N. Jovanovic, E. Kirda, C. Kruegel, and G. Vigna, “Saner: Composing static and dynamic anal- ysis to validate sanitization in web applications,” in IEEE Symposium on Security and Privacy (S&P) , 2008, pp. 387–401

2008
[20]

Predicting common web application vulnerabilities from input validation and sanitization code patterns,

L. K. Shar and H. B. K. Tan, “Predicting common web application vulnerabilities from input validation and sanitization code patterns,” in IEEE/ACM International Conference on Automated Software Engineer- ing, ASE , 2012, pp. 310–313

2012
[21]

Preventing input validation vulnerabilities in web applications through automated type analysis,

T. Scholte, W. K. Robertson, D. Balzarotti, and E. Kirda, “Preventing input validation vulnerabilities in web applications through automated type analysis,” in 36th Annual IEEE Computer Software and Applica- tions Conference, COMPSAC , 2012, pp. 233–243

2012
[22]

Web application intrusion detection system for input validation attack,

Y . Park and J. Park, “Web application intrusion detection system for input validation attack,” in Third International Conference on Conver- gence and Hybrid Information Technology , vol. 2. IEEE, 2008, pp. 498–504

2008
[23]

Testing deep neural networks,

Y . Sun, X. Huang, and D. Kroening, “Testing deep neural networks,”
[24]

Tensorfuzz: Debugging neural networks with coverage-guided fuzzing,

A. Odena and I. J. Goodfellow, “Tensorfuzz: Debugging neural networks with coverage-guided fuzzing,” 2018. [Online]. Available: http://arxiv.org/abs/1807.10875

2018 arXiv
[25]

MODE: automated neural network model debugging via state differential analysis and input selection,

S. Ma, Y . Liu, W. Lee, X. Zhang, and A. Grama, “MODE: automated neural network model debugging via state differential analysis and input selection,” in Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the F oundations of Softwa...

2018
[26]

Deepsafe: A data-driven approach for checking adversarial robustness in neural networks,

D. Gopinath, G. Katz, C. S. Pasareanu, and C. Barrett, “Deepsafe: A data-driven approach for checking adversarial robustness in neural networks,” 2017. [Online]. Available: http://arxiv.org/abs/1710.00486

2017 arXiv
[27]

Deepmutation: Mutation testing of deep learning systems,

L. Ma, F. Zhang, J. Sun, M. Xue, B. Li, F. Juefei-Xu, C. Xie, L. Li, Y . Liu, J. Zhao, and Y . Wang, “Deepmutation: Mutation testing of deep learning systems,” in 29th IEEE International Symposium on Software Reliability Engineering, ISSRE , 2018, pp. 100–111

2018
[28]

Deepgauge: multi- granularity testing criteria for deep learning systems,

L. Ma, F. Juefei-Xu, F. Zhang, J. Sun, M. Xue, B. Li, C. Chen, T. Su, L. Li, Y . Liu, J. Zhao, and Y . Wang, “Deepgauge: multi- granularity testing criteria for deep learning systems,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineeri...

2018
[29]

Concolic testing for deep neural networks,

Y . Sun, M. Wu, W. Ruan, X. Huang, M. Kwiatkowska, and D. Kroening, “Concolic testing for deep neural networks,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineer- ing, ASE , 2018, pp. 109–119

2018
[30]

Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems,

M. Zhang, Y . Zhang, L. Zhang, C. Liu, and S. Khurshid, “Deeproad: Gan-based metamorphic testing and input validation framework for autonomous driving systems,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, ASE , 2018, pp. 132–142

2018
[31]

Deepxplore: Automated whitebox testing of deep learning systems,

K. Pei, Y . Cao, J. Yang, and S. Jana, “Deepxplore: Automated whitebox testing of deep learning systems,” in Proceedings of the 26th Symposium on Operating Systems Principles , 2017, pp. 1–18

2017
[32]

Deeptest: automated testing of deep-neural-network-driven autonomous cars,

Y . Tian, K. Pei, S. Jana, and B. Ray, “Deeptest: automated testing of deep-neural-network-driven autonomous cars,” in Proceedings of the 40th International Conference on Software Engineering, ICSE , 2018, pp. 303–314. 3

2018
[2018]

Available: http://arxiv.org/abs/1803.04792

[Online]. Available: http://arxiv.org/abs/1803.04792

arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Real time road edges detection and road signs recognition,

J. Duan and V . Malichenko, “Real time road edges detection and road signs recognition,” in International Conference on Control, Automation and Information Sciences, ICCAIS , 2015, pp. 107–112

work page 2015

[2] [2]

Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,

H. Greenspan, B. van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Trans. Med. Imaging , vol. 35, no. 5, pp. 1153–1159, 2016

work page 2016

[3] [3]

Testing advanced driver assistance systems using multi-objective search and neural networks,

R. B. Abdessalem, S. Nejati, L. C. Briand, and T. Stifter, “Testing advanced driver assistance systems using multi-objective search and neural networks,” in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ASE , 2016, pp. 63–74

work page 2016

[4] [4]

A domain strategy for computer program testing,

L. J. White and E. I. Cohen, “A domain strategy for computer program testing,” IEEE Transactions on Software Engineering , no. 3, pp. 247– 257, 1980

work page 1980

[5] [5]

Mitv: multiple- implementation testing of user-input validators for web applications,

K. Taneja, N. Li, M. R. Marri, T. Xie, and N. Tillmann, “Mitv: multiple- implementation testing of user-input validators for web applications,” in 25th IEEE/ACM International Conference on Automated Software Engineering, ASE , 2010, pp. 131–134

work page 2010

[6] [6]

Perturbation-based user-input- validation testing of web applications,

N. Li, T. Xie, M. Jin, and C. Liu, “Perturbation-based user-input- validation testing of web applications,” Journal of Systems and Software , vol. 83, no. 11, pp. 2263–2274, 2010

work page 2010

[7] [7]

Semi-valid input coverage for fuzz testing,

P. Tsankov, M. T. Dashti, and D. A. Basin, “Semi-valid input coverage for fuzz testing,” in International Symposium on Software Testing and Analysis, ISSTA, 2013, pp. 56–66

work page 2013

[8] [8]

Bugs as deviant behavior: A general approach to inferring errors in systems code,

D. Engler, D. Y . Chen, S. Hallem, A. Chou, and B. Chelf, “Bugs as deviant behavior: A general approach to inferring errors in systems code,” in ACM SIGOPS Operating Systems Review , vol. 35, no. 5, 2001, pp. 57–72

work page 2001