pith. sign in

arxiv: 1907.10159 · v1 · pith:PX2UCINMnew · submitted 2019-07-23 · 💻 cs.CR · cs.LG· cs.SE

Efficient Detection and Quantification of Timing Leaks with Neural Networks

Pith reviewed 2026-05-24 17:08 UTC · model grok-4.3

classification 💻 cs.CR cs.LGcs.SE
keywords timing side channelsneural networksinformation leakage quantificationMILP analysisdynamic analysisside-channel detectionprogram timing models
0
0 comments X

The pith

Neural networks learn timing behaviors of programs with thousands of methods and can be analyzed to quantify timing side-channel leaks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that splitting timing side-channel analysis into two tasks—learning a neural network model of a program's execution times and then analyzing that model with mixed-integer linear programming—makes both detection and quantification practical for real applications. Static techniques are too slow for large codebases and usually return only yes-or-no results, while this dynamic method can also report how much information is leaked. A sympathetic reader would care because many programs must intentionally reveal some data, so measuring the size of any unintended timing leak helps judge the actual risk. Experiments on micro-benchmarks and real-world code show the networks capture timing for programs with thousands of methods and that the resulting models remain solvable even when they contain thousands of neurons.

Core claim

The authors claim that a neural network can be trained to predict program timing from inputs and that an MILP encoding of this network can then be solved to compute the maximum timing difference between secret-dependent paths, thereby both detecting and quantifying the strength of timing side channels at a scale unreachable by direct static analysis.

What carries the argument

Neural network timing model combined with MILP encoding of the network to estimate side-channel strength.

If this is right

  • Real-world applications too large for static analysis can still have their timing leaks detected and measured.
  • Leakage is reported as a quantity rather than a binary answer, allowing direct comparison of threat levels.
  • The two-task split keeps both the learning step and the MILP step computationally feasible for networks with thousands of neurons.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same modeling technique generalizes, learned timing models could be reused to check other observable behaviors such as cache or power side channels.
  • Quantified leak values could be fed back into compiler passes that search for constant-time rewrites of the most leaky code paths.
  • The approach suggests a template for using learned models plus optimization for other dynamic security properties beyond timing.

Load-bearing premise

The neural network's predicted timing must match the program's actual timing closely enough for the MILP results on the model to reflect real leaks.

What would settle it

Executing the original program on inputs that the MILP analysis flags as producing a large timing gap and measuring that the observed times show no such gap would falsify the claim that the model-based quantification is reliable.

Figures

Figures reproduced from arXiv: 1907.10159 by Ashutosh Trivedi, Pavol Cerny, Saeid Tizpaz-Niari, Sriram Sankaranarayanan.

Figure 1
Figure 1. Figure 1: (a) Timing models of the sorting application as learned by NNs. (b) Pre [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Neural network architecture for side-channel discovery and quantification. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Neural Network Nδ approximating the execution-time function δ along with a reducer (Nα, Nβ) [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (a) The SSE versus the number of interface neurons for R [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The SSE versus the number of interface neurons k for case-study appli [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
read the original abstract

Detection and quantification of information leaks through timing side channels are important to guarantee confidentiality. Although static analysis remains the prevalent approach for detecting timing side channels, it is computationally challenging for real-world applications. In addition, the detection techniques are usually restricted to 'yes' or 'no' answers. In practice, real-world applications may need to leak information about the secret. Therefore, quantification techniques are necessary to evaluate the resulting threats of information leaks. Since both problems are very difficult or impossible for static analysis techniques, we propose a dynamic analysis method. Our novel approach is to split the problem into two tasks. First, we learn a timing model of the program as a neural network. Second, we analyze the neural network to quantify information leaks. As demonstrated in our experiments, both of these tasks are feasible in practice --- making the approach a significant improvement over the state-of-the-art side channel detectors and quantifiers. Our key technical contributions are (a) a neural network architecture that enables side channel discovery and (b) an MILP-based algorithm to estimate the side-channel strength. On a set of micro-benchmarks and real-world applications, we show that neural network models learn timing behaviors of programs with thousands of methods. We also show that neural networks with thousands of neurons can be efficiently analyzed to detect and quantify information leaks through timing side channels.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes splitting timing side-channel analysis into two stages: (1) learning a neural-network surrogate for a program's timing behavior from dynamic execution traces, and (2) encoding the trained NN as a mixed-integer linear program (MILP) whose solution yields bounds on timing differences over secret inputs, thereby detecting and quantifying leaks. Experiments on micro-benchmarks and real-world applications are reported to show that NNs can be trained for programs containing thousands of methods and that the resulting MILPs remain tractable for networks with thousands of neurons.

Significance. If the NN models prove sufficiently faithful and the MILP bounds are shown to be reliable, the method would supply a practical dynamic route to both detection and quantitative leak assessment in applications where static analysis is intractable. The technical device of an MILP encoding of a trained NN is a concrete contribution that could be reused beyond this setting.

major comments (2)
  1. [Experimental Evaluation] Experimental Evaluation (and abstract): no quantitative fidelity metrics (MAE, R², or held-out trace error) or direct validation of MILP-derived leak bounds against measurements on the original program are supplied. Without these, the central claim that the two-stage procedure yields sound quantifications rests on an unchecked assumption that the learned NN is a faithful surrogate.
  2. [Method] Method (two-stage description): the paper does not bound or mitigate systematic approximation error introduced by the NN (e.g., on branches, cache effects, or dispatch) before the MILP optimizer is applied; any such bias can be amplified by the optimizer, producing spurious or missed leaks. A concrete error-propagation argument or empirical sanity check against the original program is required for the quantification results to be load-bearing.
minor comments (1)
  1. [Abstract] The abstract asserts a 'significant improvement over the state-of-the-art' without naming the baselines or reporting quantitative deltas; a short comparison table would clarify the claim.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important aspects for strengthening the presentation of our results. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses
  1. Referee: [Experimental Evaluation] Experimental Evaluation (and abstract): no quantitative fidelity metrics (MAE, R², or held-out trace error) or direct validation of MILP-derived leak bounds against measurements on the original program are supplied. Without these, the central claim that the two-stage procedure yields sound quantifications rests on an unchecked assumption that the learned NN is a faithful surrogate.

    Authors: We agree that quantitative fidelity metrics and direct validation of the derived bounds would make the claims more robust. In the revised version we will report MAE, R², and held-out trace prediction error for all trained networks. For the micro-benchmarks we will also add side-by-side comparisons of MILP-computed timing differences against exhaustive measurements on the original programs; for the larger applications we will report the same metrics on a held-out set of secret inputs. revision: yes

  2. Referee: [Method] Method (two-stage description): the paper does not bound or mitigate systematic approximation error introduced by the NN (e.g., on branches, cache effects, or dispatch) before the MILP optimizer is applied; any such bias can be amplified by the optimizer, producing spurious or missed leaks. A concrete error-propagation argument or empirical sanity check against the original program is required for the quantification results to be load-bearing.

    Authors: We acknowledge that the manuscript does not yet contain an explicit error-propagation analysis. In the revision we will add a dedicated subsection discussing sources of approximation error (branch misprediction, cache effects, method dispatch) and their potential amplification under MILP optimization. We will also include the empirical sanity checks described in the response to the first comment and will state the resulting limitations on the soundness of the quantified bounds. revision: yes

Circularity Check

0 steps flagged

No circularity in two-stage NN learning and MILP analysis

full rationale

The paper's core derivation splits into independent stages: (1) supervised learning of an NN timing model from dynamic execution traces, and (2) separate MILP encoding and optimization over the fixed NN weights to bound output differences. Neither stage reduces to the other by definition or construction; the MILP step treats the trained NN as an external black-box function rather than re-deriving fitted parameters. No self-citations, uniqueness theorems, or ansatzes are invoked in the provided text to justify the central claims. The approach is therefore self-contained against external benchmarks (trace data and MILP solvers) and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only; no explicit free parameters, axioms, or invented entities are stated. The method implicitly relies on standard assumptions of NN approximation power and MILP solver correctness for the central claim.

pith-pipeline@v0.9.0 · 5788 in / 870 out tokens · 17134 ms · 2026-05-24T17:08:04.338684+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages · 5 internal anchors

  1. [1]

    American fuzzy lop (2016), http://lcamtuf.coredump.cx/afl/

  2. [2]

    In: OSDI’16

    Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghe- mawat, S., Irving, G., Isard, M., et al.: Tensorflow: A system for large-scale machine learning. In: OSDI’16. pp. 265–283 (2016)

  3. [3]

    In: Proceedings of the 27th ACM SIGPLAN-SIGACT symposium on Principles of programming languages

    Agat, J.: Transforming out timing leaks. In: Proceedings of the 27th ACM SIGPLAN-SIGACT symposium on Principles of programming languages. pp. 40–

  4. [4]

    In: USENIX Security Symposium

    Almeida, J.B., Barbosa, M., Barthe, G., Dupressoir, F., Emmi, M.: Verify- ing constant-time implementations. In: USENIX Security Symposium. pp. 53–70 (2016)

  5. [5]

    In: ACM SIGPLAN Notices

    Antonopoulos, T., Gazzillo, P., Hicks, M., Koskinen, E., Terauchi, T., Wei, S.: De- composition instead of self-composition for proving the absence of timing channels. In: ACM SIGPLAN Notices. vol. 52, pp. 362–375. ACM (2017)

  6. [6]

    Online: https://github.com/Apogee-Research/STAC

    Apogee-Research: Space/time analysis for cybersecurity (stac) repository. Online: https://github.com/Apogee-Research/STAC

  7. [7]

    Applied Soft Computing 33, 263–277 (2015)

    Arar, ¨O.F., Ayan, K.: Software defect prediction using cost-sensitive neural net- work. Applied Soft Computing 33, 263–277 (2015)

  8. [8]

    arXiv e-prints (2016)

    Arora, R., Basu, A., Mianjy, P., Mukherjee, A.: Understanding Deep Neural Net- works with Rectified Linear Units. arXiv e-prints (2016)

  9. [9]

    In: Proceedings of the 17th ACM conference on Computer and commu- nications security

    Askarov, A., Zhang, D., Myers, A.C.: Predictive black-box mitigation of timing channels. In: Proceedings of the 17th ACM conference on Computer and commu- nications security. pp. 297–307. ACM (2010)

  10. [10]

    In: S&P’09 (2009)

    Backes, M., K¨ opf, B., Rybalchenko, A.: Automatic discovery and quantification of information leaks. In: S&P’09 (2009)

  11. [11]

    In: Computer Security Foundations Workshop, 2004

    Barthe, G., D’Argenio, P.R., Rezk, T.: Secure information flow by self-composition. In: Computer Security Foundations Workshop, 2004. Proceedings. 17th IEEE. pp. 100–114. IEEE (2004)

  12. [12]

    Computer Networks 48(5), 701–716 (2005)

    Brumley, D., Boneh, D.: Remote timing attacks are practical. Computer Networks 48(5), 701–716 (2005)

  13. [13]

    In: CCS (2017)

    Chen, J., Feng, Y., Dillig, I.: Precise detection of side-channel vulnerabilities using quantitative cartesian hoare logic. In: CCS (2017)

  14. [14]

    In: S&P’10 (2010)

    Chen, S., Wang, R., Wang, X., Zhang, K.: Side-channel leaks in web applications: A reality today, a challenge tomorrow. In: S&P’10 (2010)

  15. [15]

    Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

    Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 (2016)

  16. [16]

    Mathe- matics of Control, Signals, and Systems 2, 303–314 (12 1989)

    Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathe- matics of Control, Signals, and Systems 2, 303–314 (12 1989)

  17. [17]

    ACM Transactions on Information and System Security (TISSEC) 18(1), 4 (2015)

    Doychev, G., K¨ opf, B., Mauborgne, L., Reineke, J.: Cacheaudit: A tool for the static analysis of cache side channels. ACM Transactions on Information and System Security (TISSEC) 18(1), 4 (2015)

  18. [18]

    In: NASA Formal Methods Symposium

    Dutta, S., Jha, S., Sankaranarayanan, S., Tiwari, A.: Output range analysis for deep feedforward neural networks. In: NASA Formal Methods Symposium. pp. 121–138. Springer (2018)

  19. [19]

    In: International Conference on Computer Aided Verification

    Eldib, H., Wang, C.: Synthesis of masking countermeasures against side channel attacks. In: International Conference on Computer Aided Verification. pp. 114–130. Springer (2014) Quantification of Leaks with NNs 17

  20. [20]

    ACM Transactions on Software Engineering and Methodology (TOSEM) 24(2), 11 (2014)

    Eldib, H., Wang, C., Schaumont, P.: Formal verification of software countermea- sures against side-channel attacks. ACM Transactions on Software Engineering and Methodology (TOSEM) 24(2), 11 (2014)

  21. [21]

    Constraints 23(3), 296–309 (2018)

    Fischetti, M., Jo, J.: Deep neural networks and mixed integer linear optimization. Constraints 23(3), 296–309 (2018)

  22. [22]

    In: Security and Privacy, 1982 IEEE Symposium on

    Goguen, J.A., Meseguer, J.: Security policies and security models. In: Security and Privacy, 1982 IEEE Symposium on. pp. 11–11. IEEE (1982)

  23. [23]

    In: FSE’07

    Goldsmith, S.F., Aiken, A.S., Wilkerson, D.S.: Measuring empirical computational complexity. In: FSE’07. pp. 395–404. ACM (2007)

  24. [24]

    In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

    Guo, S., Wu, M., Wang, C.: Adversarial symbolic execution for detecting concurrency-related cache timing leaks. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. pp. 377–388. ACM (2018)

  25. [25]

    gurobi.com

    Gurobi Optimization, L.: Gurobi optimizer reference manual (2018), http://www. gurobi.com

  26. [26]

    Neural Networks 2, 359–366 (1989)

    Hornik, K., Stinchcombe, M.B., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)

  27. [27]

    In: 2013 IEEE Symposium on Security and Privacy

    Hund, R., Willems, C., Holz, T.: Practical timing side channel attacks against kernel space aslr. In: 2013 IEEE Symposium on Security and Privacy. pp. 191–205. IEEE (2013)

  28. [28]

    In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security

    Kersten, R., Luckow, K., P˘ as˘ areanu, C.S.: Poster: Afl-based fuzzing for java with kelinci. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. pp. 2511–2513. ACM (2017)

  29. [29]

    Adam: A Method for Stochastic Optimization

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  30. [30]

    In: Annual International Cryptology Conference

    Kocher, P.C.: Timing attacks on implementations of diffie-hellman, rsa, dss, and other systems. In: Annual International Cryptology Conference. pp. 104–113. Springer (1996)

  31. [31]

    In: CCS’07

    K¨ opf, B., Basin, D.: An information-theoretic model for adaptive side-channel at- tacks. In: CCS’07. pp. 286–296 (2007)

  32. [32]

    In: CSF’09 (2009)

    K¨ opf, B., D¨ urmuth, M.: A provably secure and efficient countermeasure against timing attacks. In: CSF’09 (2009)

  33. [33]

    In: 2017 IEEE/ACM 39th Inter- national Conference on Software Engineering (ICSE)

    Landman, D., Serebrenik, A., Vinju, J.J.: Challenges for static analysis of java reflection-literature review and empirical study. In: 2017 IEEE/ACM 39th Inter- national Conference on Software Engineering (ICSE). pp. 507–518. IEEE (2017)

  34. [34]

    Online post at: https: //rdist.root.org/2009/05/28/timing-attack-in-google-keyczar-library/ (2009)

    Lawson, N.: Timing attack in google keyczar library. Online post at: https: //rdist.root.org/2009/05/28/timing-attack-in-google-keyczar-library/ (2009)

  35. [35]

    nature 521(7553), 436 (2015)

    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521(7553), 436 (2015)

  36. [36]

    VulDeePecker: A Deep Learning-Based System for Vulnerability Detection

    Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., Zhong, Y.: Vuldeepecker: A deep learning-based system for vulnerability detection. arXiv:1801.01681 (2018)

  37. [37]

    In: USENIX Security Symposium

    Livshits, V.B., Lam, M.S.: Finding security vulnerabilities in java applications with static analysis. In: USENIX Security Symposium. vol. 14, pp. 18–18 (2005)

  38. [38]

    In: Formal Techniques for Distributed Systems, pp

    Milushev, D., Beck, W., Clarke, D.: Noninterference via symbolic execution. In: Formal Techniques for Distributed Systems, pp. 152–168. Springer (2012)

  39. [39]

    Biometrika 78(3), 691–692 (1991)

    Nagelkerke, N.J., et al.: A note on a general definition of the coefficient of deter- mination. Biometrika 78(3), 691–692 (1991)

  40. [40]

    In: AAAI’18 (2018)

    Narodytska, N., Kasiviswanathan, S., Ryzhyk, L., Sagiv, M., Walsh, T.: Verifying properties of binarized deep neural networks. In: AAAI’18 (2018)

  41. [41]

    DifFuzz: Differential Fuzzing for Side-Channel Analysis

    Nilizadeh, S., Noller, Y., Pasareanu, C.S.: Diffuzz: Differential fuzzing for side- channel analysis. ICSE (2019), http://arxiv.org/abs/1811.07005 18 Tizpaz-Niari, et al

  42. [42]

    NDSS (2019)

    Rosner, N., Burak Kadron, I., Bang, L., Bultan, T.: Profit: Detecting and quanti- fying side channels in networked applications. NDSS (2019)

  43. [43]

    IEEE Jour- nal on selected areas in communications (2003)

    Sabelfeld, A., Myers, A.C.: Language-based information-flow security. IEEE Jour- nal on selected areas in communications (2003)

  44. [44]

    In: FoSSaCS’09 (2009)

    Smith, G.: On the foundations of quantitative information flow. In: FoSSaCS’09 (2009)

  45. [45]

    Sung, C., Paulsen, B., Wang, C.: Canal: A cache timing analysis framework via llvm transformation. pp. 904–907. ASE 2018

  46. [46]

    In: WINCOM’16 (2016)

    Tang, T.A., Mhamdi, L., McLernon, D., Zaidi, S.A.R., Ghogho, M.: Deep learn- ing approach for network intrusion detection in software defined networking. In: WINCOM’16 (2016)

  47. [47]

    Online: http://llvm

    libFuzzer team, G.: Libfuzzer: Coverage-based fuzz testing. Online: http://llvm. org/docs/LibFuzzer.html (2016)

  48. [48]

    In: Interna- tional Static Analysis Symposium

    Terauchi, T., Aiken, A.: Secure information flow as a safety problem. In: Interna- tional Static Analysis Symposium. pp. 352–367. Springer (2005)

  49. [49]

    In: TACAS’17

    Tizpaz-Niari, S., ˇCern´ y, P., Chang, B.Y.E., Sankaranarayanan, S., Trivedi, A.: Discriminating traces with time. In: TACAS’17. pp. 21–37. Springer (2017)

  50. [50]

    In: AAAI’18

    Tizpaz-Niari, S., ˇCern´ y, P., Chang, B.E., Trivedi, A.: Differential performance debugging with discriminant regression trees. In: AAAI’18. pp. 2468–2475 (2018)

  51. [51]

    arXiv preprint arXiv:1808.10502 (2018)

    Tizpaz-Niari, S., ˇCern´ y, P., Trivedi, A.: Data-driven debugging for functional side channels. arXiv:1808.10502 (2018)

  52. [52]

    In: Computer Aided Verification (CAV)

    Tizpaz-Niari, S., ˇCern´ y, P., Trivedi, A.: Quantitative mitigation of timing side channels. In: Computer Aided Verification (CAV). pp. 140–160 (2019)

  53. [53]

    Mitigating Power Side Channels during Compilation

    Wang, J., Sung, C., Wang, C.: Mitigating power side channels during compilation. arXiv preprint arXiv:1902.09099 (2019)

  54. [54]

    In: 26th USENIX Security Symposium

    Wang, S., Wang, P., Liu, X., Zhang, D., Wu, D.: Cached: Identifying cache-based timing channels in production software. In: 26th USENIX Security Symposium. pp. 235–252 (2017)

  55. [55]

    In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

    Wu, M., Guo, S., Schaumont, P., Wang, C.: Eliminating timing side-channel leaks using program repair. In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 15–26. ACM (2018)

  56. [56]

    In: Pro- ceedings of the 17th ACM conference on Computer and communications security

    Zhang, K., Li, Z., Wang, R., Wang, X., Chen, S.: Sidebuster: automated detection and quantification of side-channel leaks in web application development. In: Pro- ceedings of the 17th ACM conference on Computer and communications security. pp. 595–606. ACM (2010)