Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Monitoring Robustness and Individual Fairness

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that runtime input-output robustness of a deployed black-box classifier reduces to online fixed-radius nearest neighbor search, and that exact monitors can report every similar past input with a dissimilar output.

desk verdict Useful runtime-monitoring framework with a genuinely new BDD-based FRNN index and an effective L-infinity parallelization, but the formal reduction to FRNN is not a metric reduction and Algorithm 3 misses the output-dissimilarity filter, so the "always correct by design" claim is currently unsupported. read the letter →

arxiv 2506.00496 v1 pith:ZB3X3CXB submitted 2025-05-31 cs.AI

Ashutosh Gupta , Thomas A. Henzinger , Konstantin Kueffner , Kaushik Mallik , David Pape This is my paper
classification cs.AI
keywords runtimemonitoringinput-outputrobustnessindividualfairnessadversarialsemanticfixed-radiusnearestneighborsearchbinarydecisiondiagramsblack-boxAImodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces runtime input-output robustness: a property of a specific decision sequence rather than of the classifier, requiring that any two inputs seen so far that are within $\epsilon_X$ in input distance have outputs within $\delta_Z$. It claims that this runtime property can be monitored online by reducing the problem to fixed-radius nearest neighbor (FRNN) search over input-output pairs, and it presents Clemont, a suite of exact monitors: brute force, periodically re-indexed $k$-d trees and sorting-based neighbors, a binary-decision-diagram-based method for $L_\infty$, and a parallelized $L_\infty$ wrapper. On standard robustness and fairness benchmarks, the monitors detect violations that even robustly trained models still exhibit, processing each decision in fractions of a second to a few seconds and scaling beyond 100,000 feature dimensions and 350 million parameters. This matters because deployed black-box models can be watched continuously, with every violation reported together with a witness set of past similar inputs, complementing offline verification and robust training.

What carries the argument

The load-bearing object is the reduction of the monitoring problem to an online FRNN problem, carried by the product space $Q = X \times Z$ and a distance $d_Q((x,z),(x',z'))$ that is $d_X(x,x')$ when the output-distance condition holds and infinity otherwise, so that the output filter is absorbed into the metric and the neighbor search alone identifies violations. The implementations rely on three mechanisms: periodic re-indexing of static FRNN structures with a short-term brute-force buffer; a BDD-based index that discretizes each dimension into intervals of width $\epsilon_Q$, encodes seen labels and label adjacency symbolically, and uses brute force only to resolve false positives among adjacent-label candidates; and a parallelization wrapper for $L_\infty$ that splits coordinates into blocks, runs FRNN per block, and composes results by matching unique point identifiers.

What would settle it

Evaluate the function $d_Q$ defined in Section 3.4 on the same pair of input-output points while varying which point is called the new input: if the value changes because $z_{n+1}$ and $z_i$ appear in the condition, then $d_Q$ is not a distance and the formal claim that monitoring is FRNN in a metric space fails. Separately, run Clemont on a hand-built classifier with a planted pair of inputs that are within $\epsilon_X$ in input distance but produce outputs more than $\delta_Z$ apart, with the second input arriving after the first; the monitor must output that pair in its witness set or the exactness claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that monitoring runtime $(\epsilon_X, \delta_Z)$-input-output robustness of a black-box classifier reduces at each step to an online FRNN query over the product set $Q = X \times Z$, where the distance $d_Q$ equals $d_X$ when the outputs are at least $\delta_Z$ apart and infinity otherwise, and where the FRNN radius is $\epsilon_Q = \epsilon_X$. Under this reduction, any point returned by the exact FRNN query is precisely a past decision that is input-similar and output-dissimilar, so the monitor returns a complete witness set by design. The paper further claims that static FRNN algorithms transfer to the online setting through periodic index recomputation, that a new binary-decision-diagram-based algorithm plus a dimension-splitting parallelization for the $L_\infty$ norm make the monitors practical, and that on adversarial robustness, semantic robustness, and individual fairness benchmarks the monitors report positive violation rates even for models trained with robust or fair algorithms, at speeds fast enough for online use.

Load-bearing premise

The whole approach depends on the monitor retrieving every past input within $\epsilon_X$ of the new input (or a full superset of them) before filtering on output distance; if an index returns fewer, a violation can be missed, and the paper's formal reduction is expressed through a query-dependent function that is not actually a metric.

Editorial extensions

If this is right

  • Every deployed classifier can be watched online: after each decision, the monitor returns the exact set of past inputs within $\epsilon_X$ whose outputs differed by more than $\delta_Z$, so violations are not missed if the underlying FRNN query is exact.
  • A decision sequence that is runtime robust does not imply the classifier is globally robust; runtime monitoring is a weaker, deployment-focused guarantee, and offline robust training reduces but does not eliminate runtime violations.
  • Because monitors treat the model as a black box, their runtime cost depends on input dimension and history length rather than model size, making the approach feasible for networks beyond the reach of static verification.
  • Once a violation appears in a decision sequence, every future extension of that sequence is also a violation, so alarms are persistent and the witness set remains available for human scrutiny.
  • The parallelized $L_\infty$ wrapper can be applied on top of any exact FRNN monitor, splitting the search by coordinate blocks and composing the per-block results by unique identifiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader could extend the FRNN casting to approximate FRNN algorithms, which the paper mentions but does not adopt; those would trade exact witness sets for speed and introduce a tunable false-negative rate, possibly acceptable when worst-case completeness is not required.
  • The same monitoring scheme should transfer to regression and other continuous-output models once a concrete output metric $d_Z$ is chosen, since the formulation is not classification-specific even though the paper demonstrates only classifiers.
  • The BDD discretization's exactness is tied to the $L_\infty$ norm; for Euclidean or other metrics, same- or adjacent-label intervals do not bound distances in the same way, so a BDD-style filter would need enlarged label neighborhoods or an exact pre-filter to avoid missing violations.
  • A direct end-to-end check is to run the monitors on a hand-built classifier with a planted near-robustness-boundary pair; the violation rate should be zero on a genuinely robust sequence and positive on the planted sequence, testing the full pipeline from discretization to witness output.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces runtime input-output robustness (i.o.r.) for black-box classifiers: a decision sequence (x1,z1)...(xn,zn) violates runtime (ε_X,δ_Z)-i.o.r. if there are indices i,j with d_X(x_i,x_j)≤ε_X and d_Z(D(x_i),D(x_j))>δ_Z. It defines the monitoring problem as reporting, after each new decision, the set of all past decisions that witness such a violation. It then claims that this problem reduces to online fixed-radius nearest neighbor (FRNN) search, and uses that reduction to design and implement monitors in a tool called Clemont: a brute-force monitor, monitors that periodically rebuild k-d tree and SNN indexes, a novel BDD-based monitor, and a parallelization wrapper for L∞ distances. The experimental section evaluates violation rates and per-decision time/memory on adversarial robustness, semantic robustness, and individual fairness benchmarks, including ImageNet-scale inputs and models with hundreds of millions of parameters.

Significance. The paper addresses a timely and practically relevant question: can a deployed black-box model be watched online for robustness/fairness violations, with a witness set for each violation? The unification of adversarial robustness, semantic robustness, and individual fairness under one runtime property is useful, and the idea of reusing static FRNN indexes via periodic rebuilding and of using BDDs for L∞ range search is original. The tool is open-source and the experimental comparison covers standard benchmarks and large-scale models, which is valuable for practitioners. The paper also honestly identifies limits of its approach, such as the restriction of the BDD monitor to L∞ and the possibility of using approximate FRNN algorithms. However, the formal reduction that underpins the exactness claim is currently not valid as written, and the BDD pseudocode contains a case that produces false positives. These issues are local and repairable, but until they are fixed the central 'always correct by design' claim is not supported.

major comments (2)
  1. [Section 3.4] The reduction of Problem 1 to Problem 2 is not established because the displayed definition of d_Q uses the undefined indices z_{n+1} and z_i on the right-hand side. These indices refer to the current query and to a past point, not to the two arguments (x,z) and (x',z') of d_Q, so the expression is not a function from Q×Q to R_{\ge 0}. Even after replacing them with z and z', the definition has two further metricity problems: it uses d_Z ≥ δ_Z whereas Problem 1 requires d_Z > δ_Z, and d_Q((x,z),(x,z)) = ∞ because d_Z(z,z)=0 < δ_Z, violating the identity axiom. Consequently the statement in the Abstract and Section 1 that the monitoring problem 'can be cast as' FRNN is not supported by the formal apparatus. Please either give a genuine metric on X×Z that exactly encodes the violation condition, or present the monitors as performing FRNN on X followed by the output filter d_Z > δ_Z and prove correctness for that two-stage procedure.
  2. [Appendix A, Algorithm 3 (lines 16–17)] In case (b), when the query's discretized label has appeared before and no neighboring label has, the algorithm sets R←Δ(q) and outputs it without testing the output condition d_Z(z_new, z_i)>δ_Z. A past decision with the same discretized input label is within ε_Q in the L∞ metric, but if its output is within δ_Z of the new output it is not a runtime i.o.r. violation under Definition 3.2 and Problem 1. Reporting it contradicts the claim in Section 8.1 that 'our monitors always output correct answers by design.' The fix is to filter Δ(q) by the output condition before outputting, or to send case (b) through the same bottom-level brute-force check as case (c). Because the violation-rate numbers in Table 1 for the BDD monitor are affected by such false positives, the BDD rows should be re-evaluated after the correction.
minor comments (5)
  1. [Appendix B] In the proof of Theorem 3.3(2), the expression 'd_x(x,x')≤ε_X\{x_1,...,x_n\}' is malformed; the intended statement is that x,x' are chosen from X outside the finite set {x_1,...,x_n} with d_X(x,x')≤ε_X. Please also state explicitly that the modified classifier D' agrees with D on the runtime sequence.
  2. [Sections 6 and 9] There are two typos in the text: 'The ides is' should be 'The idea is' in Section 6, and the heading 'Discusssions' in Section 9 is misspelled.
  3. [References] Reference entries [22] and [23] are identical, both citing the M-tree paper by Ciaccia et al.; please deduplicate them.
  4. [Table 1] For the individual fairness rows, the alignment of the two model-source columns ([67] and [49]) with the Violations (%) columns is not self-explanatory; please clarify the table header so that it is clear which violation rate belongs to which model.
  5. [Algorithm 2] Line 4 reuses the name S for the augmented set S⊂R^{2n}×N, which conflicts with the input set also called S in the surrounding explanation; please use a different name for one of the two sets.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the monitoring problem and the monitors are defined directly from the target violation predicate, and the reported violation rates come from external benchmark runs rather than from any fitted constant or self-referential construction.

full rationale

The paper defines runtime input-output robustness (Definition 3.2) and the monitoring function M (Problem 1) as the set of past decisions satisfying d_X(x_{n+1}, x_i) <= epsilon_X and d_Z(z_{n+1}, z_i) > delta_Z. The claimed reduction to FRNN (Section 3.4) is a reformulation: it sets Q = X x Z, epsilon_Q = epsilon_X, and defines d_Q to be d_X when the output-distance predicate holds and infinity otherwise. Although the displayed definition has a typo (it writes z_{n+1} and z_i instead of the arguments z and z') and, as written, does not define a metric, this is a correctness gap in the reduction, not a circularity: the target predicate is not used to fit any parameter, and no prediction is derived from the problem statement itself. The monitors in Algorithms 1-3 are exact procedures built on the same predicate; even if Algorithm 3's same-label shortcut can skip the d_Z > delta_Z check in the branch R <- Delta(q), that would be an algorithmic soundness bug, not an equivalence-by-construction. The experimental violation rates are measured against external RobustBench and fairness benchmarks, and the paper does not fit parameters and then 'predict' those same rates. The related-work citations to the authors' earlier monitoring papers are not load-bearing for the central reduction. Thus, while the paper may contain technical flaws in the precise statement and implementation, they are not circularity flaws.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim introduces no new physical or mathematical entities and fits no scientific constants. The only free parameters are engineering hyperparameters (reindex interval, thread count) of the monitoring algorithms.

free parameters (2)
  • tau (periodic reindex interval) = not reported
    Algorithm 1 rebuilds the static FRNN index every tau new inputs. The paper gives a cost model T(tau) but does not report the tau values used in the experiments.
  • number of parallel threads = 1, 2, 16, 96
    The parallelized FRNN wrapper in Algorithm 2 requires choosing the number of workers; experiments show a sweet spot but no selection rule is given.
assumptions (3)
  • domain assumption Each real-valued feature dimension has a bounded range so the BDD discretization into finitely many intervals of width epsilon_Q is possible.
    Section 6 states 'Assuming that the real dimensions have a bounded range'. Unbounded features would require an infinite number of labels.
  • domain assumption The input and output distances d_X and d_Z are computable for every observed pair, and the monitor sees the full, uncorrupted decision sequence.
    The monitor's exactness relies on evaluating distances and on observing every decision; partial observations or noisy features are not handled.
  • domain assumption For the BDD and parallelized monitors, the input distance is the infinity norm.
    Both the same/adjacent-label indexing and the dimensionwise parallel composition rely on infinity-norm geometry; the paper explicitly limits these monitors to that norm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Monitoring Robustness and Individual Fairness." pith.science (2026). https://pith.science/paper/ZB3X3CXB

@misc{pith2026250600496,
  author       = {Pith},
  title        = {Pith review of: Monitoring Robustness and Individual Fairness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZB3X3CXB}},
  note         = {Machine review of arXiv:2506.00496}
}
abstract

Input-output robustness appears in various different forms in the literature, such as robustness of AI models to adversarial or semantic perturbations and individual fairness of AI models that make decisions about humans. We propose runtime monitoring of input-output robustness of deployed, black-box AI models, where the goal is to design monitors that would observe one long execution sequence of the model, and would raise an alarm whenever it is detected that two similar inputs from the past led to dissimilar outputs. This way, monitoring will complement existing offline ``robustification'' approaches to increase the trustworthiness of AI decision-makers. We show that the monitoring problem can be cast as the fixed-radius nearest neighbor (FRNN) search problem, which, despite being well-studied, lacks suitable online solutions. We present our tool Clemont, which offers a number of lightweight monitors, some of which use upgraded online variants of existing FRNN algorithms, and one uses a novel algorithm based on binary decision diagrams -- a data-structure commonly used in software and hardware verification. We have also developed an efficient parallelization technique that can substantially cut down the computation time of monitors for which the distance between input-output pairs is measured using the $L_\infty$ norm. Using standard benchmarks from the literature of adversarial and semantic robustness and individual fairness, we perform a comparative study of different monitors in \tool, and demonstrate their effectiveness in correctly detecting robustness violations at runtime.

Figures

Figures reproduced from arXiv: 2506.00496 by the authors.

Figure 1
Figure 1. Schematic diagram of input-output robustness mon [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Dis￾cretization of 𝑄 = R 2 . The points have neigh￾boring label, but are 𝜖𝑄 apart. Suppose 𝑄 has some real and some categorical dimensions. Assuming that the real dimensions have a bounded range, we will discretize the real di￾mensions into finitely many (non￾overlapping) intervals of width 𝜖𝑄 . Each interval has a representative point as its label, and every other points inside this interval is “approximated” by th… view at source ↗
Figure 3
Figure 3. Par￾allelized FRNN monitoring. The parallelized algorithm is pre￾sented in Algorithm 2, and we explain it using a simple example in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Semantic robustness violations identified during [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: LEFT: Performance comparison on the HIGGS dataset with 10 million entries. The rows correspond to 12 and 24 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Sample of flagged inputs for semantic robustness on CIFAR-10C, Frost corruption, severity 2, baseline model [26]. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Flagged inputs for semantic robustness on Imagenet-3DCC, Fog corruption, severity 2, top model [70]. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Monitoring of Static Fairness

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Finite-sample PAC monitors for fairness of unknown Markov-chain decision makers, with pointwise and time-uniform soundness guarantees.

Reference graph

Works this paper leans on

88 extracted references · 59 canonical work pages · cited by 1 Pith paper

  1. [1]

    Aws Albarghouthi and Samuel Vinitsky. 2019. Fairness-aware programming. InProceedings of the Conference on Fairness, Accountability, and Transparency. 211–219

    2019

  2. [2]

    Sajjad Amini, Mohammadreza Teymoorianfard, Shiqing Ma, and Amir Houmansadr. 2024. MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification.arXiv preprint arXiv:2406.05927 (2024)

    arXiv 2024

  3. [3]

    Alexandr Andoni and Piotr Indyk. 2008. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions.Commun. ACM51, 1 (2008), 117–122

    2008

  4. [4]

    Alexandr Andoni and Piotr Indyk. 2017. Nearest neighbors in high-dimensional spaces. InHandbook of Discrete and Computational Geometry. Chapman and Hall/CRC, 1135–1155

    2017

  5. [5]

    Alexandr Andoni, Piotr Indyk, Huy L Nguyen, and Ilya Razenshteyn. 2014. Be- yond locality-sensitive hashing. InProceedings of the twenty-fifth annual ACM- SIAM symposium on Discrete algorithms. SIAM, 1018–1028

    2014

  6. [6]

    Alexandr Andoni, Piotr Indyk, and Ilya Razenshteyn. 2018. Approximate nearest neighbor search in high dimensions. InProceedings of the International Congress of Mathematicians: Rio de Janeiro 2018. World Scientific, 3287–3318

    2018

  7. [7]

    Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. 2021. Recent Advances in Adversarial Training for Adversarial Robustness. (2021)

    2021

  8. [8]

    2018.Lectures on runtime verification

    Ezio Bartocci and Yliès Falcone. 2018.Lectures on runtime verification. Springer

    2018

Show all 88 references
  1. [9]

    Brian R Bartoldson, James Diffenderfer, Konstantinos Parasyris, and Bhavya Kailkhura. 2024. Adversarial Robustness Limits via Scaling-Law and Human- Alignment Studies.arXiv preprint arXiv:2404.09349(2024). Monitoring Robustness and Individual Fairness KDD ’25, August 3–7, 2025...

    2024 arXiv

  2. [10]

    Jon Louis Bentley. 1975. Multidimensional binary search trees used for associative searching.Commun. ACM18, 9 (1975), 509–517

    1975

  3. [11]

    Elias Benussi, Andrea Patane, Matthew Wicker, Luca Laurenti, and Marta Kwiatkowska. 2022. Individual Fairness Guarantees for Neural Networks. In31st International Joint Conference on Artificial Intelligence, IJCAI 2022. International Joint Conferences on Artificial Intelligenc...

    2022

  4. [12]

    Alina Beygelzimer, Sham Kakade, and John Langford. 2006. Cover trees for nearest neighbor. InProceedings of the 23rd international conference on Machine learning. 97–104

    2006

  5. [13]

    Sumon Biswas and Hridesh Rajan. 2023. Fairify: Fairness verification of neural networks. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1546–1558

    2023

  6. [14]

    Guy E Blelloch and Magdalen Dobson. 2022. Parallel Nearest Neighbors in Low Dimensions with Batch Updates. In2022 Proceedings of the Symposium on Algorithm Engineering and Experiments (ALENEX). SIAM, 195–208

    2022

  7. [15]

    Randal E Bryant. 2018. Binary decision diagrams.Handbook of model checking (2018), 191–217

    2018

  8. [16]

    Yu Cao, Xiaojiang Zhang, Boheng Duan, Wenjing Zhao, and Huizan Wang. 2020. An improved method to build the KD tree based on presorted results. In2020 IEEE 11th International Conference on Software Engineering and Service Science (ICSESS). IEEE, 71–75

    2020

  9. [17]

    Marco Casadio, Ekaterina Komendantskaya, Matthew L Daggitt, Wen Kokke, Guy Katz, Guy Amir, and Idan Refaeli. 2022. Neural network robustness as a verification property: a principled case study. InInternational conference on computer aided verification. Springer, 219–231

    2022

  10. [18]

    Martín Chalela, Emanuel Sillero, Luis Pereyra, Mario Alejandro García, Juan B Cabral, Marcelo Lares, and Manuel Merchán. 2021. Grispy: A python package for fixed-radius nearest neighbors search.Astronomy and Computing34 (2021), 100443. KDD ’25, August 3–7, 2025, Toronto, ON, C...

    2021

  11. [19]

    Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. 2024. End-to-end autonomous driving: Challenges and frontiers. IEEE Transactions on Pattern Analysis and Machine Intelligence(2024)

    2024

  12. [20]

    Xinye Chen and Stefan Güttel. 2024. Fast and exact fixed-radius neighbor search based on sorting.PeerJ Computer Science10 (2024), e1929

    2024

  13. [21]

    Byn Choi, Rakesh Komuravelli, Victor Lu, Hyojin Sung, Robert L Bocchino Jr, Sarita V Adve, and John C Hart. 2010. Parallel SAH kD tree construction.. In High performance graphics. Citeseer, 77–86

    2010

  14. [23]

    Paolo Ciaccia, Marco Patella, Pavel Zezula, et al. 1997. M-tree: An efficient access method for similarity search in metric spaces. InVldb, Vol. 97. Citeseer, 426–435

    1997

  15. [24]

    Kenneth L Clarkson. 1997. Nearest neighbor queries in metric spaces. InPro- ceedings of the twenty-ninth annual ACM symposium on Theory of computing. 609–617

    1997

  16. [25]

    Michael Connor and Piyush Kumar. 2010. Fast construction of k-nearest neighbor graphs for point clouds.IEEE transactions on visualization and computer graphics 16, 4 (2010), 599–608

    2010

  17. [26]

    36M 0.948 [9] 366M 0.196 3.71 6.09 1.87 2191 2852 1946 CIFAR-100 [66] 11M 0.344 [74] 267M 0.316 5.47 32.04 9.73 2057 2878 1945 ImageNet10𝑘150.5𝑘 4 255 [26] 26M 0.767 [2] 198M 0.186 0.26s 7.73s 0.62s 64GB 75GB 65GB BF𝑘-d SNN BF𝑘-d SNN Sem. Robust. CIFAR-10-C 20𝑘 384∗ 𝐿2 7.5 [26...

    1946

  18. [27]

    Sanjoy Dasgupta and Kaushik Sinha. 2013. Randomized partition trees for exact nearest neighbor search. InConference on learning theory. PMLR, 317–337

    2013

  19. [28]

    Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. 2020. RobustBench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670(2020). Accessed: 2024-12-01

    2020 arXiv

  20. [29]

    268M Fig 4 2.36 83.75 12.75 388 519 573 CIFAR-100-C [37] 267M [29] 269M 4.53 59.26 37.80 396 519 573 ImageNet10𝑘 12.5 [50] 61M [70] 86M 32.78 0.2s 60.25 376 373 401 (∗)from DINOv2[62] embedding [67] [49] [67] [49] [67] [49] [67] [49] BF𝑘-d BDD BF𝑘-d BDD Ind. Fair. German1𝑘31 𝐿...

  21. [30]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition. Ieee, 248–255

    2009

  22. [31]

    James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. 2021. A winning hand: Compressing deep networks can improve out-of-distribution robustness.Advances in neural information processing systems 34 (2021), 664–676

    2021

  23. [32]

    Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou. 2024. The faiss library.arXiv preprint arXiv:2401.08281(2024)

    2024 arXiv

  24. [33]

    Laurent Doyen, Thomas A Henzinger, Axel Legay, and Dejan Nickovic. 2010. Robustness of sequential circuits. In2010 10th International Conference on Appli- cation of Concurrency to System Design. IEEE, 77–84

    2010

  25. [34]

    Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. InProceedings of the 3rd innovations in theoretical computer science conference. 214–226

    2012

  26. [35]

    Sepehr Eghbali, Hassan Ashtiani, and Ladan Tahvildari. 2019. Online nearest neighbor search using hamming weight trees.IEEE Transactions on Pattern Analysis and Machine Intelligence42, 7 (2019), 1729–1740

    2019

  27. [36]

    Yizhak Yisrael Elboher, Justin Gottschlich, and Guy Katz. 2020. An abstraction- based framework for neural network verification. InComputer Aided Verification: 32nd International Conference, CA V 2020, Los Angeles, CA, USA, July 21–24, 2020, Proceedings, Part I 32. Springer, 43–65

    2020

  28. [37]

    Ada Wai-chee Fu, Polly Mei-shuen Chan, Yin-Ling Cheung, and Yiu Sang Moon

  29. [38]

    Antonin Guttman. 1984. R-trees: A dynamic index structure for spatial searching. InProceedings of the 1984 ACM SIGMOD international conference on Management of data. 47–57

    1984

  30. [39]

    Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. 2018. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In2018 IEEE symposium on security and privacy (SP). IEEE, 3–18

    2018

  31. [40]

    Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli

  32. [41]

    Thomas A Henzinger, Konstantin Kueffner, and Kaushik Mallik. 2023. Monitoring algorithmic fairness under partial observations. InInternational Conference on Runtime Verification. Springer, 291–311

    2023

  33. [42]

    Thomas A Henzinger, Jan Otop, and Roopsha Samanta. 2014. Lipschitz Robustness of Finite-state Transducers. In34th International Conference on Foundation of Software Technology and Theoretical Computer Science (FSTTCS 2014). Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 431–443

    2014

  34. [43]

    Thomas Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik

  35. [44]

    Anan Kabaha and Dana Drachsler Cohen. 2024. Verification of Neural Net- works’ Global Robustness.Proceedings of the ACM on Programming Languages8, OOPSLA1 (2024), 1010–1039

    2024

  36. [45]

    Thomas A Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik

  37. [46]

    InInternational Conference on Computer Aided Verification

    Monitoring algorithmic fairness. InInternational Conference on Computer Aided Verification. Springer, 358–382

  38. [47]

    Guy Katz, Derek A Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, et al

  39. [48]

    Orçun Kaya, Jan Schildbach, Deutsche Bank AG, and Stefan Schneider. 2019. Artificial intelligence in banking.Artificial intelligence(2019)

    2019

  40. [49]

    Philips George John, Deepak Vijaykeerthy, and Diptikalyan Saha. 2020. Verifying individual fairness in machine learning models. InConference on Uncertainty in Artificial Intelligence. PMLR, 749–758

    2020

  41. [50]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classifi- cation with deep convolutional neural networks.Advances in neural information processing systems25 (2012)

    2012

  42. [51]

    Ibrahim Kamel and Christos Faloutsos. 1992. Parallel R-trees.ACM SIGMOD Record21, 2 (1992), 195–204

    1992

  43. [52]

    Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer

  44. [53]

    Lan Li, Tina Lassiter, Joohee Oh, and Min Kyung Lee. 2021. Algorithmic hiring in practice: Recruiter and HR Professional’s perspectives on AI use in hiring. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 166–176

    2021

  45. [54]

    Yi Lin and Yongho Jeon. 2006. Random forests and adaptive nearest neighbors.J. Amer. Statist. Assoc.101, 474 (2006), 578–590

    2006

  46. [55]

    Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.IEEE transactions on pattern analysis and machine intelligence42, 4 (2018), 824–836

    2018

  47. [56]

    Durga Keerthi Mandarapu, Vani Nagarajan, Artem Pelenitsyn, and Milind Kulka- rni. 2024. Arkade: k-Nearest Neighbor Search With Non-Euclidean Distances using GPU Ray Tracing. InProceedings of the 38th ACM International Conference on Supercomputing. 14–25

    2024

  48. [57]

    Haitham Khedr and Yasser Shoukry. 2023. Certifair: A framework for certified global fairness of neural networks. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8237–8245

    2023

  49. [58]

    Ziyang Men, Zheqi Shen, Yan Gu, and Yihan Sun. 2024. Pkd-tree: Parallel𝑘 d-tree with Batch Updates.arXiv preprint arXiv:2411.09275(2024)

    2024 arXiv

  50. [59]

    Preethi Lahoti, Krishna P Gummadi, and Gerhard Weikum. 2019. ifair: Learning individually fair data representations for algorithmic decision making. In2019 ieee 35th international conference on data engineering (icde). IEEE, 1334–1345

    2019

  51. [60]

    Klas Leino, Zifan Wang, and Matt Fredrikson. 2021. Globally-robust neural networks. InInternational Conference on Machine Learning. PMLR, 6212–6222

    2021

  52. [61]

    Stephen M Omohundro. 1989. Five balltree construction algorithms. (1989)

    1989

  53. [62]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El- Nouby, et al. 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193(2023)

    2023 arXiv

  54. [63]

    Sushil K Prasad, Michael McDermott, Xi He, and Satish Puri. 2015. GPU-based Parallel R-tree Construction and Querying. In2015 IEEE International Parallel and Distributed Processing Symposium Workshop. IEEE, 618–627

    2015

  55. [64]

    Mahfuzur Rahman, Teoh Hui Ming, Tarannum Azim Baigh, and Moniruzzaman Sarker. 2023. Adoption of artificial intelligence in banking services: an empirical analysis.International Journal of Emerging Markets18, 10 (2023), 4270–4300

    2023

  56. [65]

    Ravi Mangal, Aditya V Nori, and Alessandro Orso. 2019. Robustness of neural networks: A probabilistic and practical approach. In2019 IEEE/ACM 41st Inter- national Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE, 93–96

    2019

  57. [66]

    Leslie Rice, Eric Wong, and Zico Kolter. 2020. Overfitting in adversarially robust deep learning. InInternational conference on machine learning. PMLR, 8093–8104

    2020

  58. [67]

    Mark Huasong Meng, Guangdong Bai, Sin Gee Teo, Zhe Hou, Yan Xiao, Yun Lin, and Jin Song Dong. 2022. Adversarial robustness of deep neural networks: A survey from a formal verification perspective.IEEE Transactions on Dependable and Secure Computing(2022)

    2022

  59. [68]

    Vani Nagarajan, Durga Mandarapu, and Milind Kulkarni. 2023. Rt-knns unbound: Using rt cores to accelerate unrestricted neighbor search. InProceedings of the 37th International Conference on Supercomputing. 289–300

    2023

  60. [69]

    Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification.Advances in Neural Information Processing Systems33 (2020), 18583–18599

    2020

  61. [70]

    Rui Tian, Zuxuan Wu, Qi Dai, Han Hu, and Yu-Gang Jiang. 2022. Deeper In- sights into the Robustness of ViTs towards Common Corruptions.arXiv preprint Monitoring Robustness and Individual Fairness KDD ’25, August 3–7, 2025, Toronto, ON, Canada arXiv:2204.12143(2022)

    2022 arXiv

  62. [71]

    Vincent Tjeng, Kai Y Xiao, and Russ Tedrake. 2017. Evaluating Robustness of Neural Networks with Mixed Integer Programming. InInternational Conference on Learning Representations

    2017

  63. [72]

    Caterina Urban, Maria Christakis, Valentin Wüstholz, and Fuyuan Zhang. 2020. Perfectly parallel fairness certification of neural networks.Proceedings of the ACM on Programming Languages4, OOPSLA (2020), 1–30

    2020

  64. [73]

    Parikshit Ram and Kaushik Sinha. 2019. Revisiting kd-tree for nearest neighbor search. InProceedings of the 25th acm sigkdd international conference on knowledge discovery & data mining. 1378–1388

    2019

  65. [74]

    Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan

  66. [75]

    Anian Ruoss, Mislav Balunovic, Marc Fischer, and Martin Vechev. 2020. Learn- ing certified individually fair representations.Advances in neural information processing systems33 (2020), 7584–7596

    2020

  67. [76]

    Alex Serban, Erik Poll, and Joost Visser. 2020. Adversarial examples on object recognition: A comprehensive survey.ACM Computing Surveys (CSUR)53, 3 (2020), 1–38

    2020

  68. [77]

    Rahul Yesantharao, Yiqiu Wang, Laxman Dhulipala, and Julian Shun. 2021. Paral- lel Batch-Dynamic𝑘d-Trees.arXiv preprint arXiv:2112.06188(2021)

    2021 arXiv

  69. [78]

    Peter N Yianilos. 1993. Data structures and algorithms for nearest neighbor search in general metric spaces. InSoda, Vol. 93. 311–21

    1993

  70. [79]

    Simin You, Jianting Zhang, and Le Gruenwald. 2013. Parallel spatial query processing on gpus using r-trees. InProceedings of the 2Nd ACM SIGSPATIAL international workshop on analytics for big geospatial data. 23–31

    2013

  71. [80]

    Ekim Yurtsever, Jacob Lambert, Alexander Carballo, and Kazuya Takeda. 2020. A survey of autonomous driving: Common practices and emerging technologies. IEEE access8 (2020), 58443–58469. KDD ’25, August 3–7, 2025, Toronto, ON, Canada Ashutosh Gupta, Thomas A. Henzinger, Konstan...

    2020

  72. [81]

    Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter. 2021. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification.Advances in Neural Information Processing Systems34 (2021), 29909–29921

    2021

  73. [83]

    InInternational Conference on Machine Learning

    Better diffusion models further improve adversarial training. InInternational Conference on Machine Learning. PMLR, 36246–36263

  74. [84]

    Daniel Whiteson. 2014. HIGGS. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5V312

    2014 doi

  75. [85]

    Samuel Yeom and Matt Fredrikson. 2020. Individual Fairness Revisited: Trans- ferring Techniques from Adversarial Robustness. InTwenty-Ninth International Joint Conference on Artificial Intelligence

    2020

  76. [2000]

    Dynamic vp-tree indexing for n-nearest neighbor search given pair-wise distances.The VLDB Journal9 (2000), 154–173

    2000

  77. [2017]

    In Computer Aided Verification: 29th International Conference, CA V 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30

    Reluplex: An efficient SMT solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CA V 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30. Springer, 97–117

    2017

  78. [2019]

    InComputer Aided Verification: 31st International Conference, CA V 2019, New York City, NY, USA, July 15-18, 2019, Proceedings, Part I 31

    The marabou framework for verification and analysis of deep neural networks. InComputer Aided Verification: 31st International Conference, CA V 2019, New York City, NY, USA, July 15-18, 2019, Proceedings, Part I 31. Springer, 443–452

    2019

  79. [2020]

    Uncovering the limits of adversarial training against norm-bounded adver- sarial examples.arXiv preprint arXiv:2010.03593(2020)

    2020 arXiv

  80. [2023]

    InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency

    Runtime monitoring of dynamic fairness properties. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 604–614

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.