Pith. sign in

REVIEW 5 major objections 4 minor 89 references

Leveraging Data Characteristics for Bug Localization in Deep Learning Programs

T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Combining dataset characteristics with source code lets Theia localize 57 of 75 deep-learning structural bugs, versus 17 for the leading pre-training checker.

desk verdict Theia's dataset-aware static bug detection idea is worth a look, but the 57/75 headline doesn't survive the paper's own tables: TP counts are internally inconsistent and the denominator includes unsupported bug types. read the letter →

arxiv 2412.05775 v1 pith:JGM6EQVB submitted 2024-12-08 cs.SE

Ruchira Manke , Mohammad Wardat , Foutse Khomh , Hridesh Rajan This is my paper · ORCID
classification cs.SE
keywords deeplearningbugsbuglocalizationstructuralstaticanalysistrainingdatacharacteristicsverificationrulesKerasPyTorch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep-learning programs often fail silently when their model structure is wrong for the data they train on, for example too few convolution layers for color images, a softmax head on a regression task, or a learning rate far outside a useful range. Such structural bugs rarely crash, so they surface only as low or stuck accuracy after expensive training. Theia tries to catch them at the start of training by checking the model's parsed configuration against characteristics of the training dataset, including image type, number of classes, problem type, input range, and dataset size. On 40 real-world buggy programs, Theia localizes 57 of 75 structural bugs, where the leading source-code-only pre-training approach localizes 17. The paper's claim, read sympathetically, is that dataset-aware static analysis is the missing ingredient for pre-training bug localization in deep-learning programs.

What carries the argument

The load-bearing mechanism is Theia's meta-model, a library-independent representation that captures dataset characteristics alongside model properties extracted through Keras's get_config() and PyTorch's modules() APIs. Call-string analysis and parameter-sensitive analysis walk the meta-model and apply 12 verification rules with fixed thresholds, such as requiring at least three convolution layers for color images, keeping convolution filters between 16 and 512 for color inputs, learning rate in [0.0001, 0.01], and batch size in [32, 256]. The rules were derived from 105 developer Q&A discussions and design literature, then spot-checked on a 100-program fault benchmark.

What would settle it

Run Theia on a held-out set of, say, 100 known-good Keras and PyTorch models that train to high accuracy and count how many trigger at least one of the 12 rules; if a substantial share of healthy models violate the rules, the thresholds are over-broad and the reported 57/75 detection rate does not demonstrate that the tool separates buggy from healthy structure.

Watch

Extended reading notes

Core claim

The central claim is that a deep-learning bug localizer becomes substantially more effective when it is given the characteristics of the training data rather than only the parsed model code. Theia builds a library-independent meta-model of a Keras or PyTorch program that records both model properties, such as layers, filters, units, dropout, normalization, activation, loss, optimizer, learning rate, and batch size, and data properties, such as input type, number of classes, problem type, input range, and training-set size. Twelve verification rules then flag mismatches between the two, and on 40 unseen buggy programs Theia localizes 57 of 75 structural bugs while NeuraLint localizes 17. Applying Theia's suggested fixes improved measured performance in 34 of the 40 buggy programs, and the paper reports 12 false alarms when the same fixes are applied to 40 repaired normal programs.

Load-bearing premise

The thresholds inside the 12 verification rules were derived from 105 developer Q&A posts and design literature, then applied as fixed universal rules to the 40 unseen benchmark programs, so the central effectiveness result stands on those thresholds generalizing to the broader population of deep-learning programs.

Editorial extensions

If this is right

  • Structural bugs that today announce themselves only as low or stuck accuracy could be reported and located before training begins, saving compute and debugging effort.
  • Dataset-aware static analysis can outperform source-only static analysis on architecture-level bugs such as loss/activation mismatches and networks that are too shallow for their input.
  • Theia's reports include the buggy layer and a concrete fix, and applying those fixes improved measured performance in 34 of 40 buggy programs.
  • The same meta-model design is intended to extend to other architectures, such as recurrent networks, by adding new verification rules.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: the data-versus-structure checking recipe could likely be carried to transformer and RNN programs by replacing the 12 rules with architecture-specific invariants, an extension the paper only gestures at.
  • Beyond the paper's claims: because the thresholds are fixed numbers, a testable refinement would make them depend on dataset size or model capacity; for instance, the normalization and dropout rules that produced 12 false alarms on normal programs might be gated on model complexity.
  • Beyond the paper's claims: since the ground truth comes from developer Q&A answers, an independent check on version-control commits with confirmed regressions would test whether the result transfers outside the Q&A population.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper presents Theia, a pre-training static checker that localizes 12 types of structural bugs in Keras and PyTorch deep learning programs by combining source-code analysis (call-strings and parameter-sensitive analysis) with training-data characteristics such as input type, number of classes, and problem type. The verification rules and their thresholds were derived from 105 Stack Overflow posts and design literature, and the approach is evaluated on 40 unseen Stack Overflow buggy programs, comparing against NeuraLint. The central claim is that Theia localizes 57/75 structural bugs versus NeuraLint's 17/75.

Significance. The idea of using data characteristics beyond the model code is potentially useful and addresses a real gap: prior static tools such as NeuraLint do not consider the dataset, so architecture-to-data mismatches go undetected. The replication package and the comparison on a 40-program Stack Overflow benchmark are tangible contributions, and the paper honestly documents several limitations in RQ3. However, the reported effectiveness numbers are not currently trustworthy because of internal inconsistencies in the true-positive accounting, an inflated denominator, and a self-referential false-positive definition. If the evaluation were redone with a principled ground truth, a version of the central claim might still hold, but the paper as written does not support it.

major comments (5)
  1. [§5.2.3, Tables 5/6 and 7/8] The TP counts in Tables 5 and 6 are internally inconsistent with the actionable-fix evaluation in Tables 7 and 8. Table 5 credits SO#58609115 with TP=2 and SO#59278771 with TP=1, and Table 6 credits SO#54923573 and SO#37229086 with TP=1 each; yet Table 7 marks Theia's fix for these four programs as FP, with accuracy drops from 99.78% to 0.40%, 97.33% to 88.00%, 93.84% to 92.86%, and 86.25% to 78.56%, respectively. The paper never subtracts these false alarms from the 57/75 total. Please recompute the TP counts after excluding warnings whose suggested fix degrades performance, or explicitly justify why a detection whose fix is harmful still counts as a true positive.
  2. [§5.5 and Table 11] The headline '57/75' overstates the result by including 12 'Other' bugs that Theia does not support, as Section 5.5 explicitly states. Table 11 shows these 12 unsupported bugs in the denominator; the supported-bug denominator is 63, so the supported catch rate is 57/63. The abstract and Section 5.6 should report the supported and unsupported rates separately so the reader can see both the 57/63 recall on supported bug types and the 57/75 overall figure.
  3. [§5.2.3 and §5.3.4] The paper contains contradictory statements about false positives. Section 5.2.3 says 'we have not encountered any false positive cases,' but Section 5.3.4 reports 12 false alarms from Theia on the 40 repaired 'normal' programs (FP=Yes in Tables 9 and 10). Moreover, the definition of a true positive is circular: any Theia warning not in the Stack Overflow ground truth is declared a true positive if the authors' own repair improved accuracy, and the same manual verification is used to judge both the fix and the TP. Please use an independent ground truth (e.g., only accepted Stack Overflow answers, with additional findings reported separately) and integrate the normal-program false alarms into a precision analysis.
  4. [§4.1.11 and Table 2] The LOB rule is written with a logical AND in Table 2 ('if learning_rate >0.01 and learning_rate <0.0001'), which is an unsatisfiable condition; the prose in §4.1.11 correctly uses OR. If the implementation follows Table 2, no learning-rate bug could ever be detected, contradicting the 4/4 LOB detections in Table 11. Correct the inconsistency and say explicitly whether the table or the prose is authoritative.
  5. [§3.3 and Table 2] The rule thresholds (e.g., INF filter bounds, ICL layer counts, LOB learning-rate bounds, IBS batch-size bounds) are hand-derived from 105 Stack Overflow posts and then applied to 40 unseen Stack Overflow posts. Since the training and test sets come from the same population and the thresholds are fixed parameters, the reported 57/63 recall may partly reflect fitting to the same kind of programs. Please add a sensitivity analysis (varying each threshold) or validate on an independent benchmark (e.g., the full defect4ML benchmark, not just 12 programs) to show the result is not an artifact of threshold selection.
minor comments (4)
  1. [Algorithm 1, line 13] The condition `layer_names[i] "averagepooling2d"` is missing the equality operator; it should be `layer_names[i] == "averagepooling2d"`.
  2. [Tables 4 and 8] The regression improvement is reported as 'improvement % = loss after fix - loss of buggy model', but for regression a decreasing loss should be shown as a positive improvement. Table 8 row 15 shows a negative value marked with an up-arrow, which is confusing; please define the improvement sign unambiguously.
  3. [§5.2.3] The phrase 'we have not encountered any false positive cases' is contradicted by the paper's own normal-program evaluation (Section 5.3.4); please reword to describe the buggy-program setting or qualify the statement.
  4. [§6, External Threat] The threat-to-validity paragraph acknowledges generalizability concerns but does not quantify how much the thresholds could vary; a brief discussion of the threshold derivation's impact would help readers calibrate confidence in the rules.

Circularity Check

1 steps flagged · score 6.0 of 10

Theia's 57/75 headline rests on a self-referential true-positive rule: detections outside the Stack Overflow ground truth are counted as TP whenever the authors' own fix helps, and the same detections are marked FP when performance drops, without adjusting the total.

  1. self definitional [Section 5.2.3 'Results Representation'; Tables 5, 6, 7, 8; Section 5.3.3]
    "Both the approaches, Theia, and NeuraLint, also detect the bugs that are not recommended by Stack Overflow users. ... We do not consider such fixes as false positives as addressing these structural inefficiencies helps improve the DL program's structure as discussed in Section 4.1 which in turn helps improve the model's performance. Therefore, we have not encountered any false positive cases and do not report them in Table 5 and Table 6."

    The paper first defines the Stack Overflow recommended fix as ground truth, then expands the ground truth to include any fix the authors found helpful and declares Theia-only warnings non-false-positives because they address 'structural inefficiencies' defined by Theia's own rules. This makes 'true positive' mean 'detected by Theia and judged helpful by the authors'; false positives are removed by definition rather than by an independent oracle. The inconsistency is concrete: Table 7/8 mark Theia's fixes for SO#37229086, SO#54923573, SO#58609115, and SO#59278771 as FP (accuracy drops, e.g., 99.78% to 0.40%), yet Table 5 credits the same programs with TP=1, 1, 2, and 1. The 57/75 total is therefore not an independent validation but an artifact of the post-hoc scoring rule.

full rationale

The core methodology is largely non-circular: thresholds derived from 105 Stack Overflow posts are applied to 40 unseen posts, and the defect4ML benchmark is used as an external check, so the rule-induction step is a train/test split rather than circular reasoning. The circularity is in the evaluation oracle. After declaring the Stack Overflow recommended fix to be ground truth, Section 5.2.3 expands that ground truth with non-accepted fixes the authors found helpful and refuses to count Theia-only structural warnings as false positives, so TP counts are constructed from Theia's own output plus author judgment. Tables 7 and 8 even flag four of these detections as false positives while Tables 5 and 6 count the same detections as true positives, making the headline 57/75 internally inconsistent. A secondary denominator issue (57/75 includes 12 unsupported 'Other' bugs from Table 11 that Section 5.5 says Theia does not support) compounds the headline but is not itself a circular step.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; the meta-model and verification rules are software artifacts, not independently evidenced entities. The free parameters are the hand-fitted thresholds for the twelve bug rules, all derived from a single corpus of Stack Overflow posts. The main burden is the domain assumption that dataset characteristics determine appropriate model structure, which the paper states but does not prove.

free parameters (10)
  • INF conv filter bounds = grayscale: <6 or >256; color: <16 or >512
    Thresholds for the Inaccurate Number of Filters rule, chosen from the 105 Stack Overflow posts and design literature. They determine whether a conv layer is flagged.
  • ICL minimum conv layers = 2 for grayscale, 3 for color
    Threshold for Inappropriate Number of Convolution Layers, derived from the same corpus and applied universally.
  • IFL maximum dense layers = 3
    Threshold for Improper Number of Fully Connected Layers; dense layer counts above 3 produce a warning.
  • IDS pooling interval = pooling required after at most 4 consecutive conv layers
    Threshold for Insufficient Downsampling, chosen from the corpus and literature.
  • LOB learning rate bounds = 0.0001 to 0.01
    Learning rates outside this range are reported as bugs.
  • IBS batch size bounds = 32 to 256
    Batch sizes outside this range are reported as bugs.
  • MRD dropout count threshold = >1
    More than one dropout applied to a single dense/conv layer is reported as redundant.
  • IDN input range = [0,1] or [-1,1]
    Input data must lie in one of these ranges, otherwise the data is reported as unnormalized.
  • INN dense units rule = units <= input size; progressive decrease in CNN
    Dense units exceeding the layer input size, or increasing toward the output in a CNN, are flagged.
  • MNL normalization rule = BatchNorm required after every dense/conv layer before activation
    Missing normalization is reported whenever this pattern is not present.
assumptions (5)
  • standard math Universal Approximation Theorem and standard deep learning design guidelines (e.g., non-linear activations in hidden layers) justify the detection rules.
    Invoked in Section 4.1.1 (CNL rationale) and other rule justifications.
  • domain assumption Dataset characteristics (input type, number of classes, problem type) are sufficient to determine whether a model structure is appropriate.
    The entire method relies on this mapping from data to model configuration, presented in Table 1 and used by rules such as ICL and INF; it is asserted from Stack Overflow posts and not proven.
  • domain assumption The 105 Stack Overflow posts and the defect4ML benchmark are representative enough to set thresholds that generalize to unseen DL programs.
    Thresholds in Section 3.3 and Table 2 are derived from these sources; generalization to the 40-program benchmark is claimed without external validation.
  • domain assumption Stack Overflow accepted/useful answers, plus the authors' own repair experiments, provide correct ground truth for bugs.
    Used in Section 5.2.3; the ground truth is a mix of community answers and author-applied fixes where the original answer was not accepted.
  • domain assumption The meta-model built from get_config() and modules() accurately captures the executed model structure.
    Used in Section 4; if parsing misses configuration details, the rules will fire incorrectly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Data Characteristics for Bug Localization in Deep Learning Programs." pith.science (2026). https://pith.science/paper/JGM6EQVB

@misc{pith2026241205775,
  author       = {Pith},
  title        = {Pith review of: Leveraging Data Characteristics for Bug Localization in Deep Learning Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JGM6EQVB}},
  note         = {Machine review of arXiv:2412.05775}
}
read the original abstract

Deep Learning (DL) is a class of machine learning algorithms that are used in a wide variety of applications. Like any software system, DL programs can have bugs. To support bug localization in DL programs, several tools have been proposed in the past. As most of the bugs that occur due to improper model structure known as structural bugs lead to inadequate performance during training, it is challenging for developers to identify the root cause and address these bugs. To support bug detection and localization in DL programs, in this paper, we propose Theia, which detects and localizes structural bugs in DL programs. Unlike the previous works, Theia considers the training dataset characteristics to automatically detect bugs in DL programs developed using two deep learning libraries, Keras and PyTorch. Since training the DL models is a time-consuming process, Theia detects these bugs at the beginning of the training process and alerts the developer with informative messages containing the bug's location and actionable fixes which will help them to improve the structure of the model. We evaluated Theia on a benchmark of 40 real-world buggy DL programs obtained from Stack Overflow. Our results show that Theia successfully localizes 57/75 structural bugs in 40 buggy programs, whereas NeuraLint, a state-of-the-art approach capable of localizing structural bugs before training localizes 17/75 bugs.

Figures

Figures reproduced from arXiv: 2412.05775 by the authors.

Figure 1
Figure 1. An example from Stack Overflow [16] with structural bug. The developer in the post mentions the various CNN architectures that were attempted but did not achieve the desired results. In response, another Stack Overflow user pointed out the following issues in the CNN program. First, the dataset used to train the model is very small, and the user recommends increasing the dataset size by adding high-quality data. Sec… view at source ↗
Figure 2
Figure 2. Details of DL programs used for mapping. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Overview of Theia. function, DL programs follow different structures where the model is built sequentially by calling different layer APIs, e.g., Conv2D, Activation, MaxPooling2D, provided by DL libraries one after the other. For traditional programs, the call-strings approach is used to keep track of how many times each function is called, how many times it is returned, and what other functions are called by it. In… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Meta-Model used in Theia. and dense layer activation function is used once. Also, the choice of activation depends on the type of the task, i.e., regression or classification. For example, for image data, Krizhevsky et al. [67] have shown the benefit of using non-satur…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 73 canonical work pages

  1. [1]

    CCannot train a neural network solving XOR mapping

    2015. CCannot train a neural network solving XOR mapping. https://stackoverflow.com/questions/34311586/

    work page arXiv 2015

  2. [2]

    How to prepare a dataset for Keras? https://stackoverflow.com/questions/31880720/

    2015. How to prepare a dataset for Keras? https://stackoverflow.com/questions/31880720/

    work page arXiv 2015

  3. [3]

    Trying Kaggle Titanic with keras

    2015. Trying Kaggle Titanic with keras .. getting loss and valid_loss -0.0000. https://stackoverflow.com/questions/ 31627380/

    2015

  4. [4]

    Accuracy not high enough for dogs_cats classification dataset using CNN with Keras-Tf python

    2016. Accuracy not high enough for dogs_cats classification dataset using CNN with Keras-Tf python. https: //stackoverflow.com/questions/40045159/

    work page arXiv 2016

  5. [5]

    How to train and tune an artificial multilayer perceptron neural network using Keras? https://stackoverflow

    2016. How to train and tune an artificial multilayer perceptron neural network using Keras? https://stackoverflow. com/questions/34673164/

    work page arXiv 2016

  6. [6]

    Keras low accuracy classification task

    2016. Keras low accuracy classification task. https://stackoverflow.com/questions/38648195/

    work page arXiv 2016

  7. [7]

    Neural network accuracy optimization

    2016. Neural network accuracy optimization. https://stackoverflow.com/questions/39525358/

    work page arXiv 2016

  8. [8]

    Why can’t my CNN learn? https://stackoverflow.com/questions/37229086/

    2016. Why can’t my CNN learn? https://stackoverflow.com/questions/37229086/

    work page arXiv 2016

Show all 89 references
  1. [9]

    How does Keras handle multilabel classification? https://stackoverflow.com/questions/44164749/

    2017. How does Keras handle multilabel classification? https://stackoverflow.com/questions/44164749/

    2017

  2. [10]

    Keras: Training loss decrases (accuracy increase) while validation loss increases (accuracy decrease)

    2017. Keras: Training loss decrases (accuracy increase) while validation loss increases (accuracy decrease). https: //stackoverflow.com/questions/47272383/

    2017

  3. [11]

    Why does a binary Keras CNN always predict 1? https://stackoverflow.com/questions/45378493/

    2017. Why does a binary Keras CNN always predict 1? https://stackoverflow.com/questions/45378493/

    2017

  4. [12]

    CNN not efficient on my dataset in Keras

    2018. CNN not efficient on my dataset in Keras. https://stackoverflow.com/questions/51749207/

    2018

  5. [13]

    CNN train accuracy gets better during training, but test accuracy stays around 40%

    2018. CNN train accuracy gets better during training, but test accuracy stays around 40%. https://stackoverflow.com/ questions/48594888/

    2018

  6. [14]

    CNN with keras, accuracy not improving

    2018. CNN with keras, accuracy not improving. https://stackoverflow.com/questions/50079585/

    2018

  7. [15]

    Create a square function estimator with Keras

    2018. Create a square function estimator with Keras. https://stackoverflow.com/questions/48221692/

    2018

  8. [16]

    How to improve the performance of CNN Model for a specific Dataset? Getting Low Accuracy on both training and Testing Dataset

    2018. How to improve the performance of CNN Model for a specific Dataset? Getting Low Accuracy on both training and Testing Dataset. https://stackoverflow.com/questions/70554413/

    2018

  9. [17]

    Input nodes in Keras NN

    2018. Input nodes in Keras NN. https://stackoverflow.com/questions/51930566/

    2018

  10. [18]

    Keras overfits on one class Cifar-10

    2018. Keras overfits on one class Cifar-10. https://stackoverflow.com/questions/51118032/

    2018

  11. [19]

    my Keras model does not predict negative values

    2018. my Keras model does not predict negative values. https://stackoverflow.com/questions/48251943/

    2018

  12. [20]

    Non linear Regression: Why isn’t the model learning? https://stackoverflow.com/questions/48934338/

    2018. Non linear Regression: Why isn’t the model learning? https://stackoverflow.com/questions/48934338/

    2018

  13. [21]

    Simple Keras neural network isn’t learning

    2018. Simple Keras neural network isn’t learning. https://stackoverflow.com/questions/48385830/

    2018

  14. [23]

    Accuracy Equals 0 CNN Python Keras

    2019. Accuracy Equals 0 CNN Python Keras. https://stackoverflow.com/questions/58844149/

    2019

  15. [24]

    Keras CNN intermediate level has no feature changes

    2019. Keras CNN intermediate level has no feature changes. https://stackoverflow.com/questions/54923573/

    2019

  16. [25]

    Keras CNN model with a wrong ROC curve and low accuracy

    2019. Keras CNN model with a wrong ROC curve and low accuracy. https://stackoverflow.com/questions/56914715/

    2019

  17. [26]

    Loss doesn’t decrease in Pytorch CNN

    2019. Loss doesn’t decrease in Pytorch CNN. https://stackoverflow.com/questions/58666904/

    2019

  18. [27]

    Low accuracy after training a CNN

    2019. Low accuracy after training a CNN. https://stackoverflow.com/questions/59325381/

    2019

  19. [28]

    Manual predictions of neural net go wrong

    2019. Manual predictions of neural net go wrong. https://stackoverflow.com/questions/58609115/

    2019

  20. [29]

    My cnn accuracy goes down after adding one more feature

    2019. My cnn accuracy goes down after adding one more feature. https://stackoverflow.com/questions/55343875/

    2019

  21. [30]

    Sudden 50% accuracy drop while training convolutional NN

    2019. Sudden 50% accuracy drop while training convolutional NN. https://stackoverflow.com/questions/55198221/. J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2023. 111:26 Ruchira Manke, Mohammad Wardat, Foutse Khomh, and Hridesh Rajan

    2019

  22. [31]

    Super low accuracy for neural network model

    2019. Super low accuracy for neural network model. https://stackoverflow.com/questions/59278771/

    2019

  23. [32]

    tf.keras loss becomes NaN

    2019. tf.keras loss becomes NaN. https://stackoverflow.com/questions/55328966/

    2019

  24. [33]

    Getting pretty bad accuracy using CNN Model in Keras

    2020. Getting pretty bad accuracy using CNN Model in Keras. https://stackoverflow.com/questions/65275387/

    2020

  25. [34]

    Keras model not training layers, validation accuracy always 0.5

    2020. Keras model not training layers, validation accuracy always 0.5. https://stackoverflow.com/questions/60261103/

    2020

  26. [35]

    normalize training data with channel means and standard deviation in CNN model

    2020. normalize training data with channel means and standard deviation in CNN model. https://stackoverflow.com/ questions/63027146/

    2020

  27. [36]

    Poor accuracy of CNN model with Keras

    2020. Poor accuracy of CNN model with Keras. https://stackoverflow.com/questions/64522751/

    2020

  28. [37]

    Pytorch CNN loss is not changing,

    2020. Pytorch CNN loss is not changing,. https://stackoverflow.com/questions/60003876/

    2020

  29. [38]

    why is my model performing poorly for a keras sequential model? https://stackoverflow.com/questions/64188884/

    2020. why is my model performing poorly for a keras sequential model? https://stackoverflow.com/questions/64188884/

    2020

  30. [39]

    Pytorch CNN not learning

    2021. Pytorch CNN not learning. https://stackoverflow.com/questions/65659888/

    2021

  31. [40]

    Why does the loss decreases and the accuracy dosen’t increases? PyTorch

    2021. Why does the loss decreases and the accuracy dosen’t increases? PyTorch. https://stackoverflow.com/questions/ 70428592/

    2021

  32. [41]

    Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2017. Designing neural network architectures using reinforcement learning. In ICLR’17: International Conference on Learning Representations

    2017

  33. [42]

    Houssem Ben Braiek and Foutse Khomh. 2023. Testing Feedforward Neural Networks Training Programs. ACM Trans. Softw. Eng. Methodol. 32, 4, Article 105 (may 2023), 61 pages. https://doi.org/10.1145/3529318

    2023 doi

  34. [43]

    Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. In In Neural networks: Tricks of the trade . Springer, Berlin, Heidelberg, 437–478

    2012

  35. [44]

    Sumon Biswas, Mohammad Wardat, and Hridesh Rajan. 2022. The Art and Practice of Data Science Pipelines: A Comprehensive Study of Data Science Pipelines In Theory, In-The-Small, and In-The-Large. In ICSE’22: The 44th International Conference on Software Engineering (Pittsburgh,...

    2022

  36. [45]

    Junming Cao, Bihuan Chen, Chao Sun, Longjie Hu, Shuaihong Wu, and Xin Peng. 2022. Understanding performance problems in deep learning systems. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering...

    2022

  37. [46]

    Jialun Cao, Meiziniu Li, Xiao Chen, Ming Wen, Yongqiang Tian, Bo Wu, and Shing-Chi Cheung. 2022. DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs. In ICSE’22: The 44th International Conference on Software Engineering (Pittsburgh, PA, USA)

    2022

  38. [47]

    Nwankpa Chigozie, Winifred Ijomah, Anthony Gachagan, and Stephen Marshall. 2018. Activation functions: Compari- son of trends in practice and research for deep learning. In arXiv preprint arXiv:1811.03378

    2018 arXiv

  39. [48]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In IEEE Computer Vision and Pattern Recognition (CVPR)

    2009

  40. [49]

    Hasan Ferit Eniser, Simos Gerasimou, and Alper Sen. 2019. DeepFault: Fault localization for deep neural networks. In International Conference on Fundamental Approaches to Software Engineering . Springer, 171–191

    2019

  41. [50]

    Mart´𝑖n Abadi et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation . 265–283

    2016

  42. [51]

    Francois Chollet. 2015. Keras: the Python Deep Learning library. https://keras.io/

    2015

  43. [52]

    Francois Chollet. 2015. Keras: the Python Deep Learning library. https://keras.io/api/losses/

    2015

  44. [53]

    Ali Ghanbari, Deepak-George Thomas, Muhammad Arbab Arshad, and Hridesh Rajan. 2023. Mutation-based Fault Localization of Deep Neural Networks. In ASE’2023: 38th IEEE/ACM International Conference on Automated Software Engineering (Kirchberg, Luxembourg)

    2023

  45. [54]

    David Grove, Greg DeFouw, Jeffrey Dean, and Craig Chambers. 1997. Call graph construction in object-oriented languages. In Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. 108–124

    1997

  46. [55]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR’16: IEEE conference on computer vision and pattern recognition . 770–778

    2016

  47. [56]

    Robert Hecht-Nielsen. 1992. Theory of the backpropagation neural network. Neural networks for perception (1992), 65–93

    1992

  48. [57]

    Larochelle Hugo, Yoshua Bengio, Jérôme Louradour, and Pascal Lamblin. 2009. Exploring strategies for training deep neural networks. In Journal of machine learning research , Vol. 10

    2009

  49. [58]

    Nargiz Humbatova, Gunel Jahangirova, Gabriele Bavota, Vincenzo Riccio, Andrea Stocco, and Paolo Tonella. 2020. Taxonomy of real faults in deep learning systems. In ICSE’20: The ACM/IEEE 42nd International Conference on Software Engineering. 1110–1121

    2020

  50. [59]

    Giancarlo Iannizzotto, Lucia Lo Bello, Andrea Nucita, and Giorgio Mario Grasso. 2018. A vision and speech enabled, customizable, virtual assistant for smart environments. In 2018 11th International Conference on Human System Interaction (HSI). 50–56

    2018

  51. [60]

    Pier Paolo Ippolito. 2022. Hyperparameter Tuning: The Art of Fine-Tuning Machine and Deep Learning Models to Improve Metric Results. In Applied data science in tourism: Interdisciplinary approaches, methodologies, and applications . J. ACM, Vol. 37, No. 4, Article 111. Publica...

    2022

  52. [61]

    Md Johirul Islam, Giang Nguyen, Rangeet Pan, and Hridesh Rajan. 2019. A Comprehensive Study on Deep Learning Bug Characteristics. In ESEC/FSE’19: The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (ESEC/FSE 2019)

    2019

  53. [62]

    Md Johirul Islam, Rangeet Pan, Giang Nguyen, and Hridesh Rajan. 2020. Repairing Deep Neural Networks: Fix Patterns and Challenges. In ICSE’20: The 42nd International Conference on Software Engineering (Seoul, South Korea)

    2020

  54. [63]

    Haifeng Jin, Qingquan Song, and Xia Hu. 2019. Auto-keras: An efficient neural architecture search system. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining . 1946–1956

    2019

  55. [64]

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. 2016. On large-batch training for deep learning: Generalization gap and sharp minima. In ICLR’16: International Conference on Learning Representations

    2016

  56. [65]

    Alex Krizhevsky. 2009. Learning multiple layers of features from tiny images . Technical Report

    2009

  57. [66]

    Alex Krizhevsky and Geoff Hinton. 2010. Convolutional deep belief networks on cifar-10. Unpublished manuscript 40 7 (2010), 1–9

    2010

  58. [67]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional networks. In NIPS’12: The 25th International Conference on Neural Information Processing Systems , Vol. 1. 1097–1105

    2012

  59. [68]

    LeCunand L´𝑒on, BottouGenevieve B.Orr, and Klaus-Robert M¥𝑢ller

    Yann A. LeCunand L´𝑒on, BottouGenevieve B.Orr, and Klaus-Robert M¥𝑢ller. 2012. Efficient backprop. In Springer. Berlin, Heidelberg

    2012

  60. [69]

    Denker, Donnie Henderson, Richard E

    Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel. 1989. Backpropagation applied to handwritten zip code recognition. Neural computation 1, 4 (1989), 541–551

    1989

  61. [70]

    Yann LeCun, L´𝑒on Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. In Proceedings of IEEE, Vol. 11. 2278–2324

    1998

  62. [71]

    Shiqing Ma, Yingqi Liu, Wen-Chuan Lee, Xiangyu Zhang, and Ananth Grama. 2018. MODE: automated neural network model debugging via state differential analysis and input selection. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and S...

    2018

  63. [72]

    Leveraging Data Characteristics for Bug Localization in Deep Learning Programs

    Ruchira Manke, Mohammad Wardat, Foutse Khomh, and Hridesh Rajan. 2024. Replication Package of Paper Titled: "Leveraging Data Characteristics for Bug Localization in Deep Learning Programs". https://doi.org/10.5281/zenodo. 14292112 . The replication package and results are avai...

    2024 doi

  64. [73]

    Sharir Micha and Amir Pnueli. 1978. Two approaches to interprocedural data flow analysis. In New York University. Courant Institute of Mathematical Sciences

    1978

  65. [74]

    Mohammad Mehdi Morovati, Amin Nikanjam, Foutse Khomh, and Zhen Ming Jiang. 2023. Bugs in machine learning- based systems: a faultload benchmark. In Empirical Software Engineering, Vol. 28

    2023

  66. [75]

    Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML’10: International Conference on Machine Learning

    2010

  67. [76]

    Amin Nikanjam, Braiek Ben Houssem, Morovati Mehdi Mohammad, and Khomh Foutse. 2021. Automatic fault detection for deep learning programs using graph transformations. In ACM Transactions on Software Engineering and Methodology (TOSEM), Vol. 31. 1–27

    2021

  68. [77]

    Keiron O’Shea and Ryan Nash. 2015. An introduction to convolutional neural networks.arXiv preprint arXiv:1511.08458 (2015)

    2015 arXiv

  69. [78]

    Adam Paszke, Sam Gross, Soumith Chintala, and Gregory Chanan. 2016. PyTorch: open source machine learning framework. https://pytorch.org/

    2016

  70. [79]

    Csaky Richard. 2019. Deep learning based chatbot models. In arXiv preprint arXiv:1908.08835

    2019 arXiv

  71. [80]

    Abhimanyu Roy, Jingyi Sun, Robert Mahoney, Loreto Alonzi, Stephen Adams, and Peter Beling. 2018. Deep learning detecting fraud in credit card transactions. In 2018 systems and information engineering design symposium . 129–134

    2018

  72. [81]

    Eldon Schoop, Forrest Huang, and Björn Hartmann. 2021. UMLAUT: Debugging Deep Learning Programs using Program Structure and Model Behavior. In Proceedings of the 2021 CHI Conference Extended Abstracts on Human Factors in Computing Systems

    2021

  73. [82]

    Ioffe Sergey and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning . 448–456

    2015

  74. [83]

    Olin Shivers. 1991. DataFlow Analysis and TypeRecovery in Scheme. In Topics in Advanced Language Implementation: MIT Press

    1991

  75. [84]

    Wei Shiyi and Barbara G. Ryder. 2015. Adaptive context-sensitive analysis for JavaScript. In 29th European Conference on Object-Oriented Programming (ECOOP 2015)

    2015

  76. [85]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014). J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2023. 111:28 Ruchira Manke, Mohammad Wardat, Foutse Khomh, and Hr...

    2014 arXiv

  77. [86]

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. In The journal of machine learning research , Vol. 15. 1929–1958

    2014

  78. [87]

    Mohammad Wardat, Breno Dantas Cruz, Wei Le, and Hridesh Rajan. 2022. DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs. In ICSE’22: The 44th International Conference on Software Engineering

    2022

  79. [88]

    Mohammad Wardat, Wei Le, and Hridesh Rajan. 2021. DeepLocalize: fault localization for deep neural networks. In ICSE’21: The 43nd International Conference on Software Engineering

    2021

  80. [89]

    Xiaoyu Zhang, Juan Zhai, Shiqing Ma, and Chao Shen. 2021. AUTOTRAINER: An Automatic DNN Training Problem Detection and Repair System. In ICSE’21: The 43rd International Conference on Software Engineering . 359–371

    2021

  81. [90]

    Yuhao Zhang, Chen Yifan, Cheung Shing-Chi, Xiong Yingfei, and Zhang Lu. 2018. An empirical study on TensorFlow program bugs. In 27th ACM SIGSOFT International Symposium on Software Testing and Analysis . 129–140. J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2023

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.