Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Metamorphic Testing for Pose Estimation Systems

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that metamorphic testing can reveal faults in pose estimation systems as effectively as hand-labelled ground-truth testing, without the labelling cost.

desk verdict A solid, transparent metamorphic-testing framework for pose estimation; the fault-rate comparison with classic testing overstates what the experiments show, but the core idea is useful and worth refereeing. read the letter →

arxiv 2502.09460 v1 pith:FWZBXPX7 submitted 2025-02-13 cs.SE cs.AIcs.CV

Matias Duran , Thomas Laurent , Ellen Rushe , Anthony Ventresque This is my paper · ORCID
classification cs.SEcs.AIcs.CV
keywords metamorphictestingposeestimationoracleproblemgroundtruthfreeMediapipeHolisticrelationscomputervision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MET-POSE, a metamorphic testing framework that tests pose estimation systems without needing manually labelled ground-truth keypoints. It claims that a pose estimator can be checked by transforming input images (mirroring, stretching, blurring, changing brightness or colour) and verifying that the output keypoints change exactly as the transformation dictates; any violation counts as a fault. The authors apply MET-POSE to Mediapipe Holistic on the FLIC and PHOENIX datasets and report that it uncovers faults at a rate similar to, and for higher error thresholds greater than, classic ground-truth-based testing. If this holds, practitioners can test pose estimators on their own application-specific images at a fraction of the labelling cost, and can tune which metamorphic rules to use for the failure types they care about.

What carries the argument

The central object is the metamorphic rule: a pair (transformation, relation), where the transformation produces a modified image and the relation states what the keypoint outputs on the two images must satisfy for the system to behave correctly—for example identity for brightness or blur changes, mirroring for a mirror transformation, and rotation for a rotated image. The severity of a violation is computed by $\mathrm{Err}_{\mathrm{lmks}}$, the median of per-landmark normalized Euclidean distances (using Mediapipe's shoulder, iris, and wrist normalizations), with an infinite error when keypoints are detected on only one of the two images. This mechanism carries the argument because it converts an unanswerable question, 'are these output keypoints correct?', into a checkable one, 'did the outputs change the way a correct system would change them?'.

What would settle it

Run a rule on images with known ground-truth keypoints for both the original and transformed versions; if the rule reports violations on many images where the system's keypoints are accurate on both versions, the rule is not a reliable oracle, and if, conversely, every violated image has a large ground-truth error, the framework's fault signal is confirmed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the oracle problem for pose estimation can be bypassed: instead of knowing where the keypoints should be, one only needs to know how keypoint positions should change when the input image is transformed. MET-POSE formalizes each check as a metamorphic rule pairing a transformation with a relation, evaluates violations with an error metric that returns the median normalized distance between expected and detected keypoints (and an infinite error when keypoints disappear on one side of the pair), and then reports violations and their severity. The experiments on Mediapipe Holistic show that the framework finds large numbers of rule violations on both datasets, that on FLIC the violations overlap with, and at higher thresholds exceed, the failures found by comparing against the hand-labelled ground truth, and that different rules are not subsumed by one another, so different rules surface different types of faults.

Load-bearing premise

For each metamorphic rule to be a valid fault detector, the relation it encodes must be a necessary property of any correct pose estimator; if a correct system could legitimately lose track of a subject under an extreme transformation, the rule's violation is a property of the transformation, not a fault of the system.

Editorial extensions

If this is right

  • Users can run MET-POSE on unlabelled images from their own application domain and still obtain a fault signal, so domain shift no longer forces a new labelling effort.
  • At higher error thresholds the framework reports more failures than classic ground-truth testing on FLIC, so it can act as a complement that catches faults labelled testing misses.
  • Because rule subsumption is low and dataset-dependent, a practical test campaign should include several rule families rather than rely on one transformation.
  • PHOENIX has no ground-truth keypoints, so MET-POSE provides the only fault signal there; this shows testing is possible in settings where classic testing cannot be applied.
  • The same framework extends to any pose estimator and any landmark set by swapping the transformations, relations, and error metric, without changing the underlying testing logic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to calibrate each metamorphic rule against a small labelled sample: compute how often rule violations agree with ground-truth failures on that sample, and drop rules with low agreement, which would filter out invariants that are not truly necessary for correctness.
  • The paper's own observation that some relation configurations produced artificially high error rates suggests error thresholds must be set per rule rather than globally; practitioners could derive per-rule thresholds from the distribution of violation severities on a trusted subset of images.
  • The pattern of which single rule an image violates could serve as a debugging signal for the model: images that violate only a colour rule point to colour over-reliance, images that violate only a blur rule point to texture dependence, and so on, turning the framework from a test oracle into a diagnostic tool.
  • For video-based pose estimation, temporal metamorphic relations—such as keypoint consistency under frame-rate changes or frame reordering—would extend the framework beyond the single-image rules studied here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MET-POSE, a metamorphic testing framework for pose estimation systems that checks whether a system's keypoint outputs are consistent under image transformations (spatial, image-quality, and colour-space changes), thereby avoiding the need for hand-labelled ground truth. The framework is evaluated on MediaPipe Holistic using the FLIC and PHOENIX datasets, with the error metric Err_lmks and a user-configurable threshold. The experiments report violation rates as a function of threshold, compare MET-POSE with ground-truth-based testing on FLIC, and analyse subsumption relationships among metamorphic rules. The central claim is that MET-POSE can uncover faults at a similar or higher rate than classic testing using hand-labelled data.

Significance. If the central claim were fully supported, MET-POSE would be a valuable contribution: it addresses the oracle problem and the high cost of pose-keypoint labelling, it is presented in a system- and dataset-agnostic way, and it gives practitioners a tunable mechanism for testing pose estimators under application-relevant conditions. The paper also has concrete strengths: the framework is described formally enough to reimplement, the companion repository is referenced, the identity rule is used as a determinism sanity check, and the analysis of subsumption rates is an informative way to understand which transformations contribute distinct information. However, the headline comparison with classic testing is currently under-supported because metamorphic-rule violations are treated as faults without independent validation, and because the two testing methods measure different quantities on different inputs.

major comments (4)
  1. [Section V-D] The mapping from a metamorphic-rule violation to a genuine fault of the SUT is not established. In Err_lmks, the expected keypoints on the modified image are derived from the SUT's own keypoints on the original image, so a system that is consistently wrong but transformation-invariant will pass MET-POSE, while an extreme perturbation that prevents detection is counted as an infinite violation even if a correct pose estimator would also fail to detect the subject. The paper therefore needs either to validate each relation on a trustworthy reference system or on a small set of manually labelled transformed images, or to reframe MET-POSE as measuring robustness/sensitivity rather than fault detection.
  2. [Section V-D and Table I] The post-hoc selection of SubRels is load-bearing and is presented as a fix for configurations that produce 'artificially high error rates'. This is an admission that some of the proposed relations are not necessary properties of correct behaviour in the tested contexts. The restriction from AllRels to SubRels is a free parameter that can inflate the apparent effectiveness of the method; the paper should report how the selection was made, justify it with external evidence, and show that the selected rules remain valid on held-out data or on a reference implementation.
  3. [Section VI-B and Figure 6] The RQ2 comparison with classic testing is not on equal terms. Classic testing measures the SUT's error against human labels on the original image, whereas MET-POSE measures consistency between outputs on original and transformed images. A higher failure rate under MET-POSE can reflect sensitivity to input perturbations rather than the presence of genuine pose-estimation faults, so the abstract's claim that MET-POSE 'uncover[s] faults ... at a similar or higher rate than classic testing' is not directly supported by the plotted overlap and difference percentages. A meaningful comparison requires a fault-level definition that relates transformations to ground-truth error, or at least a manual validation of which rule violations correspond to incorrect keypoints.
  4. [Sections V and VI] The empirical results are presented without confidence intervals or significance tests. Figures 4-6 compare percentages on a single SUT and two datasets, and RQ2.2 draws conclusions about overlap of failing images, yet no test such as McNemar's test is applied. Given that the central quantitative claim is about rates of fault discovery, the paper should either provide appropriate statistical support or explicitly present the results as descriptive and limit the strength of the conclusions accordingly.
minor comments (6)
  1. [Section V-B] 'Mediapipe Hollistic' is a typo for 'MediaPipe Holistic'.
  2. [Table I] The footnotes for the colour-fill values are rendered ambiguously: the superscript markers 'a' and 'b' are not clearly tied to the table entries, and the sentence 'close to skin colour on phoenix dataset' lacks a period. Please clarify the notation.
  3. [Section VIII] In the Internal Validity paragraph, the sentence beginning 'We have also added an identity metamorphic relation...' is incomplete and should be rephrased, for example: 'If the identity relation were violated, we would need to run the experiments multiple times to reach a statistically significant conclusion.'
  4. [Figure 7] The subsumption heatmaps are very hard to read because the axis labels are small and dense; consider using a colour scale with larger fonts and abbreviating the rule names in a legend.
  5. [Table II] The first column label '# failed rules' is potentially confusing because the rows are ranges of counts; please rename it to something like 'Number of failed rules per image' and add a note explaining that each row gives the number of images falling into that range.
  6. [References] Reference [17] has a typo in the venue name ('in MET)' instead of a proper workshop name); please correct it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MET-POSE's effectiveness claim is an empirical comparison against an external ground-truth benchmark, not a derivation from its own definitions.

full rationale

The paper's central claim is that MET-POSE can reveal failures in pose estimation systems without labeled ground truth, and that its violation rate is comparable to or higher than classic testing on FLIC. This is an empirical, externally anchored comparison: RQ2 defines classic test failure via normalized distance to FLIC ground-truth keypoints and compares it with metamorphic-rule violation rates across many thresholds. The expected keypoints for transformed images are derived from the SUT's own original-image keypoints, but this is the standard metamorphic testing mechanism (a consistency check), not a fitted parameter renamed as a prediction. The SubRels rule set is selected post hoc after noting that some AllRels configurations produce 'artificially high error rates,' but the paper transparently reports both AllRels and SubRels results and does not present the selection as an independent prediction; this is an experimental-design transparency issue rather than a circular derivation. Self-citations such as [11] and [23] are used only to motivate particular rule choices and are not load-bearing for the main empirical result. The acknowledged threat that a metamorphic violation may not correspond to a genuine pose error is a construct-validity concern, not circularity, and the paper partially addresses it by reporting overlap between MET-POSE failures and classic ground-truth failures. No equation or result in the paper reduces by construction to its own inputs.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The central claim rests on two domain assumptions: the metamorphic relations are valid invariants for correct pose estimators, and the normalized L2 error metric is an appropriate quality measure. Two researcher degrees of freedom (the SubRels rule subset and PHOENIX-specific colour fill values) shape the reported failure rates. These are transparency concerns, not circular reasoning.

free parameters (2)
  • SubRels rule set selection = subset of AllRels (bold rows in Table I)
    The paper selects a restricted set of rules after observing that some AllRels configurations produce 'artificially high error rates' on PHOENIX and FLIC, making this a post-hoc choice that affects the reported violation rates.
  • Dataset-specific colour fill values = [255,180,120] (close to skin colour) and [33,28,27] (close to clothes colour) in PHOENIX
    Colour fill configurations are chosen based on the PHOENIX dataset's palette, so the rule's failure rate is specific to this dataset rather than a general property of the system.
assumptions (2)
  • domain assumption The proposed metamorphic relations (e.g., identity under greyscale, mirror, gamma, colour wheel) are valid invariants; a correct pose estimator would satisfy them up to a small error on all relevant inputs.
    Section IV-A defines relations such as 'keypoints should be unchanged after the transformation', but the paper does not prove or independently validate these relations as necessary properties of a correct system across all transformations.
  • domain assumption The normalized L2 distance error metric (L2MP) is an appropriate measure of pose estimation quality and is comparable between the metamorphic and classic testing settings.
    Section V-D uses this metric for both methods; the authors note in Section VIII that different aggregation choices (mean, max) would change results, so the metric choice influences the conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Metamorphic Testing for Pose Estimation Systems." pith.science (2026). https://pith.science/paper/FWZBXPX7

@misc{pith2026250209460,
  author       = {Pith},
  title        = {Pith review of: Metamorphic Testing for Pose Estimation Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FWZBXPX7}},
  note         = {Machine review of arXiv:2502.09460}
}
read the original abstract

Pose estimation systems are used in a variety of fields, from sports analytics to livestock care. Given their potential impact, it is paramount to systematically test their behaviour and potential for failure. This is a complex task due to the oracle problem and the high cost of manual labelling necessary to build ground truth keypoints. This problem is exacerbated by the fact that different applications require systems to focus on different subjects (e.g., human versus animal) or landmarks (e.g., only extremities versus whole body and face), which makes labelled test data rarely reusable. To combat these problems we propose MET-POSE, a metamorphic testing framework for pose estimation systems that bypasses the need for manual annotation while assessing the performance of these systems under different circumstances. MET-POSE thus allows users of pose estimation systems to assess the systems in conditions that more closely relate to their application without having to label an ad-hoc test dataset or rely only on available datasets, which may not be adapted to their application domain. While we define MET-POSE in general terms, we also present a non-exhaustive list of metamorphic rules that represent common challenges in computer vision applications, as well as a specific way to evaluate these rules. We then experimentally show the effectiveness of MET-POSE by applying it to Mediapipe Holistic, a state of the art human pose estimation system, with the FLIC and PHOENIX datasets. With these experiments, we outline numerous ways in which the outputs of MET-POSE can uncover faults in pose estimation systems at a similar or higher rate than classic testing using hand labelled data, and show that users can tailor the rule set they use to the faults and level of accuracy relevant to their application.

Figures

Figures reproduced from arXiv: 2502.09460 by the authors.

Figure 1
Figure 1. gives an overview of the MET-POSE framework. MET-POSE takes as input the SUT, a set of metamorphic rules, and a set IMGorig of test images. MET-POSE applies the transformation of each rule to each test image in IMGorig, and for each pair of original and modified images it checks whether the rule is violated, and to what degree. For each input image imgorig ∈ IMGorig, and each metamorphic rule M in the input set, the… view at source ↗
Figure 2
Figure 2. Example image modified by various rules [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example inputs from a sports analytics application, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Percentage of images leading to a rule violation with [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Percentage of images in PHOENIX leading to a rule violation for varying error thresholds using hand landmarks the only dataset in our experiments with ground truth labels [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: shows: the proportion of images leading to both a classic test failure and a rule violation, a classic test failure only, and a rule violation only for different error thresholds. A classic test failure is defined as a normalised distance between the ground truth keypo…
Figure 8
Figure 8. Figure 8: Subsumption rates of various motion blur settings [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 7
Figure 7. Figure 7: SubRateM1,M2 , with M1 left and M2 top, for each dataset, using body pose landmarks and a 0.2 error threshold. (0.2). These results show that the subsumption rates between the rules are very dependent on the dataset used, indicating that the different rules make differ…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Understanding Space Is Rocket Science -- Only Top Reasoning Models Can Solve Spatial Understanding Tasks

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A new contrastive real-image benchmark shows most vision-language models fail spatial relation tasks, while chain-of-thought reasoning models approach human-level accuracy.

Reference graph

Works this paper leans on

48 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    A survey on deep learning in medical image analysis,

    G. Litjens, T. Kooi, B. E. Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. van der Laak, B. van Ginneken, and C. I. S ´anchez, “A survey on deep learning in medical image analysis,” MIA, vol. 42, pp. 60–88, 2017

    work page 2017

  2. [2]

    From scarcity to understanding: Transfer learning for the extremely low resource irish sign language,

    R. Holmes, E. Rushe, M. De Coster, M. Bonnaerens, S. Satoh, A. Sug- imoto, and A. Ventresque, “From scarcity to understanding: Transfer learning for the extremely low resource irish sign language,” in ICCV, 2023

    work page 2023

  3. [3]

    Automated tackle injury risk assessment in contact-based sports-a rugby union example,

    Z. Martin, S. Hendricks, and A. Patel, “Automated tackle injury risk assessment in contact-based sports-a rugby union example,” in CVPR, 2021

    work page 2021

  4. [4]

    MediaPipe Holistic — Simultaneous Face, Hand and Pose Prediction, on Device,

    I. Grishchenko and V . Bazarevsky, “MediaPipe Holistic — Simultaneous Face, Hand and Pose Prediction, on Device,” https://research.google/blog/ mediapipe-holistic-simultaneous-face-hand-and-pose-prediction-on-device/, [Accessed 19-08-2024]

    work page 2024

  5. [5]

    2d/3d pose estimation and action recognition using multitask deep learning,

    D. C. Luvizon, D. Picard, and H. Tabia, “2d/3d pose estimation and action recognition using multitask deep learning,” in CVPR, 2018

    work page 2018

  6. [6]

    Mediapipe pose landmark detection system,

    “Mediapipe pose landmark detection system,” [Accessed 21- 01-2025]. [Online]. Available: https://ai.google.dev/edge/mediapipe/ solutions/vision/pose landmarker

    work page 2025

  7. [7]

    The dark side of dataset scaling: Evaluating racial classification in multimodal models,

    A. Birhane, S. Dehdashtian, V . Prabhu, and V . Boddeti, “The dark side of dataset scaling: Evaluating racial classification in multimodal models,” in FAccT, 2024

    work page 2024

  8. [8]

    Auditing saliency cropping algorithms,

    A. Birhane, V . U. Prabhu, and J. Whaley, “Auditing saliency cropping algorithms,” in WACV, 2022

    work page 2022

Show all 48 references
  1. [9]

    Ethical and social risks of harm from language models,

    L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh et al., “Ethical and social risks of harm from language models,” arXiv preprint arXiv:2112.04359, 2021

    2021 arXiv

  2. [10]

    Saving face: Investigating the ethical concerns of facial recognition auditing,

    I. D. Raji, T. Gebru, M. Mitchell, J. Buolamwini, J. Lee, and E. Denton, “Saving face: Investigating the ethical concerns of facial recognition auditing,” in AIES, 2020

    2020

  3. [11]

    What’s left can’t be right– the remaining positional incompetence of contrastive vision-language models,

    N. Hoehing, E. Rushe, and A. Ventresque, “What’s left can’t be right– the remaining positional incompetence of contrastive vision-language models,” arXiv preprint arXiv:2311.11477 , 2023

    2023 arXiv

  4. [12]

    Visual spatial reasoning,

    F. Liu, G. Emerson, and N. Collier, “Visual spatial reasoning,” TACL, vol. 11, 2023

    2023

  5. [13]

    The oracle problem in software testing: A survey,

    E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,” TOSEM, vol. 41, no. 5, 2014

    2014

  6. [14]

    Deepmutation: Mutation testing of deep learning systems,

    L. Ma, F. Zhang, J. Sun, M. Xue, B. Li, F. Juefei-Xu, C. Xie, L. Li, Y . Liu, J. Zhaoet al., “Deepmutation: Mutation testing of deep learning systems,” in ISSRE. IEEE, 2018

    2018

  7. [15]

    Identifying implementation bugs in machine learning based image classifiers using metamorphic testing,

    A. Dwarakanath, M. Ahuja, S. Sikand, R. M. Rao, R. J. C. Bose, N. Dubash, and S. Podder, “Identifying implementation bugs in machine learning based image classifiers using metamorphic testing,” in ISSTA, 2018

    2018

  8. [16]

    Deepgauge: Multi-granularity testing criteria for deep learning systems,

    L. Ma, F. Juefei-Xu, F. Zhang, J. Sun, M. Xue, B. Li, C. Chen, T. Su, L. Li, Y . Liu et al. , “Deepgauge: Multi-granularity testing criteria for deep learning systems,” in ASE, 2018

    2018

  9. [17]

    Robustness evaluation in hand pose estimation models using metamorphic testing,

    M. Pu, C. Y . Chong, and M. K. Lim, “Robustness evaluation in hand pose estimation models using metamorphic testing,” in MET), 2023

    2023

  10. [18]

    Testing and validating machine learning classifiers by metamorphic testing,

    X. Xie, J. W. Ho, C. Murphy, G. Kaiser, B. Xu, and T. Y . Chen, “Testing and validating machine learning classifiers by metamorphic testing,”JSS, vol. 84, no. 4, 2011

    2011

  11. [19]

    opencv implemmentation of gamma correction,

    “opencv implemmentation of gamma correction,” [Accessed 21-01- 2025]. [Online]. Available: https://docs.opencv.org/3.4/d3/dc1/tutorial basic linear transform.html

    2025

  12. [20]

    opencv implemmentation of bilateral filtering,

    “opencv implemmentation of bilateral filtering,” [Accessed 21-01- 2025]. [Online]. Available: https://docs.opencv.org/4.x/d4/d13/tutorial py filtering.html

    2025

  13. [21]

    Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness,

    R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness,” 2022. [Online]. Available: https://arxiv.org/abs/1811.12231

    2022 arXiv

  14. [22]

    Sign language recognition with transformer networks,

    M. De Coster, M. Van Herreweghe, and J. Dambre, “Sign language recognition with transformer networks,” in LREC, 2020

    2020

  15. [23]

    Improving signer independent sign language recognition for low resource languages,

    R. Holmes, E. Rushe, F. Fowley, and A. Ventresque, “Improving signer independent sign language recognition for low resource languages,” in SLTAT, 2022

    2022

  16. [24]

    opencv implemmentation of hsv colorspace encoding,

    “opencv implemmentation of hsv colorspace encoding,” [Accessed 21-01-2025]. [Online]. Available: https://docs.opencv.org/3.4/df/d9d/ tutorial py colorspaces.html

    2025

  17. [25]

    Blazepose ghum holistic: Real-time 3d human landmarks and pose estimation,

    I. Grishchenko, V . Bazarevsky, A. Zanfir, E. G. Bazavan, M. Zanfir, R. Yee, K. Raveendran, M. Zhdanovich, M. Grundmann, and C. Smin- chisescu, “Blazepose ghum holistic: Real-time 3d human landmarks and pose estimation,” arXiv preprint arXiv:2206.11678 , 2022

    2022 arXiv

  18. [26]

    Blazepose: On-device real-time body pose tracking,

    V . Bazarevsky, I. Grishchenko, K. Raveendran, T. Zhu, F. Zhang, and M. Grundmann, “Blazepose: On-device real-time body pose tracking,” arXiv preprint arXiv:2006.10204 , 2020

    2006 arXiv

  19. [27]

    Neural sign language translation,

    N. C. Camgoz, S. Hadfield, O. Koller, H. Ney, and R. Bowden, “Neural sign language translation,” in CVPR, 2018

    2018

  20. [28]

    Modec: Multimodal decomposable models for human pose estimation,

    B. Sapp and B. Taskar, “Modec: Multimodal decomposable models for human pose estimation,” in CVPR, 2013

    2013

  21. [29]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Doll ´ar, “Microsoft coco: Common objects in context,” arXiv preprint arXiv:1405.0312 , 2014

    2014 arXiv

  22. [30]

    Model Card: MediaPipe BlazePose GHUM 3D,

    G. A. for Developers, “Model Card: MediaPipe BlazePose GHUM 3D,” https://storage.googleapis.com/mediapipe-assets/Model%20Card% 20BlazePose%20GHUM%203D.pdf, 2021, [Online; Accessed 2 July, 2024]

    2021

  23. [31]

    Model Card: MediaPipe FaceMesh,

    ——, “Model Card: MediaPipe FaceMesh,” https://storage.googleapis. com/mediapipe-assets/Model%20Card%20MediaPipe%20Face% 20Mesh%20V2.pdf, 2022, [Online; Accessed 25 September, 2024]

    2022

  24. [32]

    Model Card: MediaPipe Hands (Lite/Full),

    ——, “Model Card: MediaPipe Hands (Lite/Full),” https: //storage.googleapis.com/mediapipe-assets/Model%20Card%20Hand% 20Tracking%20(Lite Full)%20with%20Fairness%20Oct%202021.pdf, 2021, [Online; Accessed 25 September, 2024]

    2021

  25. [33]

    Companion repository for “Metamorphic Testing for Pose Estimation Systems

    M. Duran, T. Laurent, E. Rushe, and A. Ventresque, “Companion repository for “Metamorphic Testing for Pose Estimation Systems”,” [Accessed 21-01-2025]. [Online]. Available: https://github.com/MatoFD/ MeT-Pose

    2025

  26. [34]

    Testing practices of software in safety critical systems: Industrial survey

    M. Kassab, “Testing practices of software in safety critical systems: Industrial survey.” in ICEIS (2), 2018

    2018

  27. [35]

    Benchmarking neural network robustness to common corruptions and perturbations,

    D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” 2019. [Online]. Available: https://arxiv.org/abs/1903.12261

    2019 arXiv

  28. [36]

    Towards verifying robustness of neural networks against a family of semantic perturbations,

    J. Mohapatra, T.-W. Weng, P.-Y . Chen, S. Liu, and L. Daniel, “Towards verifying robustness of neural networks against a family of semantic perturbations,” in CVPR, 2020

    2020

  29. [37]

    Robustclevr: A benchmark and frame- work for evaluating robustness in object-centric learning,

    N. Drenkow and M. Unberath, “Robustclevr: A benchmark and frame- work for evaluating robustness in object-centric learning,” in WACV, 2024

    2024

  30. [38]

    Taxonomy of real faults in deep learning systems,

    N. Humbatova, G. Jahangirova, G. Bavota, V . Riccio, A. Stocco, and P. Tonella, “Taxonomy of real faults in deep learning systems,” in ICSE, 2020

    2020

  31. [39]

    Evaluating surprise adequacy for deep learning system testing,

    J. Kim, R. Feldt, and S. Yoo, “Evaluating surprise adequacy for deep learning system testing,” ACM TOSEM, vol. 32, no. 2, 2023

    2023

  32. [40]

    Gender shades: Intersectional accuracy disparities in commercial gender classification,

    J. Buolamwini and T. Gebru, “Gender shades: Intersectional accuracy disparities in commercial gender classification,” inFAccT. PMLR, 2018

    2018

  33. [41]

    Metamorphic testing of an autonomous delivery robots scheduler,

    T. Laurent, P. Arcaini, X.-Y . Zhang, and F. Ishikawa, “Metamorphic testing of an autonomous delivery robots scheduler,” in ICST. IEEE, 2024

    2024

  34. [42]

    Metamorphic testing for con- volutional neural networks: Relations over image classification,

    P. Naidu, H. Gudaparthi, and N. Niu, “Metamorphic testing for con- volutional neural networks: Relations over image classification,” in IRI. IEEE, 2021

    2021

  35. [43]

    Combating missed recalls in e-commerce search: A cot-prompting testing approach,

    S. Wu, Y . Hu, Y . Wang, J. Gu, J. Meng, L. Fan, Z. Luan, X. Wang, and Y . Zhou, “Combating missed recalls in e-commerce search: A cot-prompting testing approach,” in FSE, ser. FSE 2024. New York, NY , USA: Association for Computing Machinery, 2024. [Online]. Available: https:...

    2024

  36. [44]

    Automated functional testing of search engines using metamorphic testing,

    X. Wang, G. Yi, and Y . Wang, “Automated functional testing of search engines using metamorphic testing,” in QRS, 2021

    2021

  37. [45]

    Testing chatbots using metamorphic rela- tions,

    J. Bozic and F. Wotawa, “Testing chatbots using metamorphic rela- tions,” in Testing Software and Systems , C. Gaston, N. Kosmatov, and P. Le Gall, Eds. Cham: Springer International Publishing, 2019

    2019

  38. [46]

    Evaluating hu- man trajectory prediction with metamorphic testing,

    S. Helge, B. Nassim, G. Arnaud, and L. Nadjib, “Evaluating hu- man trajectory prediction with metamorphic testing,” arXiv preprint arXiv:2407.18756, 2024

    2024 arXiv

  39. [47]

    Model cards for model reporting,

    M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchin- son, E. Spitzer, I. D. Raji, and T. Gebru, “Model cards for model reporting,” in FAccT, 2019

    2019

  40. [48]

    Validity threats in empirical software engineering research - an initial survey,

    R. Feldt and A. Magazinius, “Validity threats in empirical software engineering research - an initial survey,” in SEKE, 2010

    2010

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.