Pith. sign in

REVIEW

Measuring AI Systems Beyond Accuracy

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2204.04211 v1 pith:HANDZI3A submitted 2022-04-07 cs.SE cs.AIcs.LG

Violet Turri , Rachel Dzombak , Eric Heim , Nathan VanHoudnos , Jay Palat , Anusha Sinha This is my paper · ORCID
classification cs.SEcs.AIcs.LG
keywords oftensystemtestingaccuracyadditionallyadvocatesapproachapproaches
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Current test and evaluation (T&E) methods for assessing machine learning (ML) system performance often rely on incomplete metrics. Testing is additionally often siloed from the other phases of the ML system lifecycle. Research investigating cross-domain approaches to ML T&E is needed to drive the state of the art forward and to build an Artificial Intelligence (AI) engineering discipline. This paper advocates for a robust, integrated approach to testing by outlining six key questions for guiding a holistic T&E strategy.

Discussion (0). Continue with ORCID to comment.

Pith tools