REVIEW 2 cited by
Testing for concept shift online
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This note continues study of exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the observations. Such processes can be used for detecting violations of the IID assumption, which is commonly made in machine learning. Violations of the IID assumption are sometimes referred to as dataset shift, and dataset shift is sometimes subdivided into concept shift, covariate shift, etc. Our primary interest is in concept shift, but we will also discuss exchangeability martingales that decompose perfectly into two components one of which detects concept shift and the other detects what we call label shift. Our methods will be based on techniques of conformal prediction.
Forward citations
Cited by 2 Pith papers
-
WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
WCTMs generalize conformal test martingales to test non-exchangeability nulls, enabling adaptation to mild covariate shifts, fast detection of harmful shifts, and root-cause diagnosis.
-
Adapting to Fragmented and Evolving Data: A Fisher Information Perspective
FADE adapts to sequential covariate shift by combining a Fisher-weighted KL shift signal with a smoothed Fisher information regularizer, reporting gains on vision, text, and tabular benchmarks.
Discussion (0). Continue with ORCID to comment.