REVIEW 4 cited by
Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We enhance the autonomy of the continuous active learning method shown by Cormack and Grossman (SIGIR 2014) to be effective for technology-assisted review, in which documents from a collection are retrieved and reviewed, using relevance feedback, until substantially all of the relevant documents have been reviewed. Autonomy is enhanced through the elimination of topic-specific and dataset-specific tuning parameters, so that the sole input required by the user is, at the outset, a short query, topic description, or single relevant document; and, throughout the review, ongoing relevance assessments of the retrieved documents. We show that our enhancements consistently yield superior results to Cormack and Grossman's version of continuous active learning, and other methods, not only on average, but on the vast majority of topics from four separate sets of tasks: the legal datasets examined by Cormack and Grossman, the Reuters RCV1-v2 subject categories, the TREC 6 AdHoc task, and the construction of the TREC 2002 filtering test collection.
Forward citations
Cited by 4 Pith papers
-
Finite-Sample Coverage Audits for High-Recall Candidate Generation: Certification and Learning-Theoretic Design
Excluded-pool auditing is minimax rate-optimal for certifying missed relevant mass: any valid zero-miss certificate needs Ω(N0/m) excluded labels, and exact binomial/hypergeometric bounds achieve this rate.
-
A Generalised and Adaptable Reinforcement Learning Stopping Method
GRLStop is a reinforcement learning stopping rule for Technology Assisted Review whose reward function lets one model serve multiple target recall levels and user-selected recall/cost tradeoffs, and it improves or mat...
-
Learning to Ask: Question-based Sequential Bayesian Product Search
QSBPS asks yes/no questions about entity presence, learned from past users, to sequentially narrow candidate products and beat retrieval baselines in Amazon simulations.
-
Overview of the TREC 2022 deep learning track
The 2022 TREC Deep Learning Track yielded a reusable 76-topic passage test collection in which pretrained neural rankers again outperformed traditional retrieval, and the best run was a sparse SPLADE system, not a den...
Discussion (0). Continue with ORCID to comment.