REVIEW 2 cited by
DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advances have led to the availability of many pre-trained language models (PLMs); however, a question that remains is how much data is truly needed to fine-tune PLMs for downstream tasks? In this work, we introduce DEFT-UCS, a data-efficient fine-tuning framework that leverages unsupervised core-set selection to identify a smaller, representative dataset that reduces the amount of data needed to fine-tune PLMs for downstream tasks. We examine the efficacy of DEFT-UCS in the context of text-editing LMs, and compare to the state-of-the art text-editing model, CoEDIT. Our results demonstrate that DEFT-UCS models are just as accurate as CoEDIT, across eight different datasets consisting of six different editing tasks, while finetuned on 70% less data.
Forward citations
Cited by 2 Pith papers
-
Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities
BIDS, a balanced influence-based data selection algorithm using per-task normalization and iterative greedy selection, improves balanced multi-capability instruction tuning and can outperform full-dataset training on ...
-
Generalizing Large Language Model Usability Across Resource-Constrained
The dissertation shows that text-centric prompting, inference-time optimization, and correct-by-construction synthetic data can improve LLM robustness and Verilog code generation under resource constraints.
Discussion (0). Continue with ORCID to comment.