DIDS dynamically reweights training domains using gradient clustering and a Fisher Information-guided KL metric, and reports matching or better LLM benchmark scores with 10% of the data.
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guil- laume Leclerc, and Aleksander Madry
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
DIDS: Domain Impact-aware Data Sampling for Large Language Model Training
DIDS dynamically reweights training domains using gradient clustering and a Fisher Information-guided KL metric, and reports matching or better LLM benchmark scores with 10% of the data.