GRAPE uses a minimax group-DRO scheme to reweight both source domains and target tasks during pretraining, improving multi-task reasoning and low-resource language modeling.
Redpajama: An open source recipe to reproduce llama training dataset, 2023
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
GRAPE: Optimize Data Mixture for Group Robust Multi-target Adaptive Pretraining
GRAPE uses a minimax group-DRO scheme to reweight both source domains and target tasks during pretraining, improving multi-task reasoning and low-resource language modeling.