StreamBP partitions backpropagation over the sequence dimension to compute exact gradients with much lower activation memory, extending maximum trainable sequence length by roughly 3-5x over gradient checkpointing.
Open R1: A fully open reproduction of deepseek-r1, January 2025
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
StreamBP partitions backpropagation over the sequence dimension to compute exact gradients with much lower activation memory, extending maximum trainable sequence length by roughly 3-5x over gradient checkpointing.