L3TC, a 3.2M-parameter RWKV-based compressor with an outlier-bypass tokenizer, saves roughly 50% of gzip's bitrate on enwik9 and decodes at megabyte-per-second rates.
HiRA is configured with a single bypass branch and a rank set to four times the main branch’s dimension, which has been proved as the optimal configuration in the main paper
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
L3TC: Leveraging RWKV for Learned Lossless Low-Complexity Text Compression
L3TC, a 3.2M-parameter RWKV-based compressor with an outlier-bypass tokenizer, saves roughly 50% of gzip's bitrate on enwik9 and decodes at megabyte-per-second rates.