{"id":"408b18c8-857a-49d5-b4e4-bf47c74a5226","arxiv_id":"2505.16256","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A unified RWKV-based model compresses both images and text losslessly with a shared vocabulary, modality-specific contextual layers, and mixture-of-experts routing, matching or improving on much larger SOTA compressors at a fraction of the size.","lead":"DualComp is a single lightweight neural network that losslessly compresses both images and text, replacing separate compressors. It matches or beats much larger language-model compressors on some benchmarks while running orders of magnitude faster on ordinary CPUs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'on par with SOTA LLM-based methods' is contradicted by Table 2 for text (DualComp-130M: 1.107 bits/byte vs Llama3-8B: 0.722 on enwik9), and the headline 9% image gain is limited to Kodak with an unverified P2LLM baseline.","rationale":"The reader's weakest assumption is benchmark comparability for the P2LLM baseline, which is a legitimate uncertainty. I agree that the 9% Kodak gain is fragile: it is one dataset, and P2LLM's 2.830 is not independently reproduced. However, I see an even more direct, internally verifiable problem: the abstract's claim of being 'on par with SOTA LLM-based methods' for text is contradicted by the paper's own Table 2, where DualComp-130M is 53% worse than Llama3-8B on enwik9. This does not require external baseline verification; it is a comparison within the paper's own results. The paper's Section 4.2 quietly retreats to 'outperforms L3TC,' a lightweight compressor, which is a materially weaker claim. For images, the 9% gain is real on Kodak but not on the other three datasets, and the paper acknowledges the Kodak result may reflect train/test resolution matching. Together, these issues mean the central claim as worded is overstated, though the underlying architecture may still be a useful lightweight dual-modality compressor. Since the reader already assigned CONDITIONAL with moderate confidence, my concern reinforces that verdict rather than changing it. The concrete test of a shared-protocol text comparison would settle the parity claim directly, and re-running P2LLM with matched preprocessing would settle the image margin.","tokens_in":18989,"tokens_out":7146,"duration_ms":53623,"concrete_test":"Reproduce the text comparison on enwik9 under a single, shared protocol: evaluate DualComp-130M, L3TC, tszip, Llama3-8B, and NNCP on the same 1GB enwik9 segment using the same tokenizer and context length. If the ratio of DualComp-130M's bits/byte to Llama3-8B's remains near 1.53 (1.107 vs 0.722), the abstract's 'on par with SOTA LLM-based methods' claim is unsupported and must be revised to 'comparable to lightweight learned compressors.' Separately, re-run P2LLM on Kodak using DualComp-I's RGB patch tokenization and context length, and also train DualComp-I on P2LLM's exact training data; if the Kodak margin shrinks below 5% or reverses, the 9% claim is not robust to evaluation protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim as stated in the Abstract — that DualComp 'achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets' — is not supported by the paper's own numbers for text. In Table 2, DualComp-130M compresses enwik9 to 1.107 bits/byte, whereas Llama3-8B achieves 0.722 and Chinchilla-70B achieves 0.642; even classical learned compressors NNCP (0.853) and CMIX (0.879) are substantially better. The 53% gap to Llama3-8B is not 'on par' under any reasonable reading. The paper's Section 4.2 actually narrows the claim to outperforming L3TC, a lightweight text compressor, which is a different statement. For images, the headline 9% improvement is restricted to the Kodak dataset (24 images): DualComp-I-96M reaches 2.571 bits/byte vs P2LLM's reported 2.830, but on CLIC-P it only ties (2.350 vs 2.350), on CLIC-M it is 1% worse (2.110 vs 2.080), and on DIV2K it is 2% worse (2.547 vs 2.510). Moreover, P2LLM's value is taken from the original paper [9] without reproducing its training data, preprocessing, tokenization, or context length; the paper's own backbone-selection experiments use a different grayscale-ASCII protocol, so cross-paper comparability of bits/byte is nontrivial. The load-bearing assumption is therefore not the architecture but the benchmark framing: the headline parity claim is contradicted internally for text, and the single-dataset image gain depends on an unverified external baseline. This makes the strongest claim, as worded, overbroad and not fully supported by the evidence presented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes DualComp, a lightweight dual-modality (image and text) lossless compression model built on the RWKV-7 backbone. The method introduces modality-unified tokenization (a shared vocabulary of 256 image sub-pixel tokens and 16K SentencePiece BPE text tokens), modality-switching contextual learning (separate R/K/V projections per modality), and a modality-routing mixture-of-experts replacing the final MLP, plus a high-rank reparameterization training strategy. Experiments on Kodak, CLIC-P, CLIC-M, DIV2K, and enwik9 report bits/byte for several model scales, with CPU/NPU/GPU speed measurements. The abstract claims parity with SOTA LLM-based compressors for text and images, and a 9% improvement over the previous best image compressor on Kodak with 1.2% of the parameters.","tokens_in":19429,"tokens_out":5629,"duration_ms":41335,"significance":"The paper is practically motivated and the experimental effort is substantial: the authors reproduce classical baselines, measure inference speed on three platforms, and provide ablations for each proposed component. The single-modality variant, DualComp-I, demonstrates that reparameterization alone yields consistent gains on Kodak. However, the headline claims as stated are not supported by the paper's own tables: text compression at 1.107 bits/byte on enwik9 is far from the 0.722 bits/byte of Llama3-8B and 0.642 of Chinchilla-70B, and the 9% image gain is limited to Kodak and rests on cross-paper comparison with P2LLM. The architectural ideas are worth publishing after claim revisions and stronger baseline verification.","major_comments":[{"comment":"The claim that DualComp 'achieves compression performance on par with the SOTA LLM-based methods for both text and image datasets' is internally contradicted for text. DualComp-130M compresses enwik9 to 1.107 bits/byte, while Table 2 lists Llama3-8B at 0.722 and Chinchilla-70B at 0.642; even classical learned compressors NNCP (0.853) and CMIX (0.879) are substantially better. Section 4.2 in fact narrows the claim to outperforming L3TC (1.280), which is a different statement. The abstract, Figure 1, and Figure 5 should be revised to reflect the actual scope of the text result, or the claim must be removed.","section":"Abstract; Section 4.2; Table 2"},{"comment":"The headline image result, a 9% gain over P2LLM on Kodak, is not supported as a general SOTA claim. DualComp-I-96M achieves 2.571 bits/byte vs. P2LLM's reported 2.830 on Kodak, but on CLIC-P it ties (2.350 vs 2.350), on CLIC-M it is 1% worse (2.110 vs 2.080), and on DIV2K it is 2% worse (2.547 vs 2.510). Moreover, the P2LLM values are taken from the original paper without retraining on the same data or preprocessing; the table's own footnote states that unmarked values are 'claimed by their original papers,' and the paper's backbone-selection experiments use a different grayscale-ASCII protocol. The 9% claim should be stated as a Kodak-specific improvement conditional on baseline comparability, or the baseline must be reproduced under identical conditions.","section":"Section 4.2; Table 2; Table 6"},{"comment":"Several hyperparameters are selected using the evaluation datasets. The auxiliary loss weight λ=0.01 (Section 3.5), the MoE expert count, top-k, hidden factor, and reparameterization rank are all chosen based on bits/byte on Kodak and enwik9 (e.g., Table 4 uses DualComp-0.3M evaluated on Kodak and enwik9). Tuning on the test sets makes the reported numbers optimistic and weakens the generality of the ablation conclusions. The authors should either use a held-out validation set for these choices or report sensitivity of the final models to these hyperparameters.","section":"Section 4.4; Table 4; Section 3.5"},{"comment":"The training protocol for the main DualComp model is described only briefly (5,500 ImageNet images and 100MB of enwik8; three-stage training with specified epochs and learning rates), and no code or trained models are provided. Given that the central comparisons are sensitive to preprocessing and tokenization, the lack of a detailed protocol for image patch order, context length, and arithmetic coding implementation makes independent verification difficult. At minimum, the authors should provide an open-source implementation or exact protocol details.","section":"Section 4.1; Section 6.2"}],"minor_comments":[{"comment":"The target distribution q in the cross-entropy term L = -Σ q log p is never defined; please clarify whether q is the one-hot target distribution or something else.","section":"Equation (1)"},{"comment":"There is a typo: 'reprodeuced' should be 'reproduced', and 'presents' should be 'indicates' in the footnote about inference speeds.","section":"Table 2 footnote"},{"comment":"The sentence 'DualComp reaches comparable compression efficiency to SOTA approaches' is ambiguous. Given the text gap to LLM-based methods, specify that the comparison is against lightweight practical compressors such as L3TC, rather than all SOTA methods.","section":"Section 4.2"},{"comment":"The backbone-selection experiment uses grayscale ASCII image encoding following [6], which differs from the final sub-pixel tokenization used in DualComp; a brief justification that the backbone ranking transfers to the final tokenization would be helpful.","section":"Section 3.1, Table 1"},{"comment":"The statement that masking non-target logits 'brings up to 20% gains' is vague; please specify the baseline (which model, which dataset) and report the exact numbers in the ablation table.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a genuine architectural contribution and thorough ablations, but the public-facing claims overstate the results. I recommend major revision with explicit request to either soften the parity claims to match Table 2 or add a properly reproduced baseline. The authors should also avoid tuning hyperparameters on the evaluation datasets. If the claims are revised and the P2LLM comparison is verified or appropriately caveated, the paper could be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read DualComp. The architecture is a sensible engineering combination—RWKV-7 backbone, shared vocabulary with logit masking, modality-switched R/K/V projections, a final routing MoE, and L3TC-style reparameterization—and the ablations genuinely support each piece. The modality-routing MoE is the most interesting bit; the expert-usage plots suggest the router does learn different behavior for images than text. If you work on lightweight learned compression, this is worth a look.\n\nWhat the paper does well: the ablations are honest and fairly complete for the design choices (number of experts, top-k, reparam rank). The dual-modality parameter-sharing story is concrete—about one-third modality-specific, two-thirds shared—and the 130M model handles both modalities with modest overhead. The speed numbers on desktop CPU and iPhone NPU are useful for deployment.\n\nThe soft spots are in the claims, not the mechanics. The abstract says 'on par with SOTA LLM-based methods for both text and image.' Their own Table 2 shows DualComp-130M at 1.107 bits/byte on enwik9 vs Llama3-8B's 0.722 and NNCP's 0.853. That is not on par; it is roughly 50% worse. Section 4.2 quietly narrows the text claim to beating L3TC, which is fine, but the abstract and Figure 1 do not match. The 9% image gain over P2LLM on Kodak is real in their table, but it flips to ties or small losses on CLIC-P, CLIC-M, and DIV2K. And the P2LLM number is taken from the published paper, not reproduced with the same training data or preprocessing, so the Kodak margin is fragile. The stress-test note is right: the load-bearing assumption is benchmark comparability, not the architecture. I would also flag that Deletang et al. and MSDZip are cited but never compared, which weakens the 'first unified' claim; a comparison against MSDZip would have been informative.\n\nNone of this is fatal. The design is sound, the ablations support the components, and the speed/complexity story is attractive. The abstract and conclusion need to be rewritten to claim what the data shows: competitive with lightweight learned compressors, with dual-modality flexibility, and a Kodak-specific edge over P2LLM that needs a properly controlled comparison to confirm. Code would help a lot; without it the cross-paper comparison stays shaky.\n\nVerdict: send it to review. A good referee can push the authors on the claims and maybe get the comparison done properly. This is a solid paper that would benefit from revision; it is not a desk reject and not a breakthrough.","headline":"Solid lightweight dual-modality compressor with honest ablations; the abstract overclaims parity with LLM compressors and the 9% Kodak gain rests on an unverified baseline.","tokens_in":20004,"tokens_out":3045,"would_cite":true,"duration_ms":21085,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DualComp is a single lightweight neural compressor that losslessly handles both images and text, matching billion-parameter language-model compressors while running near real time on a desktop CPU.","keywords":["dual-modality lossless compression","learned image compression","learned text compression","mixture-of-experts","modality-switching","RWKV","reparameterization training","entropy coding"],"falsifier":"Retrain the previous best image compressor, P2LLM, under DualComp-I's exact conditions, namely the same 5,500 ImageNet training images, the same 16×16×3 patch tokenization, and the same 16-epoch schedule, then measure its bits per byte on Kodak; if it reaches 2.571 or lower, the claimed 9% advantage disappears.","tokens_in":18780,"feed_emoji":"📦","tokens_out":9826,"duration_ms":78717,"temperature":0.7,"pith_summary":"This paper tries to establish that a single small neural network can losslessly compress both images and text, a job that previously required either separate single-modality models or billion-parameter language models. The proposed system, DualComp, builds on a lightweight recurrent backbone and adds three adjustments: a shared token vocabulary for both image sub-pixels and text, separate contextual-prediction layers that activate depending on the input modality, and a small mixture-of-experts layer that routes each token to specialized sub-networks. A reparameterization trick gives the model extra learning capacity during training without enlarging the model at deployment. The paper reports that DualComp matches far larger language-model compressors on both image and text benchmarks, and that its image-only variant beats the best previous image compressor on the Kodak dataset by about 9% while using about 1.2% of the parameters and decoding near real time on a desktop CPU. If true, learned lossless compression could be unified and practical, not just powerful.","feed_headline":"One small model compresses images and text losslessly","feed_subtitle":"It matches billion-parameter compressors on quality at a fraction of the size and speed.","key_machinery":"The load-bearing mechanism is the pair of modality-adaptive modules grafted onto the RWKV-7 backbone. Modality-switching contextual learning splits the R/K/V projection layers of each Time Mixing block into separate image and text sets, so each modality builds its own contextual predictions while sharing the rest of the block; this is what lets one model hold two different statistical models. The modality-routing mixture-of-experts replaces the final MLP with three smaller experts and a learned router that activates the top two per token, letting image and text tokens follow different nonlinear paths. Underneath both sits a unified tokenizer: text uses a 16K SentencePiece BPE vocabulary while image sub-pixels occupy a 256-token subspace, merged into one vocabulary with non-target logits masked before arithmetic coding. The reparameterization training strategy adds high-rank bypass branches to R/K/V projections during training and merges them away at inference, buying capacity without extra runtime cost.","core_discovery":"The paper's central claim is that modality heterogeneity can be handled inside a single model rather than by separate modality-specific compressors or by flattening every input into text. DualComp tokenizes images as sequences of sub-pixel values and text as BPE tokens within one shared vocabulary, then applies modality-switching contextual learning: separate R/K/V (receptance/key/value) projection layers in each RWKV-7 Time Mixing block are activated depending on whether the input is an image or text, while the output and MLP layers stay shared. In the final block, a modality-routing mixture-of-experts with three experts replaces the MLP, and a learned router selects which experts process each token. A high-rank reparameterization branch on the R/K/V projections is trained and then merged away, so inference keeps a compact single-path structure. The reported outcome is that the 130M-parameter dual-modality model reaches 2.834 bits/byte on Kodak and 1.107 bits/byte on enwik9, comparable to large language model compressors, while the 96M-parameter image-only variant reaches 2.571 bits/byte on Kodak, about 9% better than the 8B-parameter P2LLM.","pith_inferences":["An independent head-to-head rerun, retraining the 8B-parameter P2LLM baseline on the same 5,500 ImageNet images and patch preprocessing used for DualComp-I, would determine whether the 9% Kodak margin is an architecture effect or a training-protocol effect; the paper compares against the baseline's published numbers.","The same modality-switching plus routing recipe could be extended to audio and video by adding tokenizers and modality branches to the shared backbone, with the MoE router learning which experts serve which modality; the authors list this as future work rather than a demonstrated result.","The routing statistics themselves, with images showing higher expert variability than text, could be read as a per-modality complexity signal and used to allocate more parameters to harder modalities, a step the paper does not take."],"forward_implications":["A single DualComp model can replace separate image and text compressors in a deployment, cutting memory and system complexity because roughly two-thirds of its parameters are shared across modalities.","Billion-parameter LLM compressors are not needed for strong lossless ratios; the gains here come from structure, namely modality-specific context and routing, rather than model scale.","Near-real-time decoding on desktop CPUs and mobile NPUs, with small models reaching hundreds of kilobytes per second, makes learned lossless compression practical for everyday file and image archiving.","If the Kodak result transfers, the image-only variant sets a new state of the art for learned lossless image compression at 2.571 bits/byte with 96M parameters.","Adding a new modality becomes a modular operation: attach a tokenizer, modality-specific R/K/V branches, and MoE adaptation to the same shared backbone."],"supporting_citations":[{"why":"Supplies the arithmetic coding step that converts predicted context probabilities into the final lossless bitstream.","marker":"[3]"},{"why":"Provides the multi-modal LLM compression baseline whose ASCII-text treatment of images DualComp is designed to beat.","marker":"[6]"},{"why":"Is the previous best learned image compressor whose Kodak, CLIC, and DIV2K numbers are the head-to-head basis for DualComp-I's claimed 9% improvement.","marker":"[9]"},{"why":"Supplies the high-rank reparameterization training strategy and the L3TC text compressor that DualComp extends and compares against.","marker":"[10]"},{"why":"Provides the lightweight RWKV-7 backbone whose Time Mixing module carries the modality-switching contextual learning.","marker":"[19]"},{"why":"Provides the SentencePiece BPE tokenizer used for text tokens and the shared-vocabulary design in modality-unified tokenization.","marker":"[42]"},{"why":"Supplies the 5,500 ImageNet training images used to train DualComp and DualComp-I.","marker":"[43]"},{"why":"Defines the Kodak image benchmark whose 2.571 bits/byte result backs the headline image compression claim.","marker":"[44]"},{"why":"Defines the enwik9 text benchmark used for the text compression results.","marker":"[50]"}],"fun_headline_variants":["DualComp: one tiny model, lossless images and text","Single lightweight model losslessly compresses both images and text","DualComp: unified lossless compression for images and text","Small model matches billion-param compressors losslessly","DualComp: lossless images and text, 200KB/s on CPU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed 9% improvement over the previous best image compressor depends on treating that compressor's published compression numbers as directly comparable to DualComp-I's, even though the two models were trained on different data and with different preprocessing.","fun_headline_variants_meta":{"raw":{"variants":["DualComp: one tiny model, lossless images and text","Single lightweight model losslessly compresses both images and text","DualComp: unified lossless compression for images and text","Small model matches billion-param compressors losslessly","DualComp: lossless images and text, 200KB/s on CPU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000452,"raw_usage":{"total_tokens":2316,"prompt_tokens":1027,"completion_tokens":1289,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":1201}},"tokens_in":643,"tokens_out":1289,"duration_ms":8081,"temperature":1.0,"reasoning_tokens":1201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:04:20.882698+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the previous best image compressor, P2LLM, under DualComp-I's exact conditions, namely the same 5,500 ImageNet training images, the same 16×16×3 patch tokenization, and the same 16-epoch schedule, then measure its bits per byte on Kodak; if it reaches 2.571 or lower, the claimed 9% advantage disappears.","supporting_citations":[{"cited_title":"Howard and Jeffrey Scott Vitter","cited_arxiv_id":null,"evidence_quote":"Supplies the arithmetic coding step that converts predicted context probabilities into the final lossless bitstream."},{"cited_title":"Language modeling is compression","cited_arxiv_id":null,"evidence_quote":"Provides the multi-modal LLM compression baseline whose ASCII-text treatment of images DualComp is designed to beat."},{"cited_title":"L3tc: Leveraging rwkv for learned lossless low-complexity text compression","cited_arxiv_id":null,"evidence_quote":"Supplies the high-rank reparameterization training strategy and the L3TC text compressor that DualComp extends and compares against."},{"cited_title":"Kodak lossless true color image suite","cited_arxiv_id":null,"evidence_quote":"Defines the Kodak image benchmark whose 2.571 bits/byte result backs the headline image compression claim."},{"cited_title":"enwik9: Complete english wikipedia dump (xml)","cited_arxiv_id":null,"evidence_quote":"Defines the enwik9 text benchmark used for the text compression results."}],"review_version":1}