{"id":"520b69e2-abce-43f2-b904-d8fc53097d55","arxiv_id":"2509.09836","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CoDiCodec unifies continuous and discrete audio compression in one consistency-trained autoencoder, using FSQ-dropout to serve both continuous ~11 Hz embeddings and 2.38 kbps discrete tokens.","lead":"CoDiCodec is a single audio autoencoder that compresses 44.1 kHz stereo audio into both low-rate continuous embeddings and discrete tokens from the same model, trained end-to-end with one consistency loss. It reports better reconstruction quality on distribution-based metrics than several existing continuous and discrete codecs at similar bitrates, though without error bars and with a limited baseline set.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FAD is a distribution-level metric, not a per-sample reconstruction measure; Table 2's headline 'outperforms in reconstruction audio quality' rests on it while SI-SDR/ViSQOL show CoDiCodec is not the best reconstructor.","rationale":"The strongest claim is an outperformance claim about reconstruction audio quality. The evidence in Table 2 includes both distributional (FAD/FAD_clap) and per-sample (SI-SDR/ViSQOL) metrics. The paper itself notes the per-sample metrics do not favor CoDiCodec. Therefore the only evidence for the 'outperforms' wording is FAD. But FAD does not measure whether each reconstruction matches its source; it measures whether the whole set of reconstructions has similar feature statistics to the whole set of originals. This is a standard generative-quality metric, not a codec-fidelity metric. The internal inconsistency is explicit in §5.3. The reader's condition was largely based on metric reliability and lack of error bars; our stress test sharpens this to construct validity, which is more load-bearing: even with infinite data and no noise, FAD cannot certify reconstruction fidelity. A MUSHRA test would directly test whether the FAD advantage corresponds to perceived fidelity. If it does not, the central claim is false as worded. The method itself (unified continuous/discrete representations, FSQ-dropout, parallel decoding) is not called into question; only the comparison claim needs re-scoping.","tokens_in":12567,"tokens_out":7487,"duration_ms":83914,"concrete_test":"Conduct a paired MUSHRA listening test on 30 MusicCaps excerpts, each reconstructed by CoDiCodec-discrete (2.38 kbps) and DAC (2.67 kbps), with hidden reference and anchor, asking listeners to rate fidelity to the original. If CoDiCodec's mean fidelity score is not significantly higher than DAC's (or than DAC 8k on a second arm), the FAD-based 'outperforms in reconstruction audio quality' claim is not supported and should be reworded to 'distributional audio quality' or supported by a per-sample metric (e.g., ViSQOL with confidence intervals).","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 2's headline comparison is built on FAD/FAD_clap, but these are distribution-level metrics: they compute a Fréchet distance between feature sets of the original and reconstructed MusicCaps clips, with no per-sample correspondence. A decoder that outputs plausible music statistically similar to MusicCaps, even if it ignores the specific input, can achieve low FAD; the metric therefore does not establish reconstruction. This is not hypothetical: the paper's own Table 2 shows CoDiCodec has SI-SDR of -0.01 to -0.95 dB vs +9.48 dB for DAC 8k and ViSQOL 3.89-3.95 vs 4.21 for DAC 8k, and §5.3 concedes 'general audio quality is thus prioritised over reconstruction of the exact same signal.' Using FAD to support 'outperforms ... in terms of reconstruction audio quality' is internally inconsistent. The cited [52] result that FAD-human correlation is embedding-dependent further undermines FAD_clap as a primary yardstick. This concern is about the central claim, not about the existence of a unified continuous/discrete model, which is credible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoDiCodec, a single audio autoencoder that produces both compressed continuous embeddings (~11 Hz) and discrete FSQ tokens (2.38 kbps) from the same model. The method builds on Music2Latent/Music2Latent2 and uses summary embeddings, a consistency-training loss, and a newly introduced FSQ-dropout technique that bypasses quantization with probability p during training so that one decoder can operate on both discrete and continuous latents. The architecture also supports autoregressive and a novel parallel decoding strategy. The authors report reconstruction/audio-quality metrics (FAD, FAD_clap, SI-SDR, ViSQOL) on MusicCaps, claiming that CoDiCodec outperforms existing continuous and discrete autoencoders at similar bitrates in reconstruction audio quality, and they provide ablation studies and inference speed comparisons.","tokens_in":12869,"tokens_out":3894,"duration_ms":48913,"significance":"If the claims hold, the contribution is practically valuable: a single codec feeding both continuous diffusion-style generators and discrete autoregressive generators would remove a major architectural choice in audio generation. The FSQ-dropout idea is simple and plausible, and the paper includes a thorough ablation, a code/weights release link, and comparisons to several existing systems. The manuscript also makes a credible methodological step by showing that a consistency model can be trained end-to-end with one loss while supporting both latent types. However, the headline comparative claim depends on distribution-level metrics (FAD/FAD_clap) for what is presented as reconstruction quality, and the baseline selection omits standard codecs such as EnCodec and SoundStream; the evidence as presented is therefore not yet fully convincing even though the core model is defensible.","major_comments":[{"comment":"The central claim that CoDiCodec 'outperforms existing continuous and discrete autoencoders ... in terms of reconstruction audio quality' rests on FAD and FAD_clap, which are distribution-level metrics computed on feature sets without per-sample correspondence. Low FAD can be achieved by a decoder that produces plausible music statistically similar to the evaluation set without reconstructing the specific input. This is not hypothetical: Table 2 shows CoDiCodec has SI-SDR of -0.01 to -0.95 dB and ViSQOL of 3.88–3.95, while DAC 8k achieves +9.48 dB and 4.21 on the same per-sample metrics. The paper itself concedes in §5.3 that 'general audio quality is thus prioritised over reconstruction of the exact same signal.' The headline claim should either be rephrased to 'distribution-level audio quality' or be supported by per-sample reconstruction metrics at matched bitrates (or a subjective li","section":"§5.3 / Table 2"},{"comment":"The baseline set omits two standard discrete neural codecs discussed in the introduction: SoundStream and EnCodec. Since the paper claims to outperform 'existing discrete autoencoders at similar bitrates', these are the most direct competitors. Including EnCodec (e.g., 1.5/3 kbps) and SoundStream (e.g., 3 kbps) would make the comparison credible. The current table compares only DAC (2.67/8 kbps) among RVQ codecs, and even DAC 8k is not clearly beaten on SI-SDR/ViSQOL.","section":"§5.3 / Table 2"},{"comment":"Ablation results are reported without confidence intervals, multiple seeds, or statistical significance tests. Several differences between configurations are very small—e.g., continuous FAD_clap for '+128 lat.' is 0.0154 vs. 0.0161 for 'd.o. p=0.75', and discrete FAD_clap differs by 0.0005–0.0009 across FSQ-dropout probabilities. Without variance estimates it is impossible to know whether the design choices (particularly FSQ-dropout p and the 128-latent reshaping) are actually responsible for the reported improvements. Since these ablations motivate the final architecture, this is a load-bearing issue for the internal validity of the paper.","section":"§5.1 / Table 1"}],"minor_comments":[{"comment":"Typographical errors: 'viasummary' and 'representa tions' in the abstract; 'baslines' in §5.3. These should be corrected.","section":"Abstract / Title"},{"comment":"The abbreviation 'd.o.' in Table 1 is not defined; expand to 'FSQ-dropout' there or define it near Eq. (4).","section":"§4.1 / Eq. (4)"},{"comment":"The stereo/mono column is not fully explained. Some baselines are marked mono and CoDiCodec is stereo; it is not clear whether FAD/FAD_clap are computed on downmixed mono for all systems and whether stereo input affects the comparison. This should be stated in the metric description.","section":"§5.3 / Table 2"},{"comment":"The 'Random Mixing' augmentation adds two random training samples. It would be helpful to clarify whether this is applied to the raw waveform or STFT and whether the summary embeddings/decoder targets are also mixed consistently; otherwise the augmentation may be ill-posed.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong systems paper from the Music2Latent line, and the core idea—a single consistency-trained codec with FSQ-dropout—is likely sound. However, the headline comparison to baselines is currently supported only by distribution-level metrics, and the absence of EnCodec/SoundStream and any statistical grounding makes the comparative claims brittle. These are fixable within the manuscript's scope by rephrasing the claim and adding a few experiments/seeds. I would not reject, but the authors need to address the reconstruction-versus-distribution issue explicitly before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look: the genuinely new piece is FSQ-dropout, which lets one consistency-trained autoencoder emit both discrete FSQ tokens and more expressive continuous latents from the same model. The ablation chain is honest and incremental, and the parallel decoding trick seems to work. The authors are not overselling the architecture itself — I buy the central claim that a single codec can serve both continuous diffusion and discrete autoregressive pipelines.\n\nThe soft spot is the comparison story. The 'outperforms in reconstruction quality' headline rests almost entirely on FAD and FAD_clap, which are distribution-level metrics. They do not measure per-sample reconstruction. Table 2 shows the problem: CoDiCodec's SI-SDR is around 0 dB or negative, and ViSQOL around 3.9, while DAC at 8 kbps hits +9.48 dB and 4.21. The paper even concedes that 'general audio quality is prioritised over reconstruction of the exact same signal.' So using FAD to support a reconstruction claim is internally inconsistent. A model that ignores the input but produces plausible-sounding music can score well on FAD. That doesn't invalidate the unification, but it does mean the claim as written is not established.\n\nOther issues are milder: no confidence intervals or multiple seeds, a baseline set missing EnCodec and SoundStream, and the downstream generative experiment uses an internal 100k-sample dataset that isn't released. Pretrained weights are available, which is good, but not the code. The citation pattern is fine; self-citations are to the authors' own Music2Latent lines and are appropriate given the direct lineage.\n\nBottom line: the core method is credible, the ablations are useful, and the unified continuous/discrete capability is a real step forward for latent audio work. What needs fixing is the evaluative framing: either add per-sample reconstruction metrics or a listening test, and report uncertainty. As is, it deserves a serious referee but should not be accepted on the current comparison.\n\nI'd bring it to a reading group focused on audio generation, and I'd cite it if I were building on codecs. Send it to peer review — with the expectation of revision.","headline":"Credible and useful unified codec, but the headline 'reconstruction quality' claim is built on a distribution-level metric that the paper's own per-sample numbers contradict.","tokens_in":13352,"tokens_out":1453,"would_cite":true,"duration_ms":18446,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoDiCodec proposes a single audio autoencoder that yields both compressed continuous embeddings (~11 Hz) and discrete tokens (2.38 kbps) from one end-to-end consistency-trained model, and reports better reconstruction quality than existing","keywords":["audio compression","consistency models","finite scalar quantization","discrete tokens","continuous latent representations","summary embeddings","parallel decoding","autoregressive decoding"],"falsifier":"Run a blind listening test on held-out music where human listeners rate reconstruction quality of CoDiCodec against the top-scoring baseline; if CoDiCodec's FAD and FAD_clap advantage is not reflected in listener preference, the central outperformance claim is not established.","tokens_in":12441,"feed_emoji":"🎧","tokens_out":4782,"duration_ms":48215,"temperature":0.7,"pith_summary":"The paper asks whether one compressed audio representation can serve both continuous generative models, like diffusion, and discrete ones, like autoregressive transformers, without sacrificing quality. It introduces CoDiCodec, an autoencoder trained end-to-end with a single consistency loss, which emits summary embeddings that can be read either as ~11 Hz continuous vectors or, after finite scalar quantization, as 2.38 kbps discrete tokens. The same decoder accepts both forms, enabled by FSQ-dropout, which randomly bypasses quantization during training. On reconstruction-quality metrics, CoDiCodec reports lower FAD and FAD_clap than existing continuous and discrete codecs at comparable bitrates, with a parallel decoding strategy that also speeds up inference. A sympathetic reader would care because it removes the need to choose a latent type before building a generative model.","feed_headline":"One codec yields both continuous and discrete audio latents","feed_subtitle":"Same consistency-trained autoencoder feeds diffusion and autoregressive models, with 2.38 kbps discrete tokens and ~11 Hz continuous embeddi","key_machinery":"The load-bearing object is the consistency-trained decoder conditioned on summary embeddings, plus FSQ-dropout. Summary embeddings are learned tokens that pool global context from a chunk, replacing a temporal sequence so that 128 latent vectors of dimension 4 can be reshaped freely across time and channels. Finite Scalar Quantization (FSQ) rounds each dimension to one of 11 levels, creating a discrete codebook without auxiliary losses. FSQ-dropout randomly feeds the unrounded tanh(z) to the upsampler during training, so one decoder learns to accept both continuous and discrete latents. The consistency loss trains the decoder to map noisy spectrograms of adjacent chunk pairs back to clean on","core_discovery":"The central discovery is that continuous and discrete compressed audio can come from one model rather than two. CoDiCodec encodes a spectrogram chunk into 128 four-dimensional summary embeddings; the same latents can be decoded as continuous values or as FSQ tokens with an implicit codebook of 14,641 entries, at 2.38 kbps for stereo 44.1 kHz audio. The paper shows that during training, randomly bypassing the FSQ rounding step with probability p=0.75 keeps the continuous distribution spread across [-1,1] instead of collapsing near quantization levels, while preserving discrete-token quality. With this single model, autoregressive decoding and a newly proposed parallel decoding with shifted ch","pith_inferences":["If the FAD ranking holds up under listening tests, the uniformity induced by FSQ-dropout may be useful beyond codecs: any quantized latent that later feeds a continuous generative model could benefit from dropout-style regularization instead of a separate dequantization stage.","The parallel decoding with shifted pairs is a general recipe for consistency models on long sequences; it could transfer to other consistency-based autoencoders or to image and video generation where boundary artifacts appear when decoding in blocks.","A testable extension: train the same architecture with p closer to 1 or with a curriculum that anneals p, and measure whether the continuous-discrete quality trade-off shifts; the paper fixes p=0.75, so the optimum across bitrates is unexplored.","Because the codebook is implicit and small (14,641 tokens), the discrete side may be easy to scale by increasing N or dlat while keeping FSQ's straight-through training; whether 2.38 kbps is a sweet spot or a floor is not tested."],"forward_implications":["A single codec can feed both diffusion-style and autoregressive generative models from the same latent space, removing the need to retrain a separate autoencoder for each paradigm.","The same model supports two decoding modes: sequential autoregressive decoding for low latency, and parallel decoding with shifted pairs that the paper reports as higher quality and faster for long sequences.","FSQ-dropout improves continuous reconstruction without hurting discrete-token quality, implying the continuous path is not an afterthought of the discrete codebook.","Because summary embeddings are not temporally ordered, the same total latent dimensionality can be re-shaped across time and channels, letting downstream models choose their preferred sequence length.","The paper reports lower FAD and FAD_clap than prior discrete codecs at higher bitrates while using only 2.38 kbps, suggesting the compact 4-dimensional latents carry the needed information."],"fun_headline_variants":["One codec, both continuous and discrete audio","CoDiCodec: continuous and discrete latents from one model","Single codec powers diffusion and autoregressive audio","Audio codec merges continuous and discrete compression","One codec: continuous ~11Hz, discrete 2.38kbps"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline comparison to other codecs assumes that FAD and FAD_clap scores on the same test set are a fair common yardstick for reconstruction quality across codecs with different architectures, bitrates, and mono/stereo outputs, although the paper cites evidence that FAD's correlation with human judgment depends on the embedding backbone and reports no listening test or confidence intervals.","fun_headline_variants_meta":{"raw":{"variants":["One codec, both continuous and discrete audio","CoDiCodec: continuous and discrete latents from one model","Single codec powers diffusion and autoregressive audio","Audio codec merges continuous and discrete compression","One codec: continuous ~11Hz, discrete 2.38kbps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001463,"raw_usage":{"total_tokens":5718,"prompt_tokens":738,"completion_tokens":4980,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":4908}},"tokens_in":482,"tokens_out":4980,"duration_ms":41137,"temperature":1.0,"reasoning_tokens":4908,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:36:53.534820+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blind listening test on held-out music where human listeners rate reconstruction quality of CoDiCodec against the top-scoring baseline; if CoDiCodec's FAD and FAD_clap advantage is not reflected in listener preference, the central outperformance claim is not established.","supporting_citations":[],"review_version":1}