{"id":"b34a6b19-0b2b-4e08-a16c-fb496679f474","arxiv_id":"2506.02979","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"J-Moshi, the first public Japanese full-duplex spoken dialogue model, is built from Moshi and outperforms a Japanese dGSLM baseline on naturalness and meaningfulness.","lead":"Researchers adapted Moshi to build J-Moshi, the first publicly released full-duplex spoken dialogue model for Japanese. Human ratings show it beats a Japanese dGSLM baseline in naturalness and meaningfulness, though it still trails real speech by a wide margin.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Human evaluation may not match prompts across models; if the 50 samples per model were independently drawn, Table 2 differences could reflect prompt difficulty, and no significance test is reported.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The paper makes a credible contribution: public release of code, models, and samples; detailed training setup; and an evaluation that, under the favorable interpretation, uses common prompts across models. The principal weakness is that this favorable interpretation is not explicitly stated. I agree with the reader's weakest assumption. The paper does report confidence intervals, which is a positive sign, but it does not establish that prompts were matched or that the observed differences are statistically significant. The concern is addressable and does not warrant rejection, since the meaningfulness gap is substantial and the generation procedure described in Section 4 suggests that all models may have been run on the full set of 709 prompts. No machine-checked proofs or parameter-free derivations are relevant here; the released artifacts support reproducibility but do not resolve the evaluation ambiguity. The verdict remains UNCHANGED: CONDITIONAL, pending clarification or a matched-pair re-evaluation.","tokens_in":8834,"tokens_out":5808,"duration_ms":73863,"concrete_test":"Inspect the released audio samples and evaluation code on the project page to determine whether the 50 prompt IDs were identical across all five conditions. If they were not, rerun the human evaluation on the same 50 prompts for every model, using paired ratings or a mixed-effects model with rater and prompt as random factors, and report the paired difference with a confidence interval. If the J-Moshi vs. dGSLM naturalness difference becomes non-significant, the headline claim should be softened; if it remains significant, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on Table 2. Section 4.3 states that 50 audio samples were 'randomly selected from the 709 dialogue continuations of each model,' but it never states that the same 50 prompts were used for dGSLM, J-Moshi, J-Moshi-ext, Re-synthesis, and Ground-truth. Since every model can generate a continuation for each of the 709 prompts, random selection may have been performed independently per model. If so, models are rated on different dialogues, and prompt difficulty becomes a confound. The naturalness advantage of J-Moshi over dGSLM is only 0.23 on a 5-point scale (2.67 vs. 2.44), and the paper reports no paired significance test or difference confidence interval, only per-condition 95% CIs. The meaningfulness gap is larger (2.19 vs. 1.76), but the abstract claims improvement in both naturalness and meaningfulness, so a prompt-driven shift in naturalness is enough to weaken the headline. This is the load-bearing assumption because Table 2 is the only direct evidence for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper adapts Moshi, an English full-duplex spoken dialogue model, to Japanese by replacing the text tokenizer with a Japanese SentencePiece model, randomly initializing the vocabulary-related weights, pre-training the RQ-Transformer on about 60,000 hours of J-CHAT data, and fine-tuning on 344 hours of stereo spoken dialogue plus 602 hours of multi-stream TTS synthetic dialogue. The resulting models, J-Moshi and J-Moshi-ext, are evaluated on a prompted dialogue continuation task against a Japanese dGSLM baseline, with automatic perplexity and human naturalness/meaningfulness ratings. The paper claims that the proposed models outperform the Japanese baseline in both metrics and that they acquire Japanese-specific turn-taking behavior, evidenced by more overlaps relative to English Moshi.","tokens_in":8996,"tokens_out":2973,"duration_ms":36332,"significance":"If the central claims hold, this is a useful and timely contribution: it reports the first publicly available Japanese full-duplex spoken dialogue model, releases code, models, and speech samples, and provides a detailed recipe for adapting a large English full-duplex model to another language. The paper also gives concrete evidence that Mimi can encode and resynthesize Japanese without retraining, and it offers preliminary cross-linguistic turn-taking observations that could inform future multilingual full-duplex research. The strengths include reproducible training configurations (loss weights, learning rates, batch sizes, hardware), transparent data handling, and comparison against a trained-from-scratch dGSLM baseline. The main weakness is that the human evaluation protocol leaves open a prompt-difficulty confound, and the reported statistical support is incomplete; these issues directly affect the headline claim.","major_comments":[{"comment":"The human evaluation sampling procedure is not sufficiently specified. The text says 50 audio samples were 'randomly selected from the 709 dialogue continuations of each model,' which can be read as independent selection per model. If the 50 prompts differ across dGSLM, J-Moshi, J-Moshi-ext, Re-synthesis, and Ground-truth, then the naturalness and meaningfulness differences in Table 2 may partly reflect prompt difficulty rather than model quality. Since the naturalness advantage of J-Moshi over dGSLM is only 0.23 on a 5-point scale (2.67 vs. 2.44), this confound is load-bearing for the central claim. Please state explicitly whether the same 50 prompts were used for all conditions; if not, re-run the evaluation with matched prompts or otherwise control for prompt difficulty.","section":"4.3, Table 2"},{"comment":"No significance test is reported for the human evaluation comparisons. The text claims that J-Moshi and J-Moshi-ext 'significantly surpassed dGSLM in meaningfulness,' but Table 2 only lists per-condition 95% confidence intervals. With a naturalness difference of 0.23 and no paired test, the claim that the models outperform the baseline in both naturalness and meaningfulness is not statistically supported. Please report a paired significance test (e.g., Wilcoxon signed-rank or bootstrap over rated samples) and, ideally, the distribution of per-item score differences between conditions.","section":"4.3, Table 2"},{"comment":"The cross-linguistic turn-taking comparison uses values for English Moshi reported by Défossez et al. [6] on 1,000 different samples, whereas the Japanese values come from 709 prompts from a different corpus. The authors acknowledge that 'the experimental conditions are not strictly identical' but still conclude that J-Moshi has acquired Japanese-specific behavior such as more overlaps. This is a weaker form of evidence than a matched comparison would be; the observed differences could reflect prompt content, corpus genre, or annotation differences rather than language-specific behavior. Please either temper the conclusion or provide a matched analysis, such as running English Moshi on the same prompts or using a comparable diarization/annotation pipeline.","section":"5, Table 3"}],"minor_comments":[{"comment":"Figure 3 reports perplexity but no error bars or measures of variance, making it difficult to assess whether the approximately 100-point improvement over dGSLM at τ=0.8 is stable across random seeds or test chunks.","section":"4.2, Figure 3"},{"comment":"The heading 'J-CHA T Preprocessing' contains a typographical error; it should read 'J-CHAT Preprocessing.'","section":"3.2"},{"comment":"The paper does not state how the crowd-workers were assigned to conditions or whether each sample was rated by multiple workers; reporting per-sample mean scores and rater counts would improve transparency.","section":"4.3"},{"comment":"In Table 3, the four turn-taking metrics are presented as seconds per minute, but their relationship is not fully defined (e.g., whether Overlap is fully contained within IPU or counted separately); a brief definition or reference for the exact computation would help readers interpret the sums.","section":"5"},{"comment":"The selection of synthetic speech by lowest WER is reasonable, but the overall WER of 24.6% is relatively high; a sentence noting the implications for training data quality and possible filtering effects would be helpful.","section":"3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution with a clear public-release benefit, but the headline claim rests on a human evaluation whose prompt-matching is ambiguous and for which no significance test is shown. If the authors can confirm that all models were rated on the same 50 prompts and supply a proper paired test, the paper would likely be acceptable after minor revision; as written, the central claim is not yet fully supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is the first publicly available Japanese full-duplex spoken dialogue model, and the authors release the training code, model weights, and speech samples. That alone makes the paper a useful resource for anyone working on multilingual full-duplex systems.\n\nWhat is actually new: they adapt Moshi (English-only) to Japanese by replacing the text tokenizer with a Japanese SentencePiece, randomly initializing the tied embedding/linear weights, pre-training on J-CHAT (69k hours split into pseudo-stereo by diarization), then fine-tuning on 344 hours of real stereo dialogue and 602 hours of multi-stream TTS synthetic speech. They show Mimi codec works on Japanese without retraining, and the resulting models reproduce Japanese-specific turn-taking (more overlaps, longer IPUs) relative to reported English numbers for Moshi. That is a concrete contribution: a baseline and an adaptation recipe.\n\nThe paper is also honest in its own limitations. It notes the English comparison uses reported numbers and that conditions are not strictly identical, and it states RQ-Transformer still lags behind re-synthesis. Those are the right caveats to include.\n\nThe soft spots are real but not fatal. Section 4.3 says 50 samples were 'randomly selected from the 709 dialogue continuations of each model' and never says the same 50 prompts were used across models. If the samples are drawn independently per model, prompt difficulty is a confound, and with a naturalness gap of only 0.23 on a 5-point scale, that matters. There is no paired test or difference confidence interval. The meaningfulness gap is larger and consistent, so I think the central claim still holds, but the evaluation protocol needs to be clarified. A matched-prompt design or reporting how many prompts overlap would settle it. Minor: Figure 3 has no error bars, and the turn-taking table compares under non-identical conditions, but the authors already flag that.\n\nBottom line: the paper deserves a serious referee. The contribution is real, the artifacts help the community, and the evaluation issues are fixable in revision. I would cite it if I were working on multilingual full-duplex dialogue.","headline":"First public Japanese full-duplex dialogue model with released artifacts; evaluation has a matched-prompt question, but the contribution is solid and worth refereeing.","tokens_in":9549,"tokens_out":2364,"would_cite":true,"duration_ms":24855,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"J-Moshi, the first publicly available Japanese full-duplex spoken dialogue model, outperforms the dGSLM baseline in both naturalness and meaningfulness.","keywords":["full-duplex spoken dialogue","Japanese","Moshi","RQ-Transformer","J-CHAT","multi-stream TTS","speech overlap","backchannel"],"falsifier":"Run a matched human evaluation in which all models continue the same 50 ten-second prompts, keeping temperature and evaluator instructions identical. If J-Moshi and J-Moshi-ext no longer beat dGSLM in naturalness and meaningfulness, the paper's central comparison fails.","tokens_in":8626,"feed_emoji":"🎙️","tokens_out":5091,"duration_ms":48865,"temperature":0.7,"pith_summary":"The paper reports the first publicly available full-duplex spoken dialogue model for Japanese, built by adapting the English model Moshi. It claims that the adapted model, J-Moshi, and its TTS-augmented extension J-Moshi-ext, generate dialogue continuations that human raters find more natural and more meaningful than those of dGSLM, a Japanese-trained baseline. The authors also report that J-Moshi reproduces Japanese-specific turn-taking patterns, notably more speech overlaps than English Moshi. If the claims hold, this gives the Japanese speech community a working baseline and a path for porting full-duplex dialogue capabilities to other languages.","feed_headline":"Japanese full-duplex voice model J-Moshi beats dGSLM","feed_subtitle":"First public Japanese full-duplex dialogue model scores higher on naturalness and meaningfulness in human ratings.","key_machinery":"The central mechanism is the RQ-Transformer from Moshi, a 7B-parameter Temporal Transformer plus a smaller Depth Transformer that autoregressively models 17 parallel token streams: one text stream for Moshi's inner monologue, eight audio streams for Moshi's speech, and eight for the user's speech, at 12.5 Hz. The Mimi neural codec encodes speech into semantic and acoustic tokens. Training proceeds in two stages: pre-training on J-CHAT to learn Japanese spoken dialogue, then fine-tuning on stereo data to learn overlap and backchannel timing; a multi-stream TTS stage synthesizes additional stereo dialogues and improves language quality. A temperature of τ=0.8 is used for generation.","core_discovery":"On its own terms, the paper establishes that Moshi's architecture can be transferred to Japanese by swapping the text tokenizer to a Japanese SentencePiece model, freezing the Mimi audio codec, pre-training the RQ-Transformer on 69,000 hours of J-CHAT, and then fine-tuning on 344 hours of stereo spoken dialogue. The fine-tuned J-Moshi scores 2.67 for naturalness and 2.19 for meaningfulness on a 5-point scale, versus 2.44 and 1.76 for dGSLM. Adding 602 hours of multi-stream TTS synthetic dialogue, J-Moshi-ext, raises meaningfulness to 2.30. The paper also shows higher overlap and IPU counts than English Moshi, consistent with Japanese conversational norms.","pith_inferences":["A natural next test is whether the same two-stage recipe transfers to other languages with large monophonic corpora but little stereo dialogue, such as Mandarin or Korean.","Because Mimi was frozen and resynthesis scores remain about 0.5 points below ground truth, adapting the codec to Japanese acoustic patterns may be a higher-yield improvement than scaling the language model alone.","The human evaluation's random selection of 50 samples per model, without a stated shared prompt set, leaves open the possibility that part of the score gap reflects prompt difficulty; a matched-prompt evaluation would settle that.","The high PAD ratio suggests that a tokenizer tuned for Japanese could reduce the 88% PAD rate, potentially improving both training efficiency and fluency."],"forward_implications":["J-Moshi and J-Moshi-ext are released publicly, giving researchers a Japanese full-duplex baseline to compare against.","Multi-stream TTS augmentation raises meaningfulness from 2.19 to 2.30, so synthetic stereo data is a viable route to improve language capability when real stereo corpora are scarce.","The model reproduces Japanese-specific turn-taking, with more overlaps than English Moshi, suggesting full-duplex behavior is learnable from the target language's data.","The 88% PAD token ratio in J-CHAT indicates Japanese text streams are sparser relative to audio, pointing to a need for language-specific loss weighting in future adaptations."],"supporting_citations":[{"why":"Provides Moshi's full-duplex architecture and training recipe, including the multi-stream TTS data augmentation, which J-Moshi adapts.","marker":"[6]"},{"why":"Defines the dGSLM baseline and the prompted dialogue continuation task used in evaluation.","marker":"[1]"},{"why":"J-CHAT corpus supplies the 69,000 hours of Japanese spoken dialogue used for pre-training.","marker":"[10]"},{"why":"Japanese Callhome is one of the stereo spoken dialogue corpora used in fine-tuning.","marker":"[23]"},{"why":"Provides JapanesePersonaChat and JapaneseEmpatheticDialogues text corpora used for multi-stream TTS augmentation.","marker":"[26]"},{"why":"Provides RealPersonaChat, a text dialogue corpus rewritten and synthesized for stereo training data.","marker":"[28]"},{"why":"WhisperX gives word-level timestamps used to align text tokens to audio frames and insert PAD tokens.","marker":"[19]"}],"fun_headline_variants":["J-Moshi: first Japanese full-duplex dialogue model beats dGSLM","Japanese full-duplex model J-Moshi beats dGSLM in human ratings","J-Moshi outperforms dGSLM on Japanese full-duplex dialogue"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The human evaluation compares models on 50 samples randomly selected from each model's 709 outputs, without stating that the same 50 prompts were used for every model; if prompts differ, score gaps could reflect prompt difficulty rather than model quality.","fun_headline_variants_meta":{"raw":{"variants":["J-Moshi: first Japanese full-duplex dialogue model beats dGSLM","Japanese full-duplex model J-Moshi beats dGSLM in human ratings","J-Moshi outperforms dGSLM on Japanese full-duplex dialogue"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000487,"raw_usage":{"total_tokens":2350,"prompt_tokens":843,"completion_tokens":1507,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":1440}},"tokens_in":459,"tokens_out":1507,"duration_ms":11666,"temperature":1.0,"reasoning_tokens":1440,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:10:50.935050+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a matched human evaluation in which all models continue the same 50 ten-second prompts, keeping temperature and evaluator instructions identical. If J-Moshi and J-Moshi-ext no longer beat dGSLM in naturalness and meaningfulness, the paper's central comparison fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Moshi's full-duplex architecture and training recipe, including the multi-stream TTS data augmentation, which J-Moshi adapts."},{"cited_title":"In dialogue, full-duplex refers to having simultaneous bidirectional features such as speech overlaps and backchannels","cited_arxiv_id":null,"evidence_quote":"Defines the dGSLM baseline and the prompted dialogue continuation task used in evaluation."},{"cited_title":"A Full-duplex Speech Dialogue Scheme Based On Large Language Model,","cited_arxiv_id":null,"evidence_quote":"J-CHAT corpus supplies the 69,000 hours of Japanese spoken dialogue used for pre-training."},{"cited_title":"pyannote.audio 2.1 speaker diarization pipeline: prin- ciple, benchmark, and recipe,","cited_arxiv_id":null,"evidence_quote":"Provides JapanesePersonaChat and JapaneseEmpatheticDialogues text corpora used for multi-stream TTS augmentation."},{"cited_title":"ZeRO: memory optimizations toward training trillion parameter models,","cited_arxiv_id":null,"evidence_quote":"Provides RealPersonaChat, a text dialogue corpus rewritten and synthesized for stereo training data."},{"cited_title":"Simultaneous talk—from the perspective of floor management of English and Japanese speakers,","cited_arxiv_id":null,"evidence_quote":"WhisperX gives word-level timestamps used to align text tokens to audio frames and insert PAD tokens."}],"review_version":1}