REVIEW 3 major objections 6 minor 34 references
Developing an Effective Training Dataset to Enhance the Performance of AI-based Speaker Separation Systems
T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Real recorded training data beats synthetic mixes by 1.65 dB.
desk verdict The recording protocol is a neat practical idea, but the paper never verifies that the solo recordings are true components of the mixtures, so the headline 1.65 dB gain is not yet interpretable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is a full-duplex audio playback/recording routine (MATLAB's AudioPlayerRecorder with an ASIO driver) that can write and read audio samples at the same time, letting the same hardware path first record each speaker alone and then record the simultaneous two-speaker mixture. Overrun and underrun counts are checked on every take so that a take with lost samples is redone, which is the step that makes the solo recordings usable as ground truths. The evaluation model is a deep attractor network: log-magnitude spectrogram frames pass through four bidirectional gated recurrent unit (BGRU) layers and a fully connected layer that embeds each frame in a 20-dimensional space, and frames are assigned to the nearest speaker center to build masks that reconstruct the separated speech.
What would settle it
Record the same two speakers with close microphones at the same time as the room microphone captures the mixture, then compare each speaker's close-mic signal, after time alignment, with the separately recorded solo take; if they differ by more than the noise floor, the solo takes are not the true mixture components, and the reported gain is not a clean measure of separation accuracy.
Extended reading notes
Core claim
The central claim is that a recorded, non-synthetic two-speaker separation corpus with clean ground-truth stems is feasible, and that supervised training on it transfers to real recordings better than training on digitally mixed speech. The construction records each TIMIT utterance alone through a loudspeaker and microphone to form the ground truth, then plays two utterances on two channels simultaneously and records the resulting mixture, retrying whenever the routine reports overrun or underrun sample loss. This yields Realistic_TIMIT_2mix with 30/10/5 hours of training/validation/test data. With identical hyperparameters, the BGRU deep attractor model trained on this corpus beats the synthetic-trained model by 1.65 dB SI-SDR on real mixtures and by 1.25 dB on synthetic LibriMix mixtures, and its performance stays above 8 dB SI-SDR as microphone–speaker distance varies while the synthetic model drops toward 2 dB.
Load-bearing premise
The load-bearing premise is that a speaker's solo recording, made in a separate playback pass, is exactly the signal that appears inside the two-speaker mixture; if the loudspeaker, the room, or the timing behaves differently during the joint pass, the training targets do not match the mixture.
Editorial extensions
If this is right
- Models trained on recorded mixtures should separate real two-speaker audio more accurately than models trained on digitally summed clean speech.
- The distance experiment implies that a fixed recording distance of 2 meters teaches features that keep separation above 8 dB SI-SDR across 0.5 to 3 meters in the same room.
- The recording recipe can be reused to create paired mixture/ground-truth sets for other corpora, enabling supervised separation training in environments where digital mixing is unrealistic.
- The 1.25 dB improvement on synthetic LibriMix indicates the benefit is not just matching the recording setup; the realistic-trained model also separates computer-mixed speech better.
Reading between the lines
- The paper does not test transfer to a different room or a different loudspeaker; a plausible extension is measuring how much of the 1.65 dB gain survives when the trained model is evaluated on recordings made elsewhere.
- Because ground truths are recorded in separate takes, any drift between takes enters as a corrupted target; this could be tested by recording close-microphone references simultaneously with the mixture and checking whether the solo takes match them after alignment.
- Adding real noise and reverberation to the recorded-set construction, as the paper's outlook suggests, would test whether the same full-duplex recipe can replace digitally noisy synthetic data.
- A stronger evaluation would compare the realistic-trained model against a model trained on a large synthetic noisy/reverberant corpus evaluated on the same real test set, isolating the contribution of the recording method from the contribution of training data size.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method for constructing a 'realistic' two-speaker speech-separation training set, called Realistic_TIMIT_2mix, by playing TIMIT utterances through loudspeakers in a room and recording the resulting mixture with a microphone, while obtaining ground truths by playing each speaker file alone and recording it in a separate take, all using MATLAB's AudioPlayerRecorder full-duplex function (Algorithm 1). The authors train two copies of the same BGRU deep attractor network from their prior work [32], one on this realistic set and one on a synthetic TIMIT mixture set, and compare SI-SDR on LibriMix and on their own real-mixture test recordings. They report that the model trained on the realistic set outperforms the synthetic-trained model by 1.65 dB on real mixtures and by 1.25 dB on LibriMix, and that its performance remains more stable as microphone distance varies.
Significance. If the acquisition procedure genuinely yields the true components of the recorded mixtures as ground truths, the dataset and the full-duplex recording protocol would be a practical contribution to speaker separation, targeting a real bottleneck in the field. The LibriMix result is a useful control: because LibriMix is an external synthetic corpus, the improvement there is not trivially explained by matching the recording setup. However, the validity of the central contribution depends on an unverified assumption that the separately recorded solo takes are exactly the components inside the jointly recorded mixture, and the real-mixture evaluation shares the training room and hardware. With explicit validation of the additive-component identity, independent test recordings, and statistical reporting, the result could be significant; in its current form the evidence is suggestive but not conclusive.
major comments (3)
- [Section 4.4 / Algorithm 1] The dataset construction assumes that the solo recordings gts1 and gts2 (Algorithm 1, lines 5-10) are exactly the source components inside the mixture rMix (line 11). These are separate playbacks: any change in loudspeaker response, amplifier gain, room impulse response, or microphone noise between takes makes the training targets inconsistent with the input mixture. The paper reports no check of the identity rMix ≈ gts1 + gts2 (e.g., residual energy after time alignment), even though this identity is the central assumption that makes the ground truths usable. Without it, the 1.65 dB SI-SDR gain in Table 2 can reward a model for emitting a plausible solo-like reverberant signal rather than actually separating the sources present in the mixture. Please add an explicit validation of the additive-component assumption, or modify the acquisition procedure so that the mixture and the components are recorded simultaneously with known capture paths.
- [Section 6, real-mixture test set] The real-mixture test set was recorded by the authors using the same procedure described in Algorithm 1, in the same room, with the same hardware as the training set. The comparison in Table 2 therefore measures within-protocol consistency rather than generalization to other real acoustic environments, which is the conclusion drawn in the abstract ('real-world scenarios'). The paper should evaluate on independently recorded real mixtures, for example the REAL-M corpus [16] or recordings made in different rooms with different microphones, and report the resulting domain gap. The LibriMix result in Table 1 is a helpful external check, but LibriMix is a synthetic mixture set and does not by itself establish real-world generalization.
- [Section 6, Tables 1-2 and Figure 6] The paper reports only point estimates of SI-SDR, with no number of test mixtures, no confidence intervals, and no variance across training runs. The 1.65 dB difference that supports the main claim could plausibly fall within run-to-run seed variance for a model of this type. Please report mean ± standard deviation over at least three training runs for each condition, state the test-set size, and, if feasible, a paired significance test. This is necessary to make the headline quantitative claim reliable.
minor comments (6)
- [Section 5.2, learning-curve paragraph] The learning-curve description is internally inconsistent: the text states that the synthetic model reaches its minimum (0.23) at epoch 157 and the realistic model at 0.26 at epoch 143, then says the realistic model has the lowest minimum and that the synthetic model converged faster. The numbers imply the opposite on both counts; please correct the prose or the numbers.
- [Section 4.4 / Algorithm 1] The dataset itself is not made available; no URL, repository, or data-access statement is provided. For a contribution whose main deliverable is a dataset, a download link and a data sheet are important for reproducibility and for the community to adopt the method.
- [Section 6] The real-mixture test procedure is underspecified: the paper should state how many real test mixtures were recorded, from which TIMIT subset the held-out utterances were drawn, and how speaker overlap with the training set was avoided beyond saying 'files not used for training.'
- [Sections 2 and 4] The claim that this is 'the first' real speaker-separation training set with ground truths is asserted repeatedly, but the related-work discussion does not systematically rule out earlier corpora with close-microphone or multi-channel recordings that provide both mixtures and per-speaker tracks; please narrow the claim or add explicit comparisons with such datasets.
- [Section 6, LibriMix test set] The LibriMix test configuration should be specified in detail, including the exact version (e.g., Libri2Mix, clean/noisy/reverb), sampling rate, and mixture construction, because the 1.25 dB gain in Table 1 can be interpreted only with that context.
- [Figures] Figure numbering is inconsistent: the scenario diagram appears under a caption labeled 'Figure (1)' that duplicates the earlier audio-card diagram, and subsequent figures are numbered out of sequence; please renumber all figures in order of appearance.
Circularity Check
No significant circularity: the 1.65 dB SI-SDR improvement is an empirical benchmark result, not a derivation; the only self-citation (baseline BGRU model) is not load-bearing for the dataset claim.
full rationale
The paper's central claim is empirical: two identical BGRU deep-attractor models are trained on Realistic_TIMIT_2mix and Synthetic_TIMIT_2mix built from the same TIMIT files, then evaluated on LibriMix and on held-out real recordings. No test-set parameter is fitted, and the LibriMix evaluation (Table 1) is independent of the authors' recording protocol, so the reported gains (1.65 dB on real mixtures, 1.25 dB on LibriMix) are not forced by construction. The authors' self-citation [32] supplies only the baseline network used as a test vehicle; the deep-attractor/BGRU architecture is not the claim being validated, and the dataset comparison would be meaningful with any fixed baseline. The separate-take assumption in Algorithm 1 — gts1 and gts2 are recorded in solo playbacks while rMix is a later joint playback, with the unstated assumption rMix ≈ gts1 + gts2 — is a real validity risk (if loudspeaker/room conditions drift between takes, the targets are inconsistent with the mixture), but it is an unverified measurement assumption, not an equation-level reduction of the claimed result to its inputs. Similarly, the real test set sharing the training room and hardware weakens generalization claims but does not make the comparison circular. The result is therefore self-contained as an empirical finding, with no circular step to quote.
Assumptions & free parameters
assumptions (4)
- domain assumption The TIMIT corpus is a valid source of clean speech for building a speaker separation test bed.
- domain assumption A real mixture recorded by a microphone can be approximated as the sum of the separately recorded single-source signals.
- standard math SI-SDR is an appropriate metric for speaker separation quality.
- domain assumption The BGRU deep attractor network from [32] is a representative baseline.
Cite this review
Pith. "Pith review of Developing an Effective Training Dataset to Enhance the Performance of AI-based Speaker Separation Systems." pith.science (2026). https://pith.science/paper/5QSGE3MU
@misc{pith2026241108375,
author = {Pith},
title = {Pith review of: Developing an Effective Training Dataset to Enhance the Performance of AI-based Speaker Separation Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/5QSGE3MU}},
note = {Machine review of arXiv:2411.08375}
}
read the original abstract
This paper addresses the challenge of speaker separation, which remains an active research topic despite the promising results achieved in recent years. These results, however, often degrade in real recording conditions due to the presence of noise, echo, and other interferences. This is because neural models are typically trained on synthetic datasets consisting of mixed audio signals and their corresponding ground truths, which are generated using computer software and do not fully represent the complexities of real-world recording scenarios. The lack of realistic training sets for speaker separation remains a major hurdle, as obtaining individual sounds from mixed audio signals is a nontrivial task. To address this issue, we propose a novel method for constructing a realistic training set that includes mixture signals and corresponding ground truths for each speaker. We evaluate this dataset on a deep learning model and compare it to a synthetic dataset. We got a 1.65 dB improvement in Scale Invariant Signal to Distortion Ratio (SI-SDR) for speaker separation accuracy in realistic mixing. Our findings highlight the potential of realistic training sets for enhancing the performance of speaker separation models in real-world scenarios.
Reference graph
Works this paper leans on
-
[32]
Improving deep attractor network by BGRU and GMM for speech separation,
R. Melhem, A. Jafar and R. Hamadeh, "Improving deep attractor network by BGRU and GMM for speech separation," Journal of Harbin Institute of Technology (New Series), vol. 28, no. 3, pp. 90-96, 2021
work page 2021
-
[16]
REAL-M: Towards Speech Separation on Real Mixtures
C. Subakan, M. Ravanelli, S. Cornell and F. Grondin, "REAL-M: TOWARDS SPEECH SEPARATION ON REAL MIXTURES," arXiv:2110.10812 , 20 Oct 2021
work page Pith review arXiv 2021
-
[1]
Some Experiments on the Recognition of Speech, with One and with Two Ears,,
E. C. Cherry, "Some Experiments on the Recognition of Speech, with One and with Two Ears,," J. Acoust. Soc. Am.,, vol. 25, no. 5, p. 975–979, Sep. 1953
work page 1953
-
[2]
DEEP CLUSTERING:DISCRIMINATIVE EMBEDDINGS FOR SEGMENTATION AND SEPARATION,
J. R. Hershey, Z. Chen, J. L. Roux and S. Watanabe, "DEEP CLUSTERING:DISCRIMINATIVE EMBEDDINGS FOR SEGMENTATION AND SEPARATION," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016
work page 2016
-
[3]
Deep attractor network for single -microphone speaker separation,
Z. Chen, Y. Luo and N. Mesgarani, "Deep attractor network for single -microphone speaker separation," in In 2017 IEEE Internatio nal Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 246-250). IEEE, 2017
work page 2017
-
[4]
Attention is All You Need in Speech Separation
C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi and J. Zhong, "ATTENTION IS ALL YOU NEED IN SPEECH SEPARATION," arXiv:2010.13154v1 , 25 Oct 2020
work page Pith review arXiv 2010
-
[5]
Improved Speech Separation with Time-and-Frequency Cross-domain Joint Embedding and Clustering
G.-P. Yang, C. -I. Tuan, H. -Y. Lee and L. -s. Lee, "Improved Speech Separation with Time -and- Frequency Cross-domain Joint Embedding and Clustering," arXiv:1904.07845, 16 Apr 2019
work page Pith review arXiv 1904
-
[6]
D. Yu, M. Kolbæk, T. Zheng -Hua and J. Jensen, "Permutation invariant tr aining of deep models for speaker-independent multi-talker speech separation," International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE
work page 2017
Show all 34 references
-
[7]
Multitalker speech separation with utterance -level permutation invariant training of deep recurrent neural networks,
M. Kolbæk, D. Yu, Z. -H. Tan and J. Jensen, "Multitalker speech separation with utterance -level permutation invariant training of deep recurrent neural networks," IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 25, no. 10, October 2017)
2017
-
[8]
TF -GRIDNET: MAKING TIME-FREQUENCY DOMAIN MODELS GREAT AGAIN FOR MONAURAL SPEAKER SEPARATION,
Z.-Q. Wang, S. Cornell, S. Choi, Y. Lee, B. -Y. Kim and S. Watanabe, "TF -GRIDNET: MAKING TIME-FREQUENCY DOMAIN MODELS GREAT AGAIN FOR MONAURAL SPEAKER SEPARATION," arXiv:2209.03952, 15 Mar 2023
2023 arXiv
-
[9]
TASNET: TIME -DOMAIN AUDIO SEPARATION NETWORK FOR REAL-TIME, SINGLE-CHANNEL SPEECH SEPARATION,
Y. Luo and N. Mesgarani, "TASNET: TIME -DOMAIN AUDIO SEPARATION NETWORK FOR REAL-TIME, SINGLE-CHANNEL SPEECH SEPARATION," IEEE, ICASSP 2018, 2018
2018
-
[10]
Conv -TasNet: Surpassing Ideal Time –Frequency Magnitude Masking f or Speech Separation,
Y. Luo and N. Mesgarani, "Conv -TasNet: Surpassing Ideal Time –Frequency Magnitude Masking f or Speech Separation," IEEE/ACM TRANSAC TIONS ON AUDIO, SPEECH, AND LANGUAGE P ROCESSING, vol. 27, no. 8, AUGUST 2019
2019
-
[11]
DUAL -PATH RNN: EFFICIENT LONG SEQUENCE MODELING FOR TIME -DOMAIN SINGLE -CHANNEL SPEECH SEPARATION,
Y. Luo, Z. Chen and T. Yoshioka, "DUAL -PATH RNN: EFFICIENT LONG SEQUENCE MODELING FOR TIME -DOMAIN SINGLE -CHANNEL SPEECH SEPARATION," IEEE, ICASSP 2020
2020
-
[12]
Dual -Path Transformer Network: Direct Context -Aware Modeling for End-to-End Monaural Speech Separation,
J. Chen, Q. Mao and D. L iu, "Dual -Path Transformer Network: Direct Context -Aware Modeling for End-to-End Monaural Speech Separation," arXiv:2007.13975, 14 Aug 2020
2007 arXiv
-
[13]
Wavesplit: End -to-End Speech Separation by Speaker Clustering,
N. Zeghidour and D. Grangier, "Wavesplit: End -to-End Speech Separation by Speaker Clustering," IEEE/ACM Transactions on Audio, Speech, and Language Processing 29, pp. 2840-2849, 2021
2021
-
[14]
Tiny -Sepformer: A Tiny Time -Domain Transformer Network for Speech Separation,
J. Luo, J. Wang, N. Cheng, E. Xiao, X. Zhang and J. Xiao, "Tiny -Sepformer: A Tiny Time -Domain Transformer Network for Speech Separation," arXiv:2206.13689, 30 Jun 2022
2022 arXiv
-
[15]
Divide and Conquer: A Deep CASA Approach to Talker -independent Monaural Speaker Separation,
Y. Liu and D. Wang, "Divide and Conquer: A Deep CASA Approach to Talker -independent Monaural Speaker Separation," IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019
2019
-
[17]
WHAM!: E xtending Speech Separation to Noisy Environments,
G. Wichern, J. Antognini, M. Flynn, L. Richard Zhu, E. McQuinn, D. Crow, E. Manilow and J. Le Roux, "WHAM!: E xtending Speech Separation to Noisy Environments," arXiv:1907.01160, 2 Jul 2019
1907 arXiv
-
[18]
WHAMR!: NOISY AND REVERBERANT SINGLE -CHANNEL SPEECH SEPARATION,
M. Maciejewski, G. Wichern, E. McQuinn and J. Le Roux, "WHAMR!: NOISY AND REVERBERANT SINGLE -CHANNEL SPEECH SEPARATION," ICASSP 2020 -2020 IEEE تطوير مدو نة تدريب لتحسين أداء نظم فصل المتكلمين المعتمدة على الذكاء الصنعي. م. ملحم، د.م. جعفر، د.م. الدكاك. 13 من13 International ...
2020
-
[19]
LibriMix: An Open-Source Dataset for Generalizable Speech Separation,
J. Cosentino, M. Pariente, S. Cornell, A. Deleforge and E. Vincent, "LibriMix: An Open-Source Dataset for Generalizable Speech Separation," arXiv:2005.11262, 22 May 2020
2005 arXiv
-
[20]
THE THIRD ‘CHIME’ SPEECH SEPARATION AND RECOGNITION CHALLENGE: DATASET, TASK AND BASELINES,
J. Barker, R. Marxer, E. Vincent and S. Watanabe, "THE THIRD ‘CHIME’ SPEECH SEPARATION AND RECOGNITION CHALLENGE: DATASET, TASK AND BASELINES," IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)., 2015
2015
-
[21]
The fifth ‘CHiME’ Speech Separation and Recognition Challenge: Dataset task and baselines,
J. Barker, S. Watanabe, E . Vincent and J. Trmal, "The fifth ‘CHiME’ Speech Separation and Recognition Challenge: Dataset task and baselines," arXiv preprint arXiv:1803.10609, 2018
2018 arXiv
-
[22]
The Mixer 6 Corpus:Resources for Cross -Channel and Text Independent Speaker Recognition,
L. Brandschain, D. Graff, C. Cieri, K. Walker, C. Caruso and A. Neely, "The Mixer 6 Corpus:Resources for Cross -Channel and Text Independent Speaker Recognition," Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)., 2010
2010
-
[23]
VoxCeleb: a large -scale speaker identification dataset,
A. Nagrani, J. Son Chung and A. Zisserman, "VoxCeleb: a large -scale speaker identification dataset," arXiv:1706.08612, 30 May 2018
2018 arXiv
-
[24]
TRAINING NOISY SINGLE -CHANNEL SPEECH SEPARATION WITH NOISY ORACLE SOURCES: A LARGE GAP AND A SMALL STEP,
M. Maciejewski, J. Shi, S. Watanabe and S. Khudanpur, "TRAINING NOISY SINGLE -CHANNEL SPEECH SEPARATION WITH NOISY ORACLE SOURCES: A LARGE GAP AND A SMALL STEP," arXiv:2010.12430, 22 Feb 2021
2010 arXiv
-
[25]
A Gender Mixture Detection Approach to Unsupervised Single-Channel Speech Separation Based on Deep Neural Networks,
Y. Wang, J. Du, L. -R. Dai and C. -H. Lee, "A Gender Mixture Detection Approach to Unsupervised Single-Channel Speech Separation Based on Deep Neural Networks," IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 25, no. 7, July 2017
2017
-
[26]
REMIX -CYCLE-CONSISTENT LEARNING ON ADVERSARIALLY LEARNED SEPARATOR FOR ACCURATE AND STABLE UNSUPERVISED SPEECH SEPARATION,
K. Saijo and T. Ogawa, "REMIX -CYCLE-CONSISTENT LEARNING ON ADVERSARIALLY LEARNED SEPARATOR FOR ACCURATE AND STABLE UNSUPERVISED SPEECH SEPARATION," arXiv:2203.14080, 26 Mar 2022
2022 arXiv
-
[27]
Unsupervised Sound Separation Using Mixture Invariant Training,
S. Wisdom, E. Tzinis, H. Erdogan, R. J. Weiss, K. Wilson and J. R. Hershey, "Unsupervised Sound Separation Using Mixture Invariant Training," arXiv:2006.12701v2, 24 Oct 2020
2006 arXiv
-
[28]
Teacher -Student MixIT for Unsupervised and Semi - supervised Speech Separation,
J. Zhang, C. Zorila, R. Doddipatla and J. Barker, "Teacher -Student MixIT for Unsupervised and Semi - supervised Speech Separation," arXiv:2106.07843, 9 Sep 2021
2021 arXiv
-
[29]
Heterogeneous separation consistency training for adaptation of unsupervised speech separation,
J. Han and Y. Long, "Heterogeneous separation consistency training for adaptation of unsupervised speech separation," EURASIP Journal on Audio, Speech, and Music Processing, no. 1, pp. 1 -17, 2023
2023
-
[30]
DARPA TIMIT acoustic -phonetic continous speech corpus CD -ROM. NIST speech disc 1 -1.1,
J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett and N. L. Dahlgren, "DARPA TIMIT acoustic -phonetic continous speech corpus CD -ROM. NIST speech disc 1 -1.1," NASA STI/Recon technical report n, 93, 27403, 1993
1993
-
[31]
BUILDING CORPORA FOR SINGLE -CHANNEL SPEECH SEPARATION ACROSS MULTIPLE DOMAINS,
M. Maciejewski, G. Sell, L. P. Garcia-Perera, S. Watanabe and S. Khudanpur, "BUILDING CORPORA FOR SINGLE -CHANNEL SPEECH SEPARATION ACROSS MULTIPLE DOMAINS," arXiv:1811.02641, 6 Nov 2018
2018 arXiv
-
[33]
Adam: A Method for Stoch astic Optimization,
D. P. Kingma and J. Ba, "Adam: A Method for Stoch astic Optimization," arXiv:1412.6980v9, 30 Jan 2017
2017 arXiv
-
[34]
SDR – HALF-BAKED OR WELL DONE?,
J. Le Roux, S. Wisdom, H. Erdogan and J. R. Hershey, "SDR – HALF-BAKED OR WELL DONE?," ICASSP, IEEE 2019
2019
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.