REVIEW 4 major objections 6 minor 19 references
The paper claims that earphone-recorded ExG signals, decomposed into 12 physiology-informed frequency sub-bands and pre-trained with reconstruction objectives on unlabeled free-living data, yield a single task-agnostic representation that b
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A 12-band physiology-informed tokenizer with a bidirectional Mamba encoder, pre-trained on free-living earphone ExG data, beats task-specific baselines across five-sense and public ExG benchmarks.
T0 review reviewed 2026-08-04 challenge →
load-bearing objection Real hardware-plus-data contribution, but the 'five senses' generalization claim rests on an evaluation split that leaks subject/session identity and a protocol that co-varies with facial EMG; the public-benchmark evidence is stronger and worth referee time. the 4 major comments →
Beyond Hearing: Learning Task-Agnostic ExG Representations from Earphones via Physiology-Informed Tokenization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
At the core is Physiology-informed Multi-band Tokenization (PiMT): instead of feeding raw wide-band ExG or a task-selected narrow band, every channel is passed through a fixed bank of 12 canonical sub-band filters, each tied to a known physiological modality. Each sub-band is patched and embedded, with tokens ordered frequency-first, channel-second, time-last, and processed by a bidirectional state-space encoder. Pre-training on unlabeled free-living ear-ExG uses six complementary reconstruction tasks — autoencoding, masked reconstruction, and amplitude/phase reconstruction in both original and masked forms — which jointly force the encoder to retain temporal, spectral, and contextual inform
What carries the argument
The load-bearing object is the physiology-informed filter bank: 12 fixed sub-band filters spanning EEG delta/theta/alpha/beta/gamma, EMG low/mid/high frequency, EOG overall, ECG low/high frequency, and the QRS complex. Decomposing every channel into these bands and tokenizing patches in a frequency-first ordering gives the encoder fine-grained, task-independent access to spectral content; the reconstruction pre-training (six objectives with decoders) makes the representations usable without labels. The bidirectional state-space backbone keeps sequence modeling linear-time despite the added frequency dimension.
Load-bearing premise
The DailySense sensory labels (sweet vs. sour, floral vs. sour, rough vs. smooth) are assumed to reflect the intended perceptual state, but the protocols differ in mouth, face, and sniffing muscle activity that ear electrodes can record as EMG — the paper's own appendix acknowledges auricular electrodes capture facial-expression EMG — so without session- or subject-excluded validation, the strong within-session accuracy may partly reflect muscle-artifact classification.
What would settle it
Run the taste, smell, and touch protocols with sham conditions — sipping a neutral liquid without taste, sniffing an empty bag, rubbing the same texture while blindfolded — and check whether a model trained on the true labels can still separate the classes; alternatively, evaluate with a leave-one-session-out split where each recording session is held out, since the paper's own cross-subject F1 drops to 0.586.
If this is right
- A single pre-trained ExG encoder can be fine-tuned to tasks as different as gaze estimation, taste/smell/touch classification, emotion recognition, sleep staging, and motor imagery without changing the tokenizer or choosing a frequency band.
- The multi-band decomposition alone (without pre-training) already improves over wide-band or single-band baselines; adding free-living pre-training yields further gains, showing unlabeled in-the-wild data is useful.
- Earphone electrodes capture sufficient EEG, EOG, and EMG information for these tasks, making a low-cost earhook form factor a viable alternative to lab headsets for everyday sensing.
- Reconstruction-based pre-training scales with data: larger free-living corpora reduce reconstruction loss and continue improving downstream gaze regression even after classification performance saturates.
- On-device inference is real-time on a smartphone (25 ms latency, 266 MB memory), so the approach is deployable in a wearable companion-device setting.
Where Pith is reading between the lines
- If this holds, earphone-based ExG could become a general-purpose physiological sensing layer for context-aware wearables — detecting user state across senses with one model — but only if the sensory-classification results survive session- and subject-excluded validation, which the paper's own cross-subject drop (average F1 0.586) leaves open.
- The tokenization is encoder-agnostic in spirit; the paper shows a state-space backbone beats a Transformer with the same multi-band tokens, but the band decomposition itself might transfer to other architectures or to larger foundation-model scaling regimes.
- A direct test of the physiological claim would be to run the taste, smell, and touch protocols with sham stimuli (same mouth/nose movements but no actual taste/smell/texture difference); if a model still separates classes, the labels are carried by muscle activity rather than perception.
- The 12-band boundaries are grounded in textbook physiology but are still arbitrary; a learned or data-driven band allocation would test whether the fixed physiological prior is the active ingredient or whether finer spectral resolution alone drives the gains.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents NeuroBuds, an earphone-based ExG sensing prototype, and DailySense, a dataset with 50 hours of free-living and 20 hours of labeled ExG recordings spanning sight, hearing, taste, touch, and smell. It introduces PiMT, which decomposes ExG into 12 physiology-informed sub-band tokens, encodes them with a Bidirectional-Mamba backbone, and pre-trains with six reconstruction objectives on unlabeled free-living data. The authors report an average F1 of 0.876 on DailySense and consistent improvements over PatchTST and Bidirectional-Mamba on DREAMER, SEED, Sleep-EDF, and BCI Competition IV 2b. The central claim is that PiMT learns task-agnostic ExG representations that generalize across diverse sensory and cognitive tasks.
Significance. The paper addresses a relevant problem and makes a concrete hardware/data contribution; conditional release of DailySense would be valuable. The public-benchmark results (Table 2) and the ablation tables provide useful evidence that multi-band tokenization plus reconstruction pre-training can transfer across datasets and tasks. The method is described in sufficient detail to reproduce, and the authors explicitly acknowledge limitations in Section 5.7 and Appendix L. However, the central 'task-agnostic five-sense' claim rests on a within-session evaluation with participant overlap (Section 5.1, Appendix F); the paper's own cross-subject numbers (Table 7) are much weaker. Because the conclusion is broader than the evidence currently supports, major revision is required.
major comments (4)
- [§5.1, Appendix F, Table 7] The headline DailySense result (avg F1 0.876) is obtained by randomly shuffling 4-second windows and applying an 80/20 split (Appendix F), so windows from the same participant/session appear in both training and test sets. Under a cross-subject split (Table 7), average F1 drops to 0.586, near chance for binary tasks, and PiMT no longer beats PatchTST on Video (0.612 vs 0.654) or Audio (0.578 vs 0.595). This indicates that the within-session numbers are substantially inflated by subject/session identity and/or task-correlated artifacts. The claim that PiMT learns task-agnostic representations across the five senses therefore needs a subject-disjoint, session-disjoint evaluation with artifact controls; Table 1 alone does not support it.
- [Appendix C, Appendix B, Table 3] The sensory protocols confound the target sensation with stereotyped motor behavior: taste involves sipping/holding chocolate milk vs vinegar; smell involves sniffing lavender vs vinegar; touch involves rubbing rough vs smooth materials. Appendix B states that the auricular electrodes capture facial-expression EMG, so mouth, face, and sniffing muscle activity is a natural confound. A model can classify these tasks from EMG/EOG artifacts without decoding the intended sensory state. Please provide artifact-baseline analyses (e.g., EMG-band-only classification, rest-period contrasts, or sham tasks) and/or an artifact-rejection pipeline; otherwise the 'five senses' interpretation of Table 1 is not established.
- [§5.6, Figure 7 vs Table 7] The text says that LOSO with the target subject excluded from pre-training achieves performance 'comparable to full pre-training,' but Table 7 reports a cross-subject average F1 of 0.586, far below 0.876. These statements are not reconciled. Please clarify the experimental setup for Figure 7 (is the target subject still present in fine-tuning? are the same task windows used?) and report the underlying numbers with error bars. As written, the two findings are difficult to reconcile and undermine the cross-subject generalization claim.
- [§3.2, §3.3, Appendix G, Table 5, Figure 4] The claim that PiMT is task-agnostic is weakened by the fact that several components (band ordering, patch size, λ weights, 12-band design) were selected based on performance on the same DailySense tasks. The saliency analysis then reads activation patterns back onto these tasks, so it does not provide independent confirmation of task generality. At minimum, please report which choices were made without access to DailySense labels and evaluate sensitivity to these choices on held-out tasks. The public benchmarks help, but Table 2 compares only two baselines, so the statement that PiMT 'consistently outperforms' is stronger than the evidence.
minor comments (6)
- [Figure 1 caption] The caption uses 'PiFT' in one place; this should be 'PiMT'.
- [§3.1] There is a duplicated word: 'tokens that that grant the model'.
- [§3.3] The notation for masked frequency reconstruction is inconsistent: the text writes 'producing ˆpMA and ˆpMP' but then uses 'ˆpA = D_MA' and 'ˆpP = D_MP'. Please define the masked targets consistently.
- [Appendix G vs §4] Appendix G says the representation model is trained on 'the entire 40 hours of free-living data,' while Section 4 and the abstract state 50 hours. Reconcile this discrepancy.
- [Appendix G] Typo: 'bi-directionanl mamba' should be 'bidirectional Mamba'.
- [Table 7] Cross-session standard deviations are very large (e.g., Video 0.697 ± 0.249, Audio 0.698 ± 0.188). This should be discussed, since it limits the strength of the cross-session robustness claim.
Circularity Check
No significant circularity: independent public benchmarks support the central claim; DailySense limitations are validity issues, not circular reasoning.
full rationale
PiMT's derivation chain is self-contained rather than circular. The input representation is a fixed 12-band filter bank defined from physiological references (Section 3.1); the encoder, reconstruction objectives, and fine-tuning are standard pipelines with no target label fed back into the definition of the method. The central generalization claim is anchored by four external benchmarks (DREAMER, SEED, Sleep-EDF, BCI Competition IV 2b, Table 2) with independently rerun baselines, so the conclusion does not rest on the authors' own prior result or on an identity between a fitted parameter and a reported prediction. The paper discloses the main threat to the DailySense headline: Appendix F's random 80/20 window shuffle can leak subject/session identity, and Table 7 reports cross-subject average F1 falling to 0.586, with PiMT losing to PatchTST on Video and Audio. That is a validity/leakage concern, not a circularity: no equation reduces a 'predicted' benchmark number to a fitted input by construction. Hyperparameter choices (λ values, patch size, token ordering) are described as empirically selected; if those choices were tuned on DailySense, the reported DailySense F1 is optimistically biased, but the same configuration is transferred to public datasets, which remain independent evidence. There is no load-bearing self-citation or imported uniqueness/ansatz chain. Therefore no circular step is identified.
Axiom & Free-Parameter Ledger
free parameters (6)
- 12-band filter bank edges =
See §3.1: EEG δ0.5–4/θ4–8/α8–13/β13–30/γ30–100; EMG 15–45/45–95/95–100; EOG 0.1–20; ECG; QRS 8–50
- lambda pretraining loss weights =
AE=2, MR=2, others=1
- patch size =
0.5 s
- token scanning order =
frequency-first, channel-second, time-last
- masking ratio =
50%
- Mamba depth =
8 (Appendix F) vs 16 (Appendix G) layers
axioms (4)
- domain assumption Canonical physiological bands from §3.1 are a sufficient decomposition for all relevant ExG task features.
- domain assumption Earphone electrodes (T7–T10, FT7–FT10, TP7–TP10 plus auricular/lateral electrodes) record genuine EEG/EOG/EMG with adequate SNR.
- domain assumption Four-second, randomly split windows are independent and the 80/20 per-participant split measures generalization.
- domain assumption Task labels correspond to the intended sensory modality and not to correlated motor/expression artifacts.
invented entities (1)
-
NeuroBuds earphone ExG sensing platform
no independent evidence
Cite this review
Pith. "Pith review of Beyond Hearing: Learning Task-Agnostic ExG Representations from Earphones via Physiology-Informed Tokenization." pith.science (2026). https://pith.science/paper/JSX7Z6DU
@misc{pith2026251020853,
author = {Pith},
title = {Pith review of: Beyond Hearing: Learning Task-Agnostic ExG Representations from Earphones via Physiology-Informed Tokenization},
year = {2026},
howpublished = {\url{https://pith.science/paper/JSX7Z6DU}},
note = {Machine review of arXiv:2510.20853}
}
read the original abstract
Electrophysiological (ExG) signals offer valuable insights into human physiology, yet building foundation models that generalize across everyday tasks remains challenging due to two key limitations: (i)~insufficient data diversity, as most ExG recordings are collected in controlled labs with bulky, expensive devices; and (ii)~task-specific model designs that require tailored processing (i.e., targeted frequency filters) and architectures, which limit generalization across tasks. To address these challenges, we introduce an approach for scalable, task-agnostic ExG monitoring in the wild. We collected 50 hours of unobtrusive free-living ExG data with an earphone-based hardware prototype to narrow the data diversity gap. At the core of our approach is Physiology-informed Multi-band Tokenization (PiMT), which decomposes ExG signals into 12 physiology-informed tokens, followed by a reconstruction task to learn robust representations. This enables adaptive feature recognition across the full frequency spectrum while capturing task-relevant information. Experiments on our new DailySense dataset, the first to enable ExG-based analysis across five human senses, together with four public ExG benchmarks, demonstrate that PiMT consistently outperforms state-of-the-art methods across diverse tasks.
Figures
Reference graph
Works this paper leans on
-
[5]
However, it benefits gaze regression, as the participant’s gaze is more likely to remain fixed within a shorter temporal window
A smaller patch size provides less contextual information for each classification window, which may limit performance. However, it benefits gaze regression, as the participant’s gaze is more likely to remain fixed within a shorter temporal window. In contrast, larger patch sizes offer more temporal context for classification tasks but increase the likelih...
2024
-
[7]
URLhttps://dx.doi.org/10.1088/1741-2552/aace8c
doi: 10.1088/1741-2552/aace8c. URLhttps://dx.doi.org/10.1088/1741-2552/aace8c. Robert Leeb, Clemens Brunner, G Müller-Putz, A Schlögl, and GJGUOT Pfurtscheller. Bci competi- tion 2008–graz data set b.Graz University of Technology, Austria, 16:1–6,
-
[11]
doi: 10.1109/RIVF60135.2023. 10471803. Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu. Brainbert: Self-supervised representation learning for intracranial recordings. InThe Eleventh International Conference on Learning Representations,
arXiv 2023
-
[12]
Xiuxin Xia, Yuchao Yang, Yan Shi, Wenbo Zheng, and Hong Men
URL https://proceedings.neurips.cc/paper_files/paper/ 2024/file/3fe2a777282299ecb4f9e7ebb531f0ab-Paper-Conference.pdf. Xiuxin Xia, Yuchao Yang, Yan Shi, Wenbo Zheng, and Hong Men. Decoding taste information in human brain: A temporal and spatial reconstruction data augmentation method coupled with taste eeg,
2024
-
[13]
Chaoqi Yang, M Westover, and Jimeng Sun
URLhttps://arxiv.org/abs/2307.05365. Chaoqi Yang, M Westover, and Jimeng Sun. Biot: Biosignal transformer for cross-data learning in the wild.Advances in Neural Information Processing Systems, 36:78240–78260,
-
[14]
Zhizhang Yuan, Fanqi Shen, Meng Li, Yuguo Yu, Chenhao Tan, and Yang Yang. Brainwave: A brain signal foundation model for clinical applications.arXiv preprint arXiv:2402.10251,
-
[16]
Each side includes five electrodes: the top ones on the left and right act as bias and reference, while the remaining eight serve as signal channels
13 A NEUROBUDSHARDWAREDESIGN To enable large-scale, in-the-wild ExG data collection, we built an earphone-based sensing platform consisting of two main components: Earphone-Shaped Sensing Array:To adopt a earhook-style form factor, We use a commercial earphone (Powerbeats PB123) as the backbone, and wrap conductive tape around the frame to form electrodes...
2017
-
[17]
(model: Tobii 4C Eye Tracker). • Auditory and Video Interest Inference:Inspired by SEED and DREAMER datasets (Zheng & Lu, 2015; Katsigiannis & Ramzan, 2018), this experiment explored the correlation between ExG signals and engagement with visual/auditory stimuli. Partici- pants were asked to watch or listen to video clips. After each session, they rated t...
2015
-
[18]
as our backbone architecture, which has demon- strated state-of-the-art performance across various time-series tasks (Zerveas et al., 2021; Song et al., 2022). To evaluate its effectiveness on ExG signals, we compare it against Transformer-based architecture, PatchTST (Nie et al., 2023), which showed strong performance in our main evaluation (Section 5.2)...
2021
-
[1996]
12 Hoang-Thuy-Tien V o, Thi-Nhu-Quynh Nguyen, Do Duc Cuong, and Tuan Van Huynh
doi: 10.1161/01.CIR.93.5.1043. 12 Hoang-Thuy-Tien V o, Thi-Nhu-Quynh Nguyen, Do Duc Cuong, and Tuan Van Huynh. Classification taste-eeg signals using base neural network. In2023 RIVF International Conference on Computing and Communication Technologies (RIVF), pp. 107–111,
-
[2000]
doi: 10.1109/10.867928. Daniel R Kramer, Krista Lamorie-Foote, Michael Barbaro, Morgan B Lee, Terrance Peng, Angad Gogia, George Nune, Charles Y Liu, Spencer S Kellis, and Brian Lee. Utility and lower limits of frequency detection in surface electrode stimulation for somatosensory brain-computer interface in humans.Neurosurgical focus, 48(2):E2,
-
[2006]
Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805,
-
[2015]
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang
doi: 10.1109/TAMD.2015.2431497. Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: efficient visual representation learning with bidirectional state space model. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org,
arXiv 2015
-
[2016]
Association for Computing Machinery. ISBN 9781450342636. doi: 10.1145/2994551.2994562. URL https://doi.org/10.1145/2994551.2994562. Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. InThe Eleventh International Confer- ence on Learning Representations,
-
[2017]
The standardized eeg electrode array of the ifcn.Clinical neurophysiology, 128(10):2070–2077,
Margitta Seeck, Laurent Koessler, Thomas Bast, Frans Leijten, Christoph Michel, Christoph Baum- gartner, Bin He, and Sándor Beniczky. The standardized eeg electrode array of the ifcn.Clinical neurophysiology, 128(10):2070–2077,
2070
-
[2018]
doi: 10.1109/JBHI.2017.2688239. B. Kemp, A.H. Zwinderman, B. Tuk, H.A.C. Kamphuisen, and J.J.L. Oberye. Analysis of a sleep- dependent neuronal feedback loop: the slow-wave microcontinuity of the eeg.IEEE Transactions on Biomedical Engineering, 47(9):1185–1194,
arXiv 2017
-
[2022]
Andrea Apicella, Pasquale Arpaia, Giovanna Mastrati, and Nicola Moccaldi
doi: 10.1109/ICEE55646.2022.9827406. Andrea Apicella, Pasquale Arpaia, Giovanna Mastrati, and Nicola Moccaldi. Eeg-based detection of emotional valence towards a reproducible measurement of emotions.Scientific Reports, 11(1): 21615,
arXiv 2022
-
[2023]
Multimodal multi- view spectral-spatial-temporal masked autoencoder for self-supervised emotion recognition
Pengxuan Gao, Tianyu Liu, Jia-Wen Liu, Bao-Liang Lu, and Wei-Long Zheng. Multimodal multi- view spectral-spatial-temporal masked autoencoder for self-supervised emotion recognition. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1926–1930. IEEE,
2024
-
[2024]
doi: 10.1016/j.aei.2024.102522
ISSN 1474-0346. doi: 10.1016/j.aei.2024.102522. URL https://doi.org/10.1016/j.aei.2024.102522. Iffat Farhana, Jungpil Shin, Shabbir Mahmood, Md Rabiul Islam, and Md Khademul Islam Molla. Emotion recognition using narrowband spatial features of electroencephalography.IEEE Access, 11:44019–44033,
arXiv 2024
This paper was first reviewed by deepseek-v4-flash on August 4, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.