MEEDAV: A Synchronous Web Viewer for EEG, Eye-Tracking and Speech Data

Jan Pij\'alek; Karel Vlk; Ond\v{r}ej Bojar

arxiv: 2605.18296 · v1 · pith:HQK53KIWnew · submitted 2026-05-18 · 💻 cs.HC

MEEDAV: A Synchronous Web Viewer for EEG, Eye-Tracking and Speech Data

Jan Pij\'alek , Karel Vlk , Ond\v{r}ej Bojar This is my paper

Pith reviewed 2026-05-20 08:51 UTC · model grok-4.3

classification 💻 cs.HC

keywords EEG visualizationeye-trackingspeech datamultimodal synchronizationweb-based viewerpsycholinguisticstime alignmentopen-source tool

0 comments

The pith

MEEDAV provides a browser-based viewer that aligns and visualizes EEG, eye-tracking and speech data for psycholinguistic experiments.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces MEEDAV as an open-source web application for synchronized display of EEG, eye-tracking and audio data gathered during language-related studies. It aligns signals from these different recording types, offers optional cleaning of the EEG traces, and generates interactive plots such as combined timelines, gaze heatmaps and correlation values. The system runs entirely in a web browser, processes information on demand, and adapts to various numbers of EEG channels without major reconfiguration. Researchers can select specific participants or stimuli and examine raw versus processed signals side by side. If the alignment and visualization steps function reliably, the tool lowers the technical barrier for exploring how brain activity, eye movements and spoken language interact in the same dataset.

Core claim

MEEDAV performs time alignment across EEG, eye-tracking and audio modalities collected in psycholinguistic research, supplies optional ICA-based EEG denoising, and delivers interactive Plotly visualisations that include unified EEG-audio-gaze timelines, gaze-intensity plots, event markers and spatial heatmaps of fixation and saccade patterns, all accessible through a modular web backend that supports local files or remote streaming.

What carries the argument

The channel-agnostic processing pipeline together with real-time time alignment that merges EEG, gaze and audio streams into a single interactive browser interface.

If this is right

Researchers gain the ability to inspect raw and denoised EEG alongside gaze patterns and audio events within one view.
Cross-modal correlations can be calculated directly inside the viewer for immediate inspection.
The same interface works for both four-channel wearable EEG and higher-density electrode setups.
Data access is possible from local storage or from online repositories without installing dedicated software.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

A lightweight web viewer of this kind could allow smaller labs to analyse multimodal recordings without purchasing specialised desktop packages.
The alignment and plotting methods might be extended to additional sensor streams such as video or physiological measures.
If adapted for live input, the system could support real-time monitoring during ongoing experiments.

Load-bearing premise

Accurate time alignment across EEG, eye-tracking and audio can be performed reliably in real time for arbitrary data formats and channel counts without introducing alignment errors or requiring dataset-specific tuning.

What would settle it

Processing a test dataset that contains independently verified ground-truth timestamps for EEG, eye-tracking and audio events and then checking whether the displayed timelines in MEEDAV exhibit systematic offsets larger than the sampling intervals of the original recordings.

Figures

Figures reproduced from arXiv: 2605.18296 by Jan Pij\'alek, Karel Vlk, Ond\v{r}ej Bojar.

**Figure 1.** Figure 1: High-level data flow in MEEDAV. Blue = data manipulation, green = processing, yellow = browser visualisation. Why we adopted the EMMT layout. MEEDAV was conceived for exploratory work on the EMMT corpus, and therefore using the layout of the dataset’s preprocessed folder was the most straightforward option. Required modalities. Only the EEG stream is strictly required; eye-tracking and/or audio may be omi… view at source ↗

**Figure 2.** Figure 2: MEEDAV interface: synchronised EEG, audio, gaze-intensity bars, and event markers with unified hover cursor. Tabs on the left toggle ICA-clean traces, KDE heatmaps, and the participant dashboard. ˜Ik = P t∈Bk mt maxj P t∈Bj mt . (2) The implementation employs vectorized pandas.diff() with group-by summation and scaling. The resulting ˜Ik values are displayed as signed bar plots, where positive and negati… view at source ↗

read the original abstract

MEEDAV is an open-source web-based application for the synchronised visualisation of electroencephalography (EEG), eye-tracking, and audio data collected in psycholinguistic research. While originally developed for the Eyetracked Multi-Modal Translation (EMMT) corpus, which uses four-channel EEG data from the Muse 2 headband, MEEDAV also supports higher-density EEG setups thanks to its channel-agnostic processing pipeline. The system performs time alignment across all modalities and provides optional ICA-based EEG denoising. It features interactive Plotly visualisations, including unified EEG-audio-gaze timelines, gaze-intensity plots, event markers, and spatial heatmaps of fixation/saccade patterns. Researchers can filter by participant and stimulus, inspect raw versus cleaned signals, and compute cross-modal correlations. All processing is handled in real time, with a modular backend that supports local file access or GitHub-based streaming. Although initially tailored to the structure of the EMMT dataset, MEEDAV demonstrates a generalisable approach to multimodal data exploration and offers a lightweight, browser-accessible solution for cognitive neuroscience and translation studies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

MEEDAV is a web viewer for synced EEG, eye-tracking and audio that combines modalities in the browser but provides no tests to support its generalisability claims.

read the letter

The main point is that this paper describes a new web app called MEEDAV for viewing synchronized EEG, eye-tracking, and speech data together, built for the EMMT corpus but presented as usable more broadly. It uses Plotly for interactive timelines, gaze heatmaps, and event markers, with real-time alignment, optional ICA denoising, and support for pulling data from GitHub or local files. The channel-agnostic pipeline is meant to handle higher-density EEG setups beyond the original four-channel Muse 2 headband. These features add up to a lightweight browser tool that could save researchers some setup time when exploring multimodal psycholinguistic data. The modular backend and filtering options by participant or stimulus are straightforward conveniences that the description handles clearly. What stands out as useful is the unified view across modalities plus the ability to inspect raw versus cleaned signals and compute some cross-modal correlations without leaving the browser. That combination is not entirely novel on its own, but packaging it this way for translation studies and cognitive neuroscience work is a practical step. The soft spots are straightforward and fairly large for a tools paper. The generalisability claim depends on the alignment working reliably across formats and channel counts, yet the text gives no alignment error numbers, no runs on external datasets, and no checks with denser EEG arrays. Without those, it is difficult to judge whether the pipeline needs hidden tuning or produces offsets that would break downstream analysis. There are also no performance benchmarks, no user evaluation, and almost no implementation specifics on how the synchronization actually runs in real time. This leaves the central assertions resting on description alone. The paper is aimed at researchers who already collect EEG, gaze, and audio in similar setups and want a quick, install-light way to look at their data side by side. Someone hunting for an off-the-shelf visualization aid might pick up a few ideas or even try the code if it is released. A reader expecting rigorous validation or novel methods will find little to engage with. I would send it to peer review in a tools or software track, but only with the clear expectation that the authors add concrete validation runs, alignment metrics, and code availability before acceptance.

Referee Report

2 major / 2 minor

Summary. The manuscript describes MEEDAV, an open-source web-based application for synchronized visualization of EEG, eye-tracking, and audio data from psycholinguistic experiments. Originally built for the 4-channel Muse 2 EMMT corpus, it claims a channel-agnostic pipeline that supports higher-density EEG, performs real-time time alignment across modalities, offers optional ICA denoising, and provides interactive Plotly visualizations including unified timelines, gaze heatmaps, event markers, and cross-modal correlations. The system supports participant/stimulus filtering, raw vs. cleaned signal inspection, and a modular backend for local files or GitHub streaming, positioning itself as a generalisable lightweight tool for cognitive neuroscience and translation studies.

Significance. If the real-time alignment and channel-agnostic claims prove robust, MEEDAV could offer a practical, browser-accessible open-source resource that reduces setup overhead for multimodal data exploration. The open-source release and focus on unified timelines are constructive contributions for the target communities. However, the absence of any empirical validation or benchmarks substantially tempers the potential impact.

major comments (2)

Abstract: The central claim that MEEDAV 'demonstrates a generalisable approach' via its channel-agnostic pipeline and real-time alignment for arbitrary formats and channel counts is unsupported by any reported validation runs, alignment error metrics, tests on external datasets, or higher-channel-count experiments. This directly undercuts the generalisability assertion that the work rests upon.
Abstract and overall manuscript: No implementation details, performance benchmarks, ground-truth alignment validation, or user evaluation are provided to substantiate the reliability of real-time multimodal synchronization or ICA denoising for arbitrary data.

minor comments (2)

Clarify whether the system performs any speech-specific processing beyond raw audio waveform display, given the title's reference to 'Speech Data' versus the abstract's use of 'audio data'.
Consider adding a short architecture or implementation subsection that lists key libraries, data ingestion steps, and installation instructions to improve reproducibility for potential users.

Simulated Author's Rebuttal

2 responses · 2 unresolved

Thank you for the constructive review of our manuscript. We appreciate the referee's identification of areas where the claims of generalisability and implementation details require clarification or additional support. We address each major comment below and outline the revisions we will make to the manuscript.

read point-by-point responses

Referee: Abstract: The central claim that MEEDAV 'demonstrates a generalisable approach' via its channel-agnostic pipeline and real-time alignment for arbitrary formats and channel counts is unsupported by any reported validation runs, alignment error metrics, tests on external datasets, or higher-channel-count experiments. This directly undercuts the generalisability assertion that the work rests upon.

Authors: We agree that the abstract's phrasing implies a stronger empirical demonstration than is provided. The channel-agnostic design relies on metadata-driven parsing of EEG channel counts and a modular loader that accommodates different sampling rates and formats through configuration, while alignment uses shared experimental timestamps and event logs for cross-modal synchronisation. These were developed and tested internally on the 4-channel EMMT corpus. In the revision we will reword the abstract to describe MEEDAV as offering 'a channel-agnostic pipeline designed to support general use' and will add a short pipeline-design subsection with qualitative explanation of adaptability. We will also insert an explicit limitations paragraph noting the absence of external-dataset tests and higher-channel experiments. revision: yes
Referee: Abstract and overall manuscript: No implementation details, performance benchmarks, ground-truth alignment validation, or user evaluation are provided to substantiate the reliability of real-time multimodal synchronization or ICA denoising for arbitrary data.

Authors: The current manuscript outlines the overall architecture (Plotly frontend, Python backend, MNE-Python for ICA) but does not supply low-level implementation details or quantitative metrics. We will expand the methods section with pseudocode for the timestamp-based alignment routine and basic latency observations obtained during development on standard hardware. ICA denoising follows the standard MNE implementation with user-selectable components; we will document the default parameters used. However, formal ground-truth alignment validation across arbitrary datasets and dedicated user studies fall outside the scope of this tool-description paper. We will clarify the intended contribution as an open-source prototype rather than a validated benchmark study. revision: partial

standing simulated objections not resolved

Quantitative alignment error metrics or performance benchmarks on external or higher-density EEG datasets
Formal user evaluation or ground-truth validation experiments

Circularity Check

0 steps flagged

No circularity: software description with no derivations or self-referential reductions

full rationale

The paper is a description of the MEEDAV web application for synchronized visualization of EEG, eye-tracking, and audio data. It contains no equations, fitted parameters, predictions, or derivation chains. The generalisability claim is presented as a direct consequence of the described channel-agnostic pipeline and real-time alignment features, without any reduction to self-citations, self-definitions, or fitted inputs. No load-bearing steps reduce by construction to the paper's own inputs, making the work self-contained as a practical software contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical model, derivation, or empirical claim is advanced; the paper is a tool description, so the ledger contains no entries.

pith-pipeline@v0.9.0 · 5733 in / 999 out tokens · 38347 ms · 2026-05-20T08:51:43.832327+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

[1]

EMMT:Asimultaneouseye-tracking,4-electrode EEG and audio corpus for multi-modal reading and translation scenarios

SunitBhattacharyaetal.“EMMT:Asimultaneouseye-tracking,4-electrode EEG and audio corpus for multi-modal reading and translation scenarios”. In:arXiv preprint arXiv:2204.02905(2022).url:https://arxiv.org/ abs/2204.02905

work page arXiv 2022
[2]

Independent component analysis, A new concept?

Pierre Comon. “Independent component analysis, A new concept?” In: Signal Processing36.3 (1994). Higher Order Statistics, pp. 287–314.issn: 0165-1684.doi:https : / / doi . org / 10 . 1016 / 0165 - 1684(94 ) 90029 - 9.url:https : / / www . sciencedirect . com / science / article / pii / 0165168494900299

work page 1994
[3]

Coregistration of eye movements and EEG in natu- ral reading: Analyses & Review

Olaf Dimigen et al. “Coregistration of eye movements and EEG in natu- ral reading: Analyses & Review”. In:Journal of Experimental Psychology: General140.4 (2011), pp. 552–572.doi:10.1037/a0023885

work page doi:10.1037/a0023885 2011
[4]

Matplotlib: A 2D graphics environment

J. D. Hunter. “Matplotlib: A 2D graphics environment”. In:Computing in Science & Engineering9.3 (2007), pp. 90–95.doi:10.1109/MCSE.2007. 55

work page doi:10.1109/mcse.2007 2007
[5]

Version 0.11.0

Brian McFee et al.librosa/librosa: 0.11.0. Version 0.11.0. Mar. 2025.doi: 10.5281/zenodo.15006942.url:https://doi.org/10.5281/zenodo. 15006942

work page doi:10.5281/zenodo.15006942.url:https://doi.org/10.5281/zenodo 2025
[6]

Data Structures for Statistical Computing in Python

Wes McKinney. “Data Structures for Statistical Computing in Python”. In:Proceedings of the 9th Python in Science Conference. Ed. by Stéfan van der Walt and Jarrod Millman. 2010, pp. 56–61.doi:10.25080/Majora- 92bf1922-00a

work page doi:10.25080/majora- 2010
[7]

OpenSync: An open-source platform for synchroniz- ing multiple measures in neuroscience experiments

Moein Razavi et al. “OpenSync: An open-source platform for synchroniz- ing multiple measures in neuroscience experiments”. In:Journal of neuro- science methods369 (2022), p. 109458

work page 2022
[8]

Version lat- est

The pandas development team.pandas-dev/pandas: Pandas. Version lat- est. Feb. 2020.doi:10.5281/zenodo.3509134.url:https://doi.org/ 10.5281/zenodo.3509134. MEEDAV: EEG, ET and Speech Viewer 9

work page doi:10.5281/zenodo.3509134.url:https://doi.org/ 2020
[9]

Altair: Interactive Statistical Visualizations for Python

Jacob VanderPlas et al. “Altair: Interactive Statistical Visualizations for Python”. In:Journal of Open Source Software3.32 (2018), p. 1057.doi: 10.21105/joss.01057.url:https://doi.org/10.21105/joss.01057

work page doi:10.21105/joss.01057.url:https://doi.org/10.21105/joss.01057 2018
[10]

seaborn: statistical data visualization

Michael L. Waskom. “seaborn: statistical data visualization”. In:Journal of Open Source Software6.60 (2021), p. 3021.doi:10.21105/joss.03021. url:https://doi.org/10.21105/joss.03021

work page doi:10.21105/joss.03021 2021

[1] [1]

EMMT:Asimultaneouseye-tracking,4-electrode EEG and audio corpus for multi-modal reading and translation scenarios

SunitBhattacharyaetal.“EMMT:Asimultaneouseye-tracking,4-electrode EEG and audio corpus for multi-modal reading and translation scenarios”. In:arXiv preprint arXiv:2204.02905(2022).url:https://arxiv.org/ abs/2204.02905

work page arXiv 2022

[2] [2]

Independent component analysis, A new concept?

Pierre Comon. “Independent component analysis, A new concept?” In: Signal Processing36.3 (1994). Higher Order Statistics, pp. 287–314.issn: 0165-1684.doi:https : / / doi . org / 10 . 1016 / 0165 - 1684(94 ) 90029 - 9.url:https : / / www . sciencedirect . com / science / article / pii / 0165168494900299

work page 1994

[3] [3]

Coregistration of eye movements and EEG in natu- ral reading: Analyses & Review

Olaf Dimigen et al. “Coregistration of eye movements and EEG in natu- ral reading: Analyses & Review”. In:Journal of Experimental Psychology: General140.4 (2011), pp. 552–572.doi:10.1037/a0023885

work page doi:10.1037/a0023885 2011

[4] [4]

Matplotlib: A 2D graphics environment

J. D. Hunter. “Matplotlib: A 2D graphics environment”. In:Computing in Science & Engineering9.3 (2007), pp. 90–95.doi:10.1109/MCSE.2007. 55

work page doi:10.1109/mcse.2007 2007

[5] [5]

Version 0.11.0

Brian McFee et al.librosa/librosa: 0.11.0. Version 0.11.0. Mar. 2025.doi: 10.5281/zenodo.15006942.url:https://doi.org/10.5281/zenodo. 15006942

work page doi:10.5281/zenodo.15006942.url:https://doi.org/10.5281/zenodo 2025

[6] [6]

Data Structures for Statistical Computing in Python

Wes McKinney. “Data Structures for Statistical Computing in Python”. In:Proceedings of the 9th Python in Science Conference. Ed. by Stéfan van der Walt and Jarrod Millman. 2010, pp. 56–61.doi:10.25080/Majora- 92bf1922-00a

work page doi:10.25080/majora- 2010

[7] [7]

OpenSync: An open-source platform for synchroniz- ing multiple measures in neuroscience experiments

Moein Razavi et al. “OpenSync: An open-source platform for synchroniz- ing multiple measures in neuroscience experiments”. In:Journal of neuro- science methods369 (2022), p. 109458

work page 2022

[8] [8]

Version lat- est

The pandas development team.pandas-dev/pandas: Pandas. Version lat- est. Feb. 2020.doi:10.5281/zenodo.3509134.url:https://doi.org/ 10.5281/zenodo.3509134. MEEDAV: EEG, ET and Speech Viewer 9

work page doi:10.5281/zenodo.3509134.url:https://doi.org/ 2020

[9] [9]

Altair: Interactive Statistical Visualizations for Python

Jacob VanderPlas et al. “Altair: Interactive Statistical Visualizations for Python”. In:Journal of Open Source Software3.32 (2018), p. 1057.doi: 10.21105/joss.01057.url:https://doi.org/10.21105/joss.01057

work page doi:10.21105/joss.01057.url:https://doi.org/10.21105/joss.01057 2018

[10] [10]

seaborn: statistical data visualization

Michael L. Waskom. “seaborn: statistical data visualization”. In:Journal of Open Source Software6.60 (2021), p. 3021.doi:10.21105/joss.03021. url:https://doi.org/10.21105/joss.03021

work page doi:10.21105/joss.03021 2021