{"id":"582965d1-2b49-4e85-9170-ba9da5534dcd","arxiv_id":"2608.05966","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A modular, real-time pipeline co-registers gaze, EEG, and interaction data at word and sentence level, and uses the resulting metrics to select text passages for post-task self-report.","lead":"This paper describes a web-based workflow that synchronizes eye tracking, EEG, and mouse or keyboard input during naturalistic online reading experiments. It is a methods paper, not a results paper, and the use case is presented without empirical findings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on per-word gaze-to-text mapping and fixation-triggered EEG segmentation, but the cited validation is component-level and the use case reports no end-to-end accuracy evidence, so the general-basis claim is not yet substantiated.","rationale":"This is a systems and methods paper, so its central claim should be evaluated as an engineering claim: the modules exist and the end-to-end workflow runs with sufficient temporal and assignment accuracy for word-level EEG segmentation. There is real independent support for the EyeLiveMetrics component via a prior publication, and LSL is a widely used synchronization protocol, but both are component-level. What is missing is evidence that the integrated pipeline produces reliable word-level fixation onsets in LSL time under the naturalistic conditions of §5. The manuscript itself flags the key limitation in the Discussion: \"Accurate fixation detection is essential whenever downstream analyses depend on fixation-locked windows.\" That self-identified limitation, combined with the absence of any reported quantitative output from the use case, is precisely the load-bearing weakness. My proposed check targets both per-fixation assignment accuracy and timestamp alignment; if those pass, the central claim would be materially supported. Until then, the reader's CONDITIONAL verdict is appropriate: the paper can be accepted with tempered claims, explicit acknowledgment of the component-level validation limit, and release of the code and data needed for end-to-end verification.","tokens_in":9852,"tokens_out":5457,"duration_ms":59219,"concrete_test":"Using the same news-reading setup as §5, record at least 10 participants and compare, for every fixation, the word assigned by EyeLiveMetrics against an offline reference such as Tobii Pro Lab's AOI mapping or manual annotation of a fixation subset. Compute per-fixation word-assignment agreement (e.g., Cohen's kappa) in addition to Pearson correlations; if agreement on word indices is below 95% or shows a systematic one-word lag, the fixation-locked EEG epochs in §3.5 are biased. As a second check, emit a known display-synchronization pulse (e.g., a screen flash) and measure EyeLiveMetrics-derived fixation-onset timestamps against the LSL-timestamped gaze stream; a systematic offset larger than two gaze samples (~6.7 ms at 300 Hz) would violate the alignment assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that EyeLiveMetrics' online gaze-to-word mapping is accurate and LSL-time-aligned enough to drive word-level EEG epoching and same-session sentence selection. This is not established by the cited evidence. The prior validation [9] reports Pearson correlations ≤ 0.96 against Tobii Pro Lab, but those are correlations of aggregate fixation metrics, not per-fixation word-assignment agreement. High aggregate correlations can coexist with systematic word-level misassignment, such as a one-word offset around short frequent words, and any such misassignment propagates directly into fixation-locked EEG epochs and into the selection criteria of §5 (longest fixation duration; theta power 500–1000 ms and 1000–1500 ms after first fixation onset). The Discussion explicitly states that \"Accurate fixation detection is essential whenever downstream analyses depend on fixation-locked windows,\" yet no end-to-end validation is reported: §5 describes the procedure without reporting whether the integrated pipeline ran to completion, how many participants were processed, or any output metrics. It is also not specified whether EyeLiveMetrics' fixation onsets carry LSL timestamps or are later re-aligned; without that, the phrase \"as gaze and EEG data are aligned with LSL\" in §3.5 remains an assumption rather than a demonstrated property. The claim that the workflow \"provides a general basis for multimodal research on reading\" therefore requires support that the paper does not currently supply.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a modular, web-based workflow for conducting naturalistic online-reading experiments with synchronized eye tracking, EEG, and interaction data. The workflow uses Lab Streaming Layer (LSL) for temporal alignment, the EyeLiveMetrics browser plugin for gaze-to-word mapping, and MNE-Python for EEG preprocessing and fixation-triggered epoching. A use case on selective exposure in online news reading is described, in which sentences are selected for post-task self-report based on fixation duration and theta-band power changes. The manuscript claims that the workflow 'provides a general basis for multimodal research on reading' and related cognitive processes.","tokens_in":10065,"tokens_out":2434,"duration_ms":27391,"significance":"If the workflow were demonstrated to be reliable end-to-end, it would be a valuable contribution to the growing literature on co-registered eye-tracking and EEG in naturalistic reading, particularly because it targets real-time or near-real-time integration and within-session stimulus selection. The use of existing, open components (LSL, MNE-Python, EyeLiveMetrics) and the modular design are sensible and potentially reusable. However, the paper's central claim is currently supported only by component-level validation of the gaze-to-word mapping from prior work; there is no empirical evidence that the integrated pipeline actually runs as described, and the Discussion itself concedes that accurate fixation detection is essential but does not quantify its accuracy in this setup. The significance therefore depends on the outcome of additional validation that is not present in the manuscript.","major_comments":[{"comment":"The central claim that the workflow 'provides a general basis for multimodal research on reading' (Abstract, also echoed in §6 and §7) is not supported by any end-to-end validation. Section 5, which is presented as a use case, describes hardware, procedures, and selection criteria but reports no number of participants, no indication of whether the pipeline ran to completion for any session, no fixation-to-word agreement rates, no EEG epoch yields, and no output metrics of any kind. The Discussion in §6 explicitly states that 'Accurate fixation detection is essential whenever downstream analyses depend on fixation-locked windows,' yet no accuracy evidence for the integrated pipeline is provided. As written, the paper demonstrates an architecture, not a working workflow.","section":"§5 and §6"},{"comment":"The load-bearing premise for fixation-triggered EEG segmentation is that the EyeLiveMetrics gaze-to-word mapping is correct at the level of individual fixations. The cited validation in [9] reports Pearson correlations 'of ≤0.96 across all fixation metrics' against Tobii Pro Lab, but aggregate fixation-metric correlations do not establish per-fixation word-assignment agreement. Systematic misassignments, such as a consistent one-word offset around short or high-frequency words, can leave aggregate correlations high while substantially biasing the word-level epochs described in §3.5 and the sentence-selection criteria in §5 (longest fixation duration; theta power in the 500–1000 ms and 1000–1500 ms windows). The manuscript needs to report a per-fixation or per-word assignment accuracy measure, or at minimum justify why the aggregate validation suffices.","section":"§3.4, §3.5, and §6"},{"comment":"The temporal alignment between EyeLiveMetrics's gaze output and the LSL-recorded EEG stream is not specified. Section 3.5 asserts that 'as gaze and EEG data are aligned with LSL, we extract the fixation onset for each fixated word,' but it is unclear whether EyeLiveMetrics fixations are timestamped in the LSL clock, whether they are later re-aligned, and whether any clock-offset correction is applied. Without this information, the synchronization claim is an assumption rather than a demonstrated property of the system. The paper should describe the exact timestamping path from the raw gaze samples to the word-level fixation onsets used for epoching.","section":"§3.3, §3.5"},{"comment":"The novelty claim that 'no existing approach integrates the co-registration of EEG, gaze, and other sensor data with a real-time analysis pipeline for naturalistic online reading' is stated without a systematic comparison to existing co-registration toolkits and pipelines (e.g., Dimigen et al. [4] and subsequent work on fixation-related potentials in natural reading). If the claim is intended to cover only the specific combination of browser-DOM gaze mapping plus real-time feedback selection, it should be scoped more precisely; otherwise, the manuscript overstates its novelty relative to the cited literature.","section":"§2.4"}],"minor_comments":[{"comment":"In the Figure 2 caption, 'extend of user reported feedback' should be 'extent of user-reported feedback'.","section":"§3.2, Figure 2 caption"},{"comment":"The sentence beginning 'By combining multimodal co-registration...' is long and lists several benefits; it would be clearer split into two sentences.","section":"§1, last paragraph of Intro"},{"comment":"The phrase 'we collected participants’ general attitude' should be 'we measured participants’ general attitudes' for clarity, and the scale reference [24] appears to be a book on political attitudes; please clarify whether this is the actual instrument used.","section":"§5, Pre-experiment"},{"comment":"The sentence 'Using these criteria, a maximum of 60 sentences were rated per participant' is ambiguous about whether the maximum was reached in practice; report the actual distribution or state that no data are reported.","section":"§5, Post-Experiment"},{"comment":"The footnote says 'Only two out of seven datasets (SEED [33, 5], and SEED-IV) include gaze information,' but the main text says 'only two datasets include eye-tracking data (SEED-IV [32] and MAHNOB-HCI [29])'. These two counts are inconsistent and should be reconciled.","section":"§2.1, footnote 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a systems/methods contribution, and its acceptance should hinge on whether the authors can provide at least minimal empirical evidence that the integrated pipeline runs correctly. The current manuscript does not even report a pilot run or a single participant's data flow. If the authors can supply such evidence (e.g., a small validation dataset with fixation-to-word agreement and EEG epoch counts), the contribution may be worth publishing; without it, the central 'general basis' claim is unsupported. The paper may also benefit from a more direct comparison with existing co-registration pipelines to sharpen its claimed novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Genuinely novel here is the integration: a single-session workflow that maps gaze to the DOM in real time, uses fixation onsets to segment EEG, and then selects sentences for self-report based on eye and EEG metrics. That combination doesn't exist in the prior work they review, which mostly used video or predefined trials. The architecture is clearly described and sensible—LSL for sync, EyeLiveMetrics for gaze-to-text, MNE for EEG, elfen for linguistic features—and the authors are upfront in the Discussion that fixation accuracy is the load-bearing assumption. That honesty matters, and the use case (selective exposure in news reading) is well motivated.\n\nThe soft spots are real but not fatal. The central claim that this 'provides a general basis' for multimodal reading research is not supported by end-to-end evidence. The use case is described procedurally but no results are reported: no participant count, no indication that the integrated pipeline actually ran to completion, no output metrics. The validation cited for EyeLiveMetrics is component-level, and the Pearson correlations ≤0.96 are aggregate measures, not per-fixation word-assignment agreement. The stress-test note is right that the paper doesn't specify whether EyeLiveMetrics' fixation onsets carry LSL timestamps or are re-aligned later; without that, the alignment in Section 3.5 is an assumption rather than a demonstrated property. Finally, no code or data are provided, which hurts a methods paper considerably.\n\nThese are addressable. A re-framed version that presents this as a proposed workflow with a detailed description, plus a pilot or benchmark showing the gaze-to-word mapping works under realistic web conditions, would be solid. The authors should also tone down 'fully integrated, real-time' and 'demonstrating' in the conclusion, since they describe rather than demonstrate.\n\nWho's this for? Researchers who want a turnkey starting point for naturalistic reading experiments with EEG and eye tracking. It deserves a serious referee, not a desk reject—the gap is real and the architecture is credible. I'd recommend conditional acceptance with requests for code/data, an explicit statement of the use case's actual runs, and a clear validation of the gaze-to-word component within the integrated LSL pipeline. My own verdict mirrors that: conditional.","headline":"A genuinely integrative pipeline for naturalistic reading studies, but the 'general basis' claim outruns the evidence; conditional acceptance with code, data, and tempered language would make it solid.","tokens_in":10626,"tokens_out":2895,"would_cite":false,"duration_ms":30500,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper presents a modular workflow that synchronizes eye tracking, EEG, and interaction data to link gaze to words, sentences, and areas of interest in freely chosen online text.","keywords":["eye tracking","EEG","multimodal co-registration","naturalistic reading","selective exposure","fixation-related potentials","real-time analysis","web-based experiments"],"falsifier":"Run the complete pipeline on a set of known web pages read by participants whose fixations are manually annotated word-by-word, compare the pipeline's fixation-to-word assignments with the manual annotations, and compute the fraction of fixation onsets assigned to a neighboring word or sentence; if that fraction is non-trivial or grows with smaller fonts or dynamic layouts, the fixation-locked EEG results cannot be trusted.","tokens_in":9611,"feed_emoji":"🧠","tokens_out":7167,"duration_ms":68689,"temperature":0.7,"pith_summary":"This paper introduces a reusable, web-based workflow for real-time multimodal reading experiments, and claims that it makes naturalistic online reading accessible to synchronized measurement. The workflow combines eye tracking, EEG, and mouse/keyboard interaction through a common time-synchronization protocol, maps gaze to the words, sentences, and areas of interest of the text shown in the browser, and uses word-fixation onsets to segment the brain signal. Gaze-derived metrics are computed live, while EEG and other streams are processed immediately after each task, so behavioral, neural, and linguistic features can drive the selection of sentences for post-task rating or labeling in the same session. The authors apply the system to selective exposure in news search and reading and present it as a template for other naturalistic-reading research.","feed_headline":"Workflow syncs eye gaze, brain signals, and clicks during web reading","feed_subtitle":"A single lab session produces synchronized word-level gaze and EEG metrics, then selects passages for rating on the spot.","key_machinery":"The load-bearing mechanism is the linking of the three data types in both time and space. A common-clock synchronization protocol records eye tracker, EEG, and input-device streams into one file; a browser plug-in maps raw gaze coordinates to the displayed text and emits word-fixation onsets; the EEG stream is segmented on those onsets and cleaned with artifact rejection and time-frequency analysis; and a linguistic feature extractor annotates the read text. The workflow's reusability rests on the modularity of these components, each of which can be swapped or extended.","core_discovery":"The central claim is that a modular pipeline can integrate real-time gaze-to-text mapping with post-hoc fixation-triggered EEG segmentation, and that this integration makes multimodal experiments on self-selected online reading practical rather than confined to tightly controlled trial-based paradigms. In the reported use case, participants search and read news on two topics; sentences are selected for follow-up ratings based on the longest fixation durations or the strongest theta-band power increases after first-fixation onset. The paper presents the setup as both a concrete study environment and a general template: the sensor streams, analysis scripts, and selection criteria are replaceable, so the same structure can be adapted to emotion, confusion, comprehension, or language-learning questions.","pith_inferences":["One open risk is that the gaze-to-word assignment used as the backbone has been validated on its own but not as part of the full pipeline; if fixation onsets are assigned to the wrong word at sentence boundaries, the EEG epochs and the sentences selected for rating would shift.","We would add that a stability analysis of the sentence-selection criteria across thresholds would show whether the same passages are consistently flagged, since different plausible criteria might select different sentence subsets.","The architecture could support fully adaptive experiments in which EEG features computed between sessions change the stimuli shown in later tasks, because the components already compute metrics in near-real-time.","For low-density mobile EEG settings, the paper itself notes that artifact correction is less reliable, so an extension using the available eye-openness signal for regression-based ocular correction may be needed before the workflow transfers to mobile setups."],"forward_implications":["Gaze metrics can be computed live during a reading session and immediately used to select sentences for post-task ratings, closing the loop between measurement and follow-up questions in one visit.","EEG can be segmented on the first fixation of each word in free reading, so neural responses such as theta power changes or fixation-related potentials can be tied to specific linguistic content without predefined trials.","Because the data are synchronized to a common clock, mouse and keyboard events can be aligned with gaze and brain signals, enabling analyses of search behavior alongside reading.","With the sensors and analysis modules replaceable, the same workflow can be adapted to study topics beyond selective exposure, such as emotion during reading, comprehension difficulty, or foreign-language processing.","Linguistic annotation of the read text gives researchers partial control over uncontrolled naturalistic stimuli, allowing them to account for text properties in later analysis."],"supporting_citations":[{"why":"Supplies the gaze-to-word mapping plug-in and the component-level validation that the whole pipeline inherits.","marker":"[9]"},{"why":"Provides the synchronization protocol that aligns all streams to a common clock.","marker":"[20]"},{"why":"Establishes co-registration of eye movements and EEG as the approach for naturalistic reading and identifies sources of noise that affect fixation-locked epochs.","marker":"[4]"},{"why":"Provides the EEG processing library used for filtering, artifact removal, and epoching.","marker":"[8]"},{"why":"Provides the linguistic feature extraction used to annotate the read text.","marker":"[25]"},{"why":"Supplies the artifact rejection method used to exclude bad EEG epochs.","marker":"[11]"},{"why":"Motivates the theta-band time windows used as sentence-selection criteria.","marker":"[21]"},{"why":"Supplies the faster independent-component-analysis algorithm used for artifact removal.","marker":"[1]"}],"fun_headline_variants":["Modular pipeline unifies gaze, EEG, and clicks for web reading studies","Real-time gaze and brain data sync in naturalistic online reading","One lab session yields word-level gaze and EEG, then picks rating passages","Workflow links gaze to text, segments EEG, and selects passages","Modular workflow makes naturalistic reading experiments practical"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the eye tracker's gaze-to-word assignment remains accurate enough in real web pages that every word-fixation onset marks the correct text; if a substantial share of fixations land on the wrong word or sentence, the EEG epochs, the selected sentences, and the neural conclusions would all be distorted.","fun_headline_variants_meta":{"raw":{"variants":["Modular pipeline unifies gaze, EEG, and clicks for web reading studies","Real-time gaze and brain data sync in naturalistic online reading","One lab session yields word-level gaze and EEG, then picks rating passages","Workflow links gaze to text, segments EEG, and selects passages","Modular workflow makes naturalistic reading experiments practical"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001025,"raw_usage":{"total_tokens":4275,"prompt_tokens":854,"completion_tokens":3421,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":470,"completion_tokens_details":{"reasoning_tokens":3332}},"tokens_in":470,"tokens_out":3421,"duration_ms":21887,"temperature":1.0,"reasoning_tokens":3332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T20:09:41.542435+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the complete pipeline on a set of known web pages read by participants whose fixations are manually annotated word-by-word, compare the pipeline's fixation-to-word assignments with the manual annotations, and compute the fraction of fixation onsets assigned to a neighboring word or sentence; if that fraction is non-trivial or grows with smaller fonts or dynamic layouts, the fixation-locked EEG results cannot be trusted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the gaze-to-word mapping plug-in and the component-level validation that the whole pipeline inherits."},{"cited_title":"Coregistration of eye movements and eeg in natural reading: analyses and review","cited_arxiv_id":null,"evidence_quote":"Establishes co-registration of eye movements and EEG as the approach for naturalistic reading and identifies sources of noise that affect fixation-locked epochs."},{"cited_title":"Faster ica under orthogonal constraint, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE Press, Calgary, AB, Canada","cited_arxiv_id":null,"evidence_quote":"Supplies the faster independent-component-analysis algorithm used for artifact removal."}],"review_version":1}