{"id":"d48567cc-5891-4ee0-8808-dbf06f906bf9","arxiv_id":"2510.02171","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"witheFlow continuously maps a live performer's physiological state and audio valence-arousal onto the gains of DAW effect channels via interpretable YAML rules.","lead":"witheFlow is a proof-of-concept laptop program that reshapes a musician's audio-effects mix in real time, driven by their EEG/ECG-derived attention and stress plus the felt emotion of their dry sound. It models a human-in-the-loop alternative to AI that composes alone: the machine adapts the sound, the performer keeps creative control.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DEAM-trained audio emotion regressor is the load-bearing risk: every rule uses its dry-signal V-A estimates, yet the paper concedes the train/test domain mismatch and reports no validation.","rationale":"The reader's CONDITIONAL verdict rests largely on the unvalidated PANNs/DEAM regressor, and I agree that this is the weakest load-bearing assumption. The architecture is otherwise internally consistent: the feature formulas for attention/relaxation and Baevsky SI are standard, the YAML rule mechanism is transparent, and the MIDI override provides a safeguard. Independent support is limited: the paper includes no machine-checked proofs, no latency measurements, and only a referenced repository, so reproducibility cannot be confirmed from the text; however, the proof-of-concept framing lowers the stakes. The decisive question is whether the V-A estimates on dry solo audio are trustworthy enough for the Table 1 rules to behave as intended. If the regressor's output is systematically biased on this domain, all four rulesets are compromised, not just the audio-only one. The paper itself flags the dataset mismatch in Section 2, and Section 3 explicitly defers quantitative evaluation to future work; both passages weigh in favor of a conditional verdict. The proposed test would resolve the concern: evaluating the model on held-out solo material with human V-A ratings. Should those results show near-random agreement, the conditional should be tightened to reject; but at this stage, with an existing demo and honest proof-of-concept framing, keeping the CONDITIONAL verdict is appropriate. I therefore leave the reader's verdict unchanged.","tokens_in":6818,"tokens_out":4997,"duration_ms":44707,"concrete_test":"Obtain the released witheFlow code and its PANNs/DEAM checkpoint (or retrain on DEAM with the described procedure), then (1) compute standard V-A regression metrics on the DEAM test split, and (2) run the model on a set of dry solo instrument recordings with human V-A ratings; compare predicted vs. human ratings via correlation and RMSE. If the solo-domain correlation is near zero or the DEAM test metrics are poor, the rule-based V-A comparisons in Table 1 cannot be relied on, and the 'emotion-driven enhancement' claim fails. Report at least Pearson r and RMSE for both arousal and valence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 ('Audio Emotion Regressor') states that the regressor is a PANNs CNN10 with a linear two-output head trained on DEAM, and concedes: 'datasets typically contain full music productions and not solo performances.' This concession is critical because the mixing logic in Table 1 is defined entirely in valence-arousal space: each rule selects which FX channels to boost by comparing the dry signal's V-A position with each FX channel's V-A position (e.g., 'arousal fx > arousaldry', 'closest effects channel in V A space'). A regressor that is systematically biased on dry solo audio would corrupt every ruleset, including the biosignal-driven ones, because the biosignal only selects the quadrant/strategy while the V-A comparison picks the target channel. The paper reports no DEAM test-set accuracy and no validation on solo material, so the central claim that the system modulates effects according to emotion is assumed, not shown. This is a correctness risk, not merely a missing evaluation nicety: if predicted V-A is noise on dry input, the 'emotion-driven' behavior of the system reduces to an arbitrary effect switcher. The paper's own Section 3 admits that a formal quantitative evaluation framework is future work, so the only support for enhancement is anecdotal musician reports.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces witheFlow, a proof-of-concept system for real-time audio effects modulation driven by a combination of biosignal features (EEG-derived attention/relaxation, ECG-derived Baevsky Stress Index) and audio-based valence-arousal estimates. It describes the system architecture, DAW routing, a customizable YAML rule-based mixing logic, artifact handling, and reports anecdotal positive feedback from collaborating musicians. The central contribution is the integration and open-source implementation rather than a formal empirical evaluation; the paper explicitly states that a quantitative evaluation framework is future work.","tokens_in":7017,"tokens_out":4681,"duration_ms":40559,"significance":"If the system's underlying emotion estimates are trustworthy, witheFlow is a relevant contribution to real-time co-creative performance systems: it is lightweight, local, interpretable, and provides performer control via MIDI override. The architecture is coherent, the formulas for attention/relaxation and Baevsky SI are standard, and the decision to use transparent rule-based mixing rather than an opaque end-to-end model is well motivated. The paper's strengths include its open-source commitment and its careful ethical discussion of biosignal privacy and psychological safety. The main risk is that the audio V-A regressor, which is load-bearing for all four rulesets, is not validated on the solo dry instrumental material that the system actually processes.","major_comments":[{"comment":"The audio V-A regressor is load-bearing for every rule in Table 1, since each rule compares the dry signal's V-A position with each effect channel's V-A position. The manuscript concedes that DEAM contains full music productions rather than solo performances, but reports no validation of the regressor on DEAM's test split or on solo/instrumental material. If the V-A estimates are systematically biased for dry solo input, all four rulesets are corrupted and the 'emotion-driven' behavior reduces to an arbitrary effect switcher. Please add quantitative validation (e.g., DEAM test-set metrics and a small solo-performance evaluation) or explicitly restrict the system claims to the design/proof-of-concept level.","section":"§2, Audio Emotion Regressor; Table 1"},{"comment":"The abstract and conclusion state that the system 'enhances' music performance, but the only supporting evidence is the statement in Section 3 that musicians 'have reported a generally positive experience.' No details are given on the number of musicians, instruments, tasks, instructions, or data collection. Either report a systematic qualitative evaluation (even a small-N study with a clear protocol) or reframe the claims to say that the system is intended to enhance and that only anecdotal feedback has been collected so far. The paper's own Section 3 says formal evaluation is future work, so the current wording overstates the evidence.","section":"§3, Discussion; §5, Conclusion"},{"comment":"Table 1 defines behavior in terms of 'High/Low' stress, attention, valence, and arousal, but the threshold values and calibration procedure are not specified. This makes the default behavior non-reproducible from the text alone. Please provide the default threshold values (or point to the exact configuration lines in the GitHub repository) and describe how thresholds are calibrated for individual performers.","section":"§2, Mixing Logic; Table 1"}],"minor_comments":[{"comment":"The system name is typeset inconsistently: 'Go witheFlow', 'witheFlow', and 'witheflow' (in the YouTube link). Please standardize.","section":"Title/Abstract"},{"comment":"Typo: 'The performer creates up toneffects chains' should read 'up to n effects chains' (or similar).","section":"§2, Audio Routing"},{"comment":"Use a space in '30kHz' and state the original PANNs input sampling rate for comparison.","section":"§2, Audio Emotion Regressor"},{"comment":"The institution is named 'National and Technical University of Athens' here but 'National Technical University of Athens' in the affiliation. Please make this consistent.","section":"§4, Ethical considerations"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the unvalidated DEAM-based audio regressor. The authors are transparent about the limitation, but because every mixing rule depends on the audio V-A estimate, a validation section (or at minimum a sanity-check experiment on solo material) is necessary before the system's behavior can be assessed. The paper is otherwise a reasonable system description, and the open-source implementation is a valuable asset."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is exactly what it says on the tin: a proof-of-concept for a real-time system that maps EEG attention/relaxation, ECG Baevsky SI, and audio valence-arousal into rule-based gain control over DAW effect chains. The integration itself is new relative to the cited literature, the feature formulas are standard and correctly stated, and the authors are upfront that they are not presenting a quantitative evaluation. That honesty deserves credit.\n\nWhat the paper does well: it takes off-the-shelf components and wires them together in a way that is actually usable by a performer, with a MIDI override and an artifact-detection scheme that degrades gracefully when sensors fail. The YAML-customizable mixing logic and the piecewise-function formalization are sensible design choices that leave room for a learnable controller later. The discussion of ethics, privacy, and psychological safety is more thoughtful than most papers in this space.\n\nThe soft spots are mostly ones the authors already concede, but they matter. The DEAM-trained audio regressor is the riskiest part: every rule in Table 1 compares the dry signal's V-A position against each effect channel's V-A position, so if the regressor is biased on dry solo audio, the whole 'emotion-driven' behavior collapses into an arbitrary effect switcher. The paper admits the train/test domain mismatch but provides no DEAM test-set numbers and no validation on solo material. That is a genuine correctness risk, not a missing extra. Similarly, there are no latency or resource measurements behind the 'lightweight, real-time' claim, the rule thresholds are undisclosed, and the GitHub repository is referenced without a commit hash, so the artifact is not yet reproducible. The 'musicians have reported a generally positive experience' line is anecdotal, and the authors themselves call a formal evaluation framework future work.\n\nNone of this makes the paper worthless. It is a well-scoped systems description that identifies a real gap and a plausible path to filling it. The proper referee assignment is a venue that cares about creative AI and live performance, and the referee should ask for the audio-regressor validation and latency numbers, or for the authors to explicitly scope the claim as 'works when the V-A estimates are reliable.'\n\nFor me: I would not cite it yet, but I would bring it to a reading group working on human-AI co-creativity and evaluation methodology. It deserves peer review, and it will probably get better after revision.","headline":"Honest, clearly-described proof-of-concept for bio-signal-driven audio FX; the load-bearing risk is the unvalidated audio emotion regressor, but the paper itself flags it.","tokens_in":7651,"tokens_out":1595,"would_cite":false,"duration_ms":15982,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"witheFlow maps a performer's EEG attention, ECG stress, and audio emotion into real-time gain changes on DAW effect chains, keeping the musician in creative control.","keywords":["witheFlow","real-time audio effects modulation","biosignal-driven music performance","valence-arousal","Baevsky Stress Index","EEG attention relaxation","rule-based mixing logic","human-AI co-creativity"],"falsifier":"Collect solo recordings with human valence-arousal ratings, run the system's regressor on them, and check correlation with the ratings; if predictions are uncorrelated or systematically shifted, the V-A-distance rules in all four rulesets are being driven by unreliable features. A simpler laboratory test is to feed synthetic V-A values into the mixing logic and verify that gain changes match the specified quadrant rules exactly.","tokens_in":6545,"feed_emoji":"🎛️","tokens_out":4946,"duration_ms":41703,"temperature":0.7,"pith_summary":"witheFlow is a proof-of-concept system that lets a live musician's physiology and the emotional character of their dry audio drive the mix of audio effects in real time. It combines EEG-derived attention and relaxation, an ECG-derived Baevsky Stress Index, and valence-arousal estimates from a neural audio model, then applies YAML-defined gain rules to boost or suppress parallel effect channels in a DAW. The paper argues that this kind of AI-human collaboration — where machines handle real-time technical processing and the performer keeps creative agency — is a useful alternative to AI as an autonomous composer. A sympathetic reader would care because it demonstrates a concrete, local, open-source path toward emotionally responsive instruments that performers can customize and override.","feed_headline":"witheFlow maps biosignals into live audio-effect gain","feed_subtitle":"EEG attention, ECG stress, and audio emotion drive DAW effect gains in real time, with a foot-pedal override for the performer.","key_machinery":"The load-bearing components are the Baevsky Stress Index (a heart-rate-variability metric computed from the mode and range of RR intervals), the attention and relaxation ratios derived from EEG alpha and beta power over four-second windows, and the valence-arousal regressor built by replacing the final layer of a PANNs CNN10 with a two-output linear head trained on DEAM. The mixing logic is a transparent piecewise map: the input domain (stress, attention, valence, arousal) is partitioned into regions, each paired with a Python gain function; user-editable YAML files define the region conditions and functions. This formalization is what makes the system interpretable and prepares it for a fut","core_discovery":"The central claim is that a laptop-runnable system can close a feedback loop between a performer's measurable internal state and their sonic output without altering musical content. Signals from commercial EEG and ECG sensors are converted into attention/relaxation and stress values; a PANNs CNN10 retrained on DEAM regresses the dry audio's valence and arousal; a rule-based mixing logic, formalized as a piecewise function over a partitioned input domain, translates these values into gain changes on multiple effects channels. The gain logic is organized by stress/attention quadrants: high stress pushes the mix toward V-A positions far from the dry signal, while low stress keeps it close, with","pith_inferences":["If the DEAM-trained regressor is systematically biased on solo instrument audio, every biosignal ruleset inherits that bias, since all rules compare the dry signal's V-A position with each effect channel's V-A position; a validation study on solo recordings would settle this.","The architecture is feature-agnostic, so the same rule-based loop could be closed with video, motion, or other physiological signals without redesigning the mixing logic.","The closed loop between internal state and sound raises a testable safety question: whether the system's own audio changes could feed back into the performer's physiology over long sessions; the foot-pedal override mitigates but does not measure this.","A quantitative evaluation framework for emotion-driven mixing systems, which the paper identifies as missing, could be built from the V-A-distance principle underlying the rules."],"forward_implications":["Live performers can shape their effect mix hands-free, with gain decisions driven by measurable internal state rather than manual knob-twiddling.","Because the rules are editable YAML/Python files, musicians can inspect exactly why a gain changed and customize or reverse the behavior, supporting trust and agency.","Local, laptop-based processing keeps biosignal data private and avoids cloud latency, making the approach practical in real performance settings.","The piecewise-rule formalization points directly to a learnable mixing logic, where decision trees could refine user-defined rules while staying interpretable.","The system's reliance on solo performance audio makes annotated solo-performance datasets with real-time labels a critical next resource."],"fun_headline_variants":["witheFlow turns EEG and ECG signals into live effect gain","Real-time effects shaped by brain waves and heart rate","Biosignal-driven audio effects: attention and stress go sonic","Your brain and heart now set your effect levels live","Live audio modulation from EEG attention and ECG stress"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the audio emotion regressor — PANNs CNN10 trained on DEAM — gives trustworthy valence-arousal estimates for dry solo performance audio in real time; the paper itself notes that emotion datasets typically contain full productions rather than solo performances.","fun_headline_variants_meta":{"raw":{"variants":["witheFlow turns EEG and ECG signals into live effect gain","Real-time effects shaped by brain waves and heart rate","Biosignal-driven audio effects: attention and stress go sonic","Your brain and heart now set your effect levels live","Live audio modulation from EEG attention and ECG stress"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1224,"prompt_tokens":657,"completion_tokens":567,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":401,"completion_tokens_details":{"reasoning_tokens":488}},"tokens_in":401,"tokens_out":567,"duration_ms":28215,"temperature":1.0,"reasoning_tokens":488,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T12:42:50.477651+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect solo recordings with human valence-arousal ratings, run the system's regressor on them, and check correlation with the ratings; if predictions are uncorrelated or systematically shifted, the V-A-distance rules in all four rulesets are being driven by unreliable features. A simpler laboratory test is to feed synthetic V-A values into the mixing logic and verify that gain changes match the specified quadrant rules exactly.","supporting_citations":[],"review_version":2}