{"id":"9aacfd28-ee31-4290-a6d1-83bf02d728c1","arxiv_id":"2412.06208","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A pilot-guided semantic communication system transmits audio and video features over noisy wireless channels and reports higher audio-visual event localization accuracy than unimodal and traditional digital baselines.","lead":"This paper builds a wireless system that sends only task-relevant audio and video features, with pilot signals to estimate changing channels, and tests it on audio-visual event localization. It reports better accuracy than unimodal or traditional digital transmission, but the comparison does not include prior multimodal semantic communication systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Eq.6 pilot-selection rule minimizes the LS residual, not channel-estimation error; in the scalar case the residual is identically zero, so the claimed 'best' channel estimate is not established.","rationale":"The paper's central claim is that pilot-guided channel estimation plus Euler-based semantic encoding improves audio-visual event localization over unknown fading channels. That claim rests on the pilot module in Eqs.5-7. The reader's weakest assumption flagged the pilot selection and channel-constancy issue; my concern is narrower and more damaging: even under block-fading and perfect pilot knowledge, the selection rule in Eq.6 does not minimize channel-estimation error. The LS residual and the estimation error depend on orthogonal components of the noise, so the chosen H_hat is not guaranteed to be the best estimate, and in the scalar case the rule is degenerate. This is a mathematical property of least squares, not a disagreement with the community's conventions, and it can be settled by a concrete simulation that compares the selected estimate against averaging and random selection. If the test shows averaging is better, the proposed pilot module is not doing what the paper claims, and the ablation in Table I would need to be reinterpreted. I still do not recommend rejection: the end-to-end system may work for other reasons (e.g., the semantic decoder or the joint training), and the reported experiments could be reproducible. But the mechanism claimed for the pilot-guided module is currently unsupported, so the paper should remain conditional pending this check or an equivalent analytical derivation. This is why I recommend UNCHANGED relative to the reader's CONDITIONAL verdict, while noting that the specific weak point is the Eq.6 selection criterion rather than only the channel-time-variation assumption.","tokens_in":13638,"tokens_out":8873,"duration_ms":99489,"concrete_test":"Run a synthetic Monte Carlo test on a block-fading Rayleigh channel with known H, unit-power pilot symbols, and Gaussian noise at SNR values of 0, 10, and 20 dB. For each block, compute H_hat_t from Eq.5 at every pilot time step, evaluate the Eq.6 residual, and select tmin. Compare the normalized MSE ||H_hat_{tmin} - H||^2_F / ||H||^2_F with (a) the sample average of all per-step LS estimates and (b) a randomly chosen per-step estimate, over at least 10^4 blocks. If the selected estimate is not more accurate than the average, or if all residuals are zero because x_t and y_t are scalars, then Eqs.5-6 do not implement the claimed 'best estimate' and the pilot-guided module's contribution to the reported results is unexplained.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-B2 estimates a channel per time step using least squares (Eq.5) and then selects tmin by minimizing the residual ||H_hat_t x_t - y_t||^2 (Eq.6). For a linear model y_t = H x_t + n_t, this residual is the squared norm of the component of n_t orthogonal to the pilot subspace; it is not the estimation error ||H_hat_t - H||^2. The noise component parallel to x_t directly corrupts H_hat_t but is absent from the residual, so minimizing the residual does not select the most accurate estimate. In the scalar case (x_t and y_t scalars), Eq.5 makes the residual identically zero for every t and Eq.6 degenerates, leaving tmin arbitrary. Even in the overdetermined MIMO case, the selection is biased toward time steps with small orthogonal noise rather than toward accurate channel estimates. Since H_best is then used in the zero-forcing detector (Eq.7), the pilot-guided module's central claim of accurate channel estimation is not supported by the stated rule. The paper reports only end-to-end accuracy, not channel-estimation MSE, and it does not ablate Eq.6 against averaging or a proper MMSE estimator, so the observed gains cannot be attributed to the proposed selection criterion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pilot-guided multimodal semantic communication framework for audio-visual event localization. The system uses a pilot signal to estimate the physical channel, applies an Euler-formula-based complex encoding to audio and visual semantic features, and decodes them at a multimodal receiver with a positive-sample-propagation AVE decoder. Experiments on the AVE subset of AudioSet report classification accuracy versus SNR for AWGN, Rayleigh, and Rician channels, comparing the full multimodal system against single-modality variants, a traditional JPEG/PCM+LDPC digital baseline, and, qualitatively, against AVEL and PSP. An ablation in Table I is intended to show the benefit of the pilot-guided channel estimation module.","tokens_in":13889,"tokens_out":7226,"duration_ms":71135,"significance":"If the reported results are reproducible, the framework would be a useful step toward semantic communication systems that operate without perfect channel state information, which is a real bottleneck in the field. The manuscript deserves credit for testing three channel models, evaluating a range of SNR values, and including a traditional digital baseline in addition to unimodal variants. However, the central claim of outperforming existing benchmark methods is not backed by quantitative comparisons to prior multimodal semantic communication systems. In addition, the pilot-selection rule in Section III-B2 is not a valid criterion for choosing the best channel estimate, and the Euler transformation contribution is never isolated in an ablation. The paper currently supports only the weaker claim that the proposed system improves over the particular unimodal and digital baselines tested.","major_comments":[{"comment":"The abstract and Section V claim that the framework \"outperforms existing benchmark methods\", but the only quantitative comparisons in Fig. 6 are against single-modality variants and a traditional JPEG/PCM+LDPC scheme. Prior multimodal semantic communication systems discussed in the related work, such as Refs. [12]-[14] and [18], are not compared numerically. The claim \"outperforms existing benchmark methods\" is therefore unsupported as stated. Please add quantitative comparisons to relevant task-oriented semantic communication baselines, or restrict the claim to the baselines actually tested.","section":"IV-D1 (Fig. 6)"},{"comment":"The pilot-selection rule in Eq. (6) minimizes the LS residual ||H_hat_t x_t - y_t||^2, not the channel-estimation error ||H_hat_t - H||^2. In the scalar case the residual is identically zero for every t and tmin is arbitrary; in the vector case the residual only measures the noise component orthogonal to the pilot subspace, so the selected H_best is not necessarily the most accurate estimate. Since H_best is used in the zero-forcing detector (Eq. 7), the load-bearing claim of reliable channel estimation is not established. In addition, Eq. (5) is written for a scalar channel while Eq. (7) treats H as a matrix, so the MIMO estimation rule is not dimensionally defined. Please replace or justify the selection rule and add an ablation that reports channel-estimation MSE against alternatives such as averaging over pilot steps or an MMSE estimator.","section":"III-B2, Eqs. (5)-(7)"},{"comment":"The Euler time-frequency transformation is one of the three claimed contributions, but it is never isolated in an experiment. Because the transformation in Eqs. (1) and (8) is invertible, it cannot add information by construction; whether it improves localization is an empirical question. Fig. 7 is only a qualitative comparison to AVEL and PSP, with no quantitative accuracy numbers and no ablation that removes the Euler transformation while keeping all other components fixed. Please add such an ablation and report accuracy on the zero-loss channel as well as across SNR.","section":"I (Contribution 3) and IV-D2"},{"comment":"The ablation table used to support RQ3 contains inconsistent and incomplete entries. For example, the AWGN multimodal row reports a 60.17% drop at 0 dB, which conflicts with the text's statement that the Gaussian channel suffers a relatively smaller performance loss of less than 20%; several cells in the Rayleigh and Rician audio-only rows have no percentage. It is also not clear whether the numbers are the with-estimation accuracies and the percentages are relative drops. Please correct the table, define the reported statistics, and reconcile the summary statements with the table.","section":"IV-D3, Table I"}],"minor_comments":[{"comment":"The text contains typos such as \"real-wold\", \"Memember\", and \"mulimodal\" that should be corrected.","section":"Abstract and title page"},{"comment":"The paper says the framework \"outperforms existing methods in Signal-to-Noise Ratio (SNR)\", but SNR is a channel parameter, not a performance metric; accuracy is the metric actually reported.","section":"Abstract and Section V"},{"comment":"Eq. (11) calls the operation \"LayerNorm(·)\" but the surrounding text says it is row-wise ℓ1 normalization; use a distinct notation to avoid confusion with layer normalization.","section":"III-C2, Eq. (11)"},{"comment":"The implementation details do not report the train/test split of the AVE subset or the number of test videos, which makes the quantitative claims difficult to reproduce.","section":"IV-C"},{"comment":"The traditional baseline (JPEG, PCM, LDPC) is not described with parameters such as compression ratio, code rate, and modulation order; specify these parameters for a fair comparison.","section":"IV-D1"},{"comment":"The pilot signal is described only as a 10-second sinusoidal wave; the choice of waveform and duration is not justified or ablated.","section":"III-B2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal, but the authors need to align the central claims with the experiments. The missing comparisons to prior semantic communication baselines and the invalid pilot-selection criterion are load-bearing issues, not mere presentation problems. I also recommend that the editor ask the authors to verify Table I carefully, as several entries appear inconsistent with the text. I see no evidence of misconduct, but the claims should be scaled back until the supporting comparisons and ablations are in place."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a workmanlike integration of existing pieces—VGG-19 features, AGVA attention, PSP decoder, and an EulerNet-style transform—wrapped around a pilot-based LS channel estimation module. The system setup is practical (unknown CSI, Rayle/If/Rician channels, learned pilot representation), and the ablation in Table I does show that removing the pilot module sharply degrades accuracy, especially on Rayleigh and Rician. So the paper does establish that some form of channel estimation is valuable in this setting, and the architecture is described in enough detail to be reimplemented.\n\nThe soft spots are real but mostly fixable. The biggest one is comparison: the abstract says \"outperforms existing benchmark methods,\" but the only baselines are video-only, audio-only, and JPEG/PCM/LDPC. None of the prior multimodal semantic communication systems cited in the related work are included, so the headline claim is simply not backed by the experiments shown. Second, the Euler transform is never ablated, so we don't know whether it contributes anything beyond the existing AVEL machinery. Third, the pilot-selection rule in Eq.6 is mathematically off: the residual \\|\\hat H_t x_t - y_t\\|^2 is the energy of the noise component orthogonal to the pilot, not the estimation error, and in the scalar case it is identically zero. So the \"best estimate\" claim is unsupported. There is also no channel-estimation MSE reported, no comparison to a simple average over time steps or to an MMSE estimator, and no error bars or code release.\n\nThe paper is not incoherent; the architecture makes sense and the problem is worth solving. But the evidence presented does not support the strong claims made in the abstract and conclusion. With a proper comparison to prior semantic communication baselines, an ablation of the Euler transform, and a fix or honest test of the pilot-selection rule, it could become a solid contribution. On the current version, I would not trust the headline numbers without those pieces.\n\nI'd send it to peer review, because the problem is real and the integration is plausible, but I'd expect heavy revision. A serious referee could push the authors to do the missing comparisons and either fix or properly characterize the pilot-selection mechanism.","headline":"A plausible integration of known components for AVEL over unknown fading channels, but the headline superiority claim is under-supported and the pilot-selection rule in Eq.6 does not do what the paper says.","tokens_in":14419,"tokens_out":3547,"would_cite":false,"duration_ms":38186,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a pilot-guided multimodal semantic communication framework with Euler-based encoding achieves near-optimal audio-visual event localization accuracy under high SNR and remains robust under low SNR and multipath fading.","keywords":["semantic communication","multimodal communication","audio-visual event localization","pilot-guided channel estimation","Euler transform","wireless fading channels","zero-forcing detection","multimodal semantic encoding"],"falsifier":"Transmit the same pilot and data over a channel whose matrix changes within the pilot-and-data block, for example a moving terminal realization, and compare localization accuracy with the static-channel result reported in the paper. If accuracy falls to the level of the no-estimation ablation, the constant-channel assumption is the source of the gain. Alternatively, replace the sinusoid with an equal-length different pilot waveform and check whether the estimated channel and accuracy change; a large change would show the learned encoder's fidelity to that particular pilot is doing the work.","tokens_in":13442,"feed_emoji":"📡","tokens_out":8270,"duration_ms":73238,"temperature":0.7,"pith_summary":"This paper sets out to show that semantic communication can handle a real multimodal task, audio-visual event localization, without assuming the receiver knows the wireless channel in advance. The proposed system sends a digital pilot signal, estimates the unknown channel from the received pilot, and uses that estimate to undo channel distortion before decoding audio and visual features. The authors report near-optimal localization accuracy at high SNR and robust performance under low SNR and multipath fading across AWGN, Rayleigh, and Rician channels, outperforming single-modality and traditional source-and-channel-coded baselines. If correct, the framework points to a practical route for carrying downstream intelligence tasks over dynamic physical channels.","feed_headline":"Pilots keep audio-visual event localization accurate in noise","feed_subtitle":"A pilot-guided estimator with Euler-based encoding beats single-modality and traditional baselines on fading channels.","key_machinery":"The load-bearing machinery is the pilot-guided channel estimator. At each time step the receiver computes $\\hat{H}_t = (x_t^H x_t)^{-1} x_t^H y_t$ from the transmitted and received pilot, selects the time index with the smallest squared error, and then uses $\\hat{H}_{\\mathrm{best}}$ in the zero-forcing detector $\\hat{X} = (\\hat{H}^H \\hat{H})^{-1} \\hat{H}^H Y$, turning multiplicative channel distortion into additive noise. The other carrying component is the Euler transformation, which maps each real audio or visual feature vector into a complex polar form $\\lambda e^{i\\theta}$ and back to $[\\lambda\\cos\\theta; \\lambda\\sin\\theta]$, preserving the dot-product structure while exposing time-frequency characteristics. On the receiver side, AGVA attention and positive sample propagation fuse the two modalities into segment-level event predictions.","core_discovery":"The central claim is that a pilot-guided channel-estimation module, combined with Euler-based multimodal semantic encoding and decoding, is enough to make audio-visual event localization work over unknown fading channels. The receiver estimates the channel matrix from a 10-second sinusoidal pilot using least squares, picks the time step whose estimate minimizes the reconstruction error, and applies a zero-forcing detector before semantic decoding. The paper reports that this beats methods that rely on perfect channel state information assumptions, and its ablation shows that removing the estimation module causes a large accuracy drop, exceeding 50 percent on Rayleigh and Rician channels for multimodal transmission.","pith_inferences":["Beyond the paper: comparing the framework against an end-to-end learned semantic system that also performs pilot-based channel estimation would isolate how much of the gain comes from the Euler transform rather than from the channel-estimation block itself.","Beyond the paper: the least-squares pilot estimator assumes the channel is static over the pilot-and-data block; a natural extension is to test the system on time-selective channels and with shorter pilots to see where the constant-channel assumption breaks.","Beyond the paper: because the pilot is a 10-second sinusoid that passes through the learned audio encoder, the framework's robustness may depend on that encoder's ability to reproduce the sinusoid faithfully; measuring estimation error as a function of pilot waveform would test that dependence.","Beyond the paper: the same pilot-guided transport could be applied to other temporal multimodal tasks such as sound source localization, but the paper does not report results for those tasks."],"forward_implications":["If the framework works as described, receivers for semantic communication no longer need to be handed the channel matrix; a short pilot can supply the estimate needed for reliable decoding.","The reported accuracy at high SNR over AWGN, Rayleigh, and Rician channels implies the method can approach the noiseless performance ceiling even when channel state information is unknown.","The ablation results imply that pilot-based channel estimation is not an optional add-on: removing it cuts multimodal accuracy by more than half on fading channels, so robustness in realistic deployments depends on it.","The design separates channel handling from task decoding, so the pilot-guided transport could be reused with a different task head for other segment-level multimodal problems."],"supporting_citations":[{"why":"Defines the audio-visual event localization task and supplies the AudioSet AVE subset and the segment-level evaluation protocol.","marker":"[19]"},{"why":"Positive sample propagation decoder used as the semantic decoding head and as a baseline for multimodal fusion.","marker":"[9]"},{"why":"AVEL method whose AGVA visual encoder is used to attend to audio-relevant visual regions and as a comparison baseline.","marker":"[8]"},{"why":"VGG-19 pretrained model used to extract visual features from video segments.","marker":"[28]"},{"why":"AudioSet dataset used to pretrain the VGG-like audio feature extractor.","marker":"[32]"},{"why":"Supplies the Euler-formula transformation used to map real features to complex polar form.","marker":"[29]"},{"why":"A prior task-oriented multimodal semantic communication method used as a comparison point for single-modality transmission.","marker":"[12]"},{"why":"A prior multi-user semantic communication method used as a comparison for text and image transmission baselines.","marker":"[13]"}],"fun_headline_variants":["Pilot guidance beats perfect CSI for audio-visual localization","Euler-based multimodal codec localizes events on fading channels","Pilot-aided semantic comm handles audio-visual event localization","Unknown channels? Pilots keep audio-visual event localization robust","Multimodal semantic comm with pilots outperforms baselines on SNR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the physical channel stays essentially constant over the pilot and the data block, and that the learned audio encoder reproduces the 10-second sinusoidal pilot faithfully; if the channel shifts within a block or the pilot is distorted, the zero-forcing detector amplifies noise and the reported gains may not transfer to real dynamic channels.","fun_headline_variants_meta":{"raw":{"variants":["Pilot guidance beats perfect CSI for audio-visual localization","Euler-based multimodal codec localizes events on fading channels","Pilot-aided semantic comm handles audio-visual event localization","Unknown channels? Pilots keep audio-visual event localization robust","Multimodal semantic comm with pilots outperforms baselines on SNR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00041,"raw_usage":{"total_tokens":2104,"prompt_tokens":905,"completion_tokens":1199,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1006}},"tokens_in":521,"tokens_out":1199,"duration_ms":9425,"temperature":1.0,"reasoning_tokens":1006,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:54:14.834001+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Transmit the same pilot and data over a channel whose matrix changes within the pilot-and-data block, for example a moving terminal realization, and compare localization accuracy with the static-channel result reported in the paper. If accuracy falls to the level of the no-estimation ablation, the constant-channel assumption is the source of the gain. Alternatively, replace the sinusoid with an equal-length different pilot waveform and check whether the estimated channel and accuracy change; a large change would show the learned encoder's fidelity to that particular pilot is doing the work.","supporting_citations":[{"cited_title":"Audio-visual event localization in unconstrained videos,","cited_arxiv_id":null,"evidence_quote":"Defines the audio-visual event localization task and supplies the AudioSet AVE subset and the segment-level evaluation protocol."},{"cited_title":"Eulernet: Adaptive feature interaction learning via euler’s formula for ctr prediction,","cited_arxiv_id":null,"evidence_quote":"Supplies the Euler-formula transformation used to map real features to complex polar form."},{"cited_title":"Task-Oriented Multi-User Semantic Communications for VQA Task","cited_arxiv_id":"2108.07357","evidence_quote":"A prior task-oriented multimodal semantic communication method used as a comparison point for single-modality transmission."}],"review_version":1}