Pith. sign in

REVIEW 4 major objections 6 minor 30 references

Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a pilot-guided multimodal semantic communication framework with Euler-based encoding achieves near-optimal audio-visual event localization accuracy under high SNR and remains robust under low SNR and multipath fading.

desk verdict A plausible integration of known components for AVEL over unknown fading channels, but the headline superiority claim is under-supported and the pilot-selection rule in Eq.6 does not do what the paper says. read the letter →

arxiv 2412.06208 v1 pith:A5MVWRBO submitted 2024-12-09 cs.SD cs.CVcs.MMeess.AS

Fei Yu , Zhe Xiang , Nan Che , Zhuoran Zhang , Yuandi Li , Junxiao Xue , Zhiguo Wan This is my paper · ORCID
classification cs.SDcs.CVcs.MMeess.AS
keywords semanticcommunicationmultimodalaudio-visualeventlocalizationpilot-guidedchannelestimationEulertransformwirelessfadingchannelszero-forcingdetectionencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that semantic communication can handle a real multimodal task, audio-visual event localization, without assuming the receiver knows the wireless channel in advance. The proposed system sends a digital pilot signal, estimates the unknown channel from the received pilot, and uses that estimate to undo channel distortion before decoding audio and visual features. The authors report near-optimal localization accuracy at high SNR and robust performance under low SNR and multipath fading across AWGN, Rayleigh, and Rician channels, outperforming single-modality and traditional source-and-channel-coded baselines. If correct, the framework points to a practical route for carrying downstream intelligence tasks over dynamic physical channels.

What carries the argument

The load-bearing machinery is the pilot-guided channel estimator. At each time step the receiver computes $\hat{H}_t = (x_t^H x_t)^{-1} x_t^H y_t$ from the transmitted and received pilot, selects the time index with the smallest squared error, and then uses $\hat{H}_{\mathrm{best}}$ in the zero-forcing detector $\hat{X} = (\hat{H}^H \hat{H})^{-1} \hat{H}^H Y$, turning multiplicative channel distortion into additive noise. The other carrying component is the Euler transformation, which maps each real audio or visual feature vector into a complex polar form $\lambda e^{i\theta}$ and back to $[\lambda\cos\theta; \lambda\sin\theta]$, preserving the dot-product structure while exposing time-frequency characteristics. On the receiver side, AGVA attention and positive sample propagation fuse the two modalities into segment-level event predictions.

What would settle it

Transmit the same pilot and data over a channel whose matrix changes within the pilot-and-data block, for example a moving terminal realization, and compare localization accuracy with the static-channel result reported in the paper. If accuracy falls to the level of the no-estimation ablation, the constant-channel assumption is the source of the gain. Alternatively, replace the sinusoid with an equal-length different pilot waveform and check whether the estimated channel and accuracy change; a large change would show the learned encoder's fidelity to that particular pilot is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that a pilot-guided channel-estimation module, combined with Euler-based multimodal semantic encoding and decoding, is enough to make audio-visual event localization work over unknown fading channels. The receiver estimates the channel matrix from a 10-second sinusoidal pilot using least squares, picks the time step whose estimate minimizes the reconstruction error, and applies a zero-forcing detector before semantic decoding. The paper reports that this beats methods that rely on perfect channel state information assumptions, and its ablation shows that removing the estimation module causes a large accuracy drop, exceeding 50 percent on Rayleigh and Rician channels for multimodal transmission.

Load-bearing premise

The load-bearing premise is that the physical channel stays essentially constant over the pilot and the data block, and that the learned audio encoder reproduces the 10-second sinusoidal pilot faithfully; if the channel shifts within a block or the pilot is distorted, the zero-forcing detector amplifies noise and the reported gains may not transfer to real dynamic channels.

Editorial extensions

If this is right

  • If the framework works as described, receivers for semantic communication no longer need to be handed the channel matrix; a short pilot can supply the estimate needed for reliable decoding.
  • The reported accuracy at high SNR over AWGN, Rayleigh, and Rician channels implies the method can approach the noiseless performance ceiling even when channel state information is unknown.
  • The ablation results imply that pilot-based channel estimation is not an optional add-on: removing it cuts multimodal accuracy by more than half on fading channels, so robustness in realistic deployments depends on it.
  • The design separates channel handling from task decoding, so the pilot-guided transport could be reused with a different task head for other segment-level multimodal problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: comparing the framework against an end-to-end learned semantic system that also performs pilot-based channel estimation would isolate how much of the gain comes from the Euler transform rather than from the channel-estimation block itself.
  • Beyond the paper: the least-squares pilot estimator assumes the channel is static over the pilot-and-data block; a natural extension is to test the system on time-selective channels and with shorter pilots to see where the constant-channel assumption breaks.
  • Beyond the paper: because the pilot is a 10-second sinusoid that passes through the learned audio encoder, the framework's robustness may depend on that encoder's ability to reproduce the sinusoid faithfully; measuring estimation error as a function of pilot waveform would test that dependence.
  • Beyond the paper: the same pilot-guided transport could be applied to other temporal multimodal tasks such as sound source localization, but the paper does not report results for those tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a pilot-guided multimodal semantic communication framework for audio-visual event localization. The system uses a pilot signal to estimate the physical channel, applies an Euler-formula-based complex encoding to audio and visual semantic features, and decodes them at a multimodal receiver with a positive-sample-propagation AVE decoder. Experiments on the AVE subset of AudioSet report classification accuracy versus SNR for AWGN, Rayleigh, and Rician channels, comparing the full multimodal system against single-modality variants, a traditional JPEG/PCM+LDPC digital baseline, and, qualitatively, against AVEL and PSP. An ablation in Table I is intended to show the benefit of the pilot-guided channel estimation module.

Significance. If the reported results are reproducible, the framework would be a useful step toward semantic communication systems that operate without perfect channel state information, which is a real bottleneck in the field. The manuscript deserves credit for testing three channel models, evaluating a range of SNR values, and including a traditional digital baseline in addition to unimodal variants. However, the central claim of outperforming existing benchmark methods is not backed by quantitative comparisons to prior multimodal semantic communication systems. In addition, the pilot-selection rule in Section III-B2 is not a valid criterion for choosing the best channel estimate, and the Euler transformation contribution is never isolated in an ablation. The paper currently supports only the weaker claim that the proposed system improves over the particular unimodal and digital baselines tested.

major comments (4)
  1. [IV-D1 (Fig. 6)] The abstract and Section V claim that the framework "outperforms existing benchmark methods", but the only quantitative comparisons in Fig. 6 are against single-modality variants and a traditional JPEG/PCM+LDPC scheme. Prior multimodal semantic communication systems discussed in the related work, such as Refs. [12]-[14] and [18], are not compared numerically. The claim "outperforms existing benchmark methods" is therefore unsupported as stated. Please add quantitative comparisons to relevant task-oriented semantic communication baselines, or restrict the claim to the baselines actually tested.
  2. [III-B2, Eqs. (5)-(7)] The pilot-selection rule in Eq. (6) minimizes the LS residual ||H_hat_t x_t - y_t||^2, not the channel-estimation error ||H_hat_t - H||^2. In the scalar case the residual is identically zero for every t and tmin is arbitrary; in the vector case the residual only measures the noise component orthogonal to the pilot subspace, so the selected H_best is not necessarily the most accurate estimate. Since H_best is used in the zero-forcing detector (Eq. 7), the load-bearing claim of reliable channel estimation is not established. In addition, Eq. (5) is written for a scalar channel while Eq. (7) treats H as a matrix, so the MIMO estimation rule is not dimensionally defined. Please replace or justify the selection rule and add an ablation that reports channel-estimation MSE against alternatives such as averaging over pilot steps or an MMSE estimator.
  3. [I (Contribution 3) and IV-D2] The Euler time-frequency transformation is one of the three claimed contributions, but it is never isolated in an experiment. Because the transformation in Eqs. (1) and (8) is invertible, it cannot add information by construction; whether it improves localization is an empirical question. Fig. 7 is only a qualitative comparison to AVEL and PSP, with no quantitative accuracy numbers and no ablation that removes the Euler transformation while keeping all other components fixed. Please add such an ablation and report accuracy on the zero-loss channel as well as across SNR.
  4. [IV-D3, Table I] The ablation table used to support RQ3 contains inconsistent and incomplete entries. For example, the AWGN multimodal row reports a 60.17% drop at 0 dB, which conflicts with the text's statement that the Gaussian channel suffers a relatively smaller performance loss of less than 20%; several cells in the Rayleigh and Rician audio-only rows have no percentage. It is also not clear whether the numbers are the with-estimation accuracies and the percentages are relative drops. Please correct the table, define the reported statistics, and reconcile the summary statements with the table.
minor comments (6)
  1. [Abstract and title page] The text contains typos such as "real-wold", "Memember", and "mulimodal" that should be corrected.
  2. [Abstract and Section V] The paper says the framework "outperforms existing methods in Signal-to-Noise Ratio (SNR)", but SNR is a channel parameter, not a performance metric; accuracy is the metric actually reported.
  3. [III-C2, Eq. (11)] Eq. (11) calls the operation "LayerNorm(·)" but the surrounding text says it is row-wise ℓ1 normalization; use a distinct notation to avoid confusion with layer normalization.
  4. [IV-C] The implementation details do not report the train/test split of the AVE subset or the number of test videos, which makes the quantitative claims difficult to reproduce.
  5. [IV-D1] The traditional baseline (JPEG, PCM, LDPC) is not described with parameters such as compression ratio, code rate, and modulation order; specify these parameters for a fair comparison.
  6. [III-B2] The pilot signal is described only as a 10-second sinusoidal wave; the choice of waveform and duration is not justified or ablated.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the central AVEL accuracy gains are an empirical, out-of-sample comparison; one local self-referential step in the pilot-selection rule (Eq. 6) does not drive the main result.

  1. self definitional [Section III-B2, Pilot-guided Channel Estimation, Eqs. (5) and (6)]
    "To determine the best estimate, we compute the squared error for each time step and identify tmin, the time step with the smallest squared error: tmin = arg min t ∥ ˆHtxt − yt∥2. (6) The channel response estimate with the minimum squared error is then selected as: ˆHbest = ˆHtmin. This method ensures that the channel coefficient matrix is accurately determined by selecting the estimate that minimizes the squared error across all time steps."

    Each candidate Ĥ_t in Eq. (5) is, by construction, the minimizer of the squared LS residual ‖Ĥ x_t − y_t‖² for its own time step, and Eq. (6) selects the step whose minimized residual is smallest. That residual is an in-sample goodness-of-fit measure (the noise component orthogonal to the pilot, and identically zero in the scalar/SISO case that the paper itself uses for the audio-to-video link), not the estimation error ‖Ĥ_t − H‖². The sentence claiming the method 'ensures that the channel coefficient matrix is accurately determined' therefore converts the estimate's own defining objective into a claim of accuracy: the selection criterion is the same quantity that defines the candidates, leaving the scalar case vacuous.

full rationale

The paper's central claim — that pilot guidance plus Euler-based semantic coding improves AVEL accuracy over unknown AWGN/Rayleigh/Rician channels — rests on an empirical comparison on held-out AudioSet-AVE test segments against AVEL [8], PSP [9], single-modality, and JPEG/PCM+LDPC baselines, plus a with/without channel-estimation ablation (Table I, Fig. 6). This is externally falsifiable and independent of the method's own fitted parameters, so it falls in the 0–2 band under the hard rules. I found no self-citations: none of the 32 references lists any of the present authors, and the load-bearing components (AGVA/AVEL [8], Positive Sample Propagation [9], Euler's-formula feature interaction [29], and the MUSC code base used to build the implementation) are all cited as prior work by other groups. The Euler transformation (Eqs. 1, 4, 8, 9) is an exact, invertible coordinate change — F⁻¹∘F equals the identity in noise-free transmission — so it cannot create information by construction; whether it helps is a soundness/ablation question, not circularity. One local step is genuinely self-referential: Eq. (6) selects the channel estimate by the same squared-error objective that defines each candidate in Eq. (5), so the module's stated guarantee of an 'accurately determined' channel matrix is not established by the criterion itself (in the SISO case the residual is identically zero and tmin is arbitrary). This weakens the pilot module's internal justification and the attribution of gains to the specific selection rule — a correctness/rigor concern — but it does not make the paper's central derivation circular, since end-to-end accuracy is measured against ground-truth labels on unseen data. Secondary issues (no Euler ablation, the abstract's 'outperforms in terms of SNR' phrasing, and duplicated references [8]/[19] and [9]/[25]) are clarity and rigor concerns, not circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The ledger lists the hyperparameters, modeling assumptions, and background components on which the central empirical claim rests. None of these is derived within the paper; they are either tuned on the benchmark or imported from prior AVEL and communication literature.

free parameters (3)
  • Beta loss balance weight (reported as lambda=100) = 100
    Eq.18 combines classification and audio-visual similarity losses; Section IV-C states lambda=100 was chosen based on extensive experiments and analysis, not derived from first principles.
  • Tau1 audio-visual connection threshold = 0.099
    Eq.12 and Eq.13 filter weak audio-visual connections; the value is tuned on the benchmark per Section IV-C.
  • Pilot duration = 10 seconds
    The pilot is a 10-second sinusoidal wave encoded by the audio source-channel encoder (Section III-B2); the duration is chosen by hand and affects how many LS estimates are available.
assumptions (5)
  • domain assumption Linear channel model Y=HX+N with block-constant H
    Used throughout Section III-C and in Eq.5 through Eq.7; if the channel evolves within a block, the LS estimate at t_min is stale for other symbols.
  • domain assumption Zero-forcing detector with estimated H sufficiently recovers X
    Eq.7 assumes the matrix (H_hat^H H_hat) is invertible and that estimation error does not dominate; under deep fading this assumption is fragile.
  • standard math Euler polar transform is information-preserving
    Eq.1 and Eq.8 define an exact invertible mapping via Euler's formula; used to justify that no information is lost in the complex decomposition.
  • domain assumption Pre-extracted VGG-19 and VGG-like audio features are sufficient and fixed
    Section IV-C uses pre-trained features from AVEL and PSP; the communication system cannot recover information lost in feature extraction, so the endpoint task ceiling is inherited.
  • domain assumption Positive Sample Propagation is an appropriate semantic decoder for AVEL
    Section III-C2 adopts PSP as the semantic decoder; the final accuracy is bounded by PSP's representational assumptions and thresholding behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization." pith.science (2026). https://pith.science/paper/A5MVWRBO

@misc{pith2026241206208,
  author       = {Pith},
  title        = {Pith review of: Pilot-guided Multimodal Semantic Communication for Audio-Visual Event Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A5MVWRBO}},
  note         = {Machine review of arXiv:2412.06208}
}
read the original abstract

Multimodal semantic communication, which integrates various data modalities such as text, images, and audio, significantly enhances communication efficiency and reliability. It has broad application prospects in fields such as artificial intelligence, autonomous driving, and smart homes. However, current research primarily relies on analog channels and assumes constant channel states (perfect CSI), which is inadequate for addressing dynamic physical channels and noise in real-world scenarios. Existing methods often focus on single modality tasks and fail to handle multimodal stream data, such as video and audio, and their corresponding tasks. Furthermore, current semantic encoding and decoding modules mainly transmit single modality features, neglecting the need for multimodal semantic enhancement and recognition tasks. To address these challenges, this paper proposes a pilot-guided framework for multimodal semantic communication specifically tailored for audio-visual event localization tasks. This framework utilizes digital pilot codes and channel modules to guide the state of analog channels in real-wold scenarios and designs Euler-based multimodal semantic encoding and decoding that consider time-frequency characteristics based on dynamic channel state. This approach effectively handles multimodal stream source data, especially for audio-visual event localization tasks. Extensive numerical experiments demonstrate the robustness of the proposed framework in channel changes and its support for various communication scenarios. The experimental results show that the framework outperforms existing benchmark methods in terms of Signal-to-Noise Ratio (SNR), highlighting its advantage in semantic communication quality.

Figures

Figures reproduced from arXiv: 2412.06208 by the authors.

Figure 1
Figure 1. The overall framework of the pilot-guided mulimodal semantic communication for the multimodal task, which contains five modules: (a) the audio [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The audio transmitter module in Fig. 1-(a). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. The complex module of pilot-guided physical channel and signal [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The multimodal receiver module in Fig. 1-(d). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The accuracy for various testing channels based on different trained models (for RQ1). [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The qualitative example of AVE localization. For the video, only the first three segments contain the visual and audio signals of the event male speech. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 20 canonical work pages

  1. [12]

    Task-Oriented Multi-User Semantic Communications for VQA Task

    H. Xie, Z. Qin, and G. Li, “Task-oriented semantic communications for multimodal data,” arXiv preprint arXiv:2108.07357 , 2021

    work page Pith review arXiv 2021

  2. [14]

    Content-aware semantic communica- tion for goal-oriented wireless communications,

    Y . Fu, W. Cheng, and W. Zhang, “Content-aware semantic communica- tion for goal-oriented wireless communications,” in IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2023, pp. 1–6

    work page 2023

  3. [18]

    Cross-modal semantic communi- cations,

    A. Li, X. Wei, D. Wu, and L. Zhou, “Cross-modal semantic communi- cations,” IEEE Wireless Communications , vol. 29, no. 6, pp. 144–151, 2022

    work page 2022

  4. [1]

    Towards a theory of semantic communication,

    J. Bao, P. Basu, M. Dean, C. Partridge, A. Swami, W. Leland, and J. A. Hendler, “Towards a theory of semantic communication,” in 2011 IEEE Network Science Workshop. IEEE, 2011, pp. 110–117

    2011

  5. [2]

    6g networks: Beyond shannon towards semantic and goal-oriented communications,

    E. C. Strinati and S. Barbarossa, “6g networks: Beyond shannon towards semantic and goal-oriented communications,” Computer Networks, vol. 190, p. 107930, 2021

    2021

  6. [3]

    Semantic communications: Overview, open issues, and future research directions,

    X. Luo, H.-H. Chen, and Q. Guo, “Semantic communications: Overview, open issues, and future research directions,” IEEE Wireless Communi- cations, vol. 29, no. 1, pp. 210–219, 2022

    2022

  7. [4]

    A unified multi-task semantic communication system for multimodal data,

    G. Zhang, Q. Hu, Z. Qin, Y . Cai, G. Yu, and X. Tao, “A unified multi-task semantic communication system for multimodal data,” IEEE Transactions on Communications , 2024

    work page 2024

  8. [5]

    Less data, more knowledge: Building next generation semantic communication networks,

    C. Chaccour, W. Saad, M. Debbah, Z. Han, and H. V . Poor, “Less data, more knowledge: Building next generation semantic communication networks,” IEEE Communications Surveys & Tutorials , 2024

    2024

Show all 30 references
  1. [6]

    Multimodal semantic communication accelerated bidirectional caching for 6g mec,

    C. Wang, X. Yu, L. Xu, Z. Wang, and W. Wang, “Multimodal semantic communication accelerated bidirectional caching for 6g mec,” Future Generation Computer Systems , vol. 140, pp. 225–237, 2023

    2023

  2. [7]

    Multimodal and multiuser semantic communications for channel-level information fusion,

    X. Luo, R. Gao, H.-H. Chen, S. Chen, Q. Guo, and P. N. Suganthan, “Multimodal and multiuser semantic communications for channel-level information fusion,” IEEE Wireless Communications, vol. 31, no. 2, pp. 117–125, 2022

    2022

  3. [10]

    Distributed semantic communications for multimodal audio-visual parsing tasks,

    P. Wang, J. Li, C. Liu, X. Fan, M. Ma, and Y . Wang, “Distributed semantic communications for multimodal audio-visual parsing tasks,” IEEE Transactions on Green Communications and Networking , 2024

    2024

  4. [11]

    Hybrid digital-analog semantic communications,

    H. Xie, Z. Qin, Z. Han, and K. B. Letaief, “Hybrid digital-analog semantic communications,” arXiv preprint arXiv:2405.12580 , 2024

    2024 arXiv

  5. [13]

    Task-oriented multi- user semantic communications,

    H. Xie, Z. Qin, X. Tao, and K. B. Letaief, “Task-oriented multi- user semantic communications,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 9, pp. 2584–2597, 2022

    2022

  6. [15]

    Adasem: Adaptive goal-oriented semantic communications for end-to-end camera relocalization,

    Q. Liao and T.-Y . Tung, “Adasem: Adaptive goal-oriented semantic communications for end-to-end camera relocalization,” arXiv preprint arXiv:2401.08360, 2024

    2024 arXiv

  7. [16]

    Wireless resource management in intelligent semantic communication networks,

    L. Xia, Y . Sun, X. Li, G. Feng, and M. A. Imran, “Wireless resource management in intelligent semantic communication networks,” in IEEE INFOCOM 2022 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2022, pp. 1–6

    2022

  8. [17]

    Semantic and effective communication for remote control tasks with dynamic feature compression,

    P. Talli, F. Pase, F. Chiariotti, A. Zanella, and M. Zorzi, “Semantic and effective communication for remote control tasks with dynamic feature compression,” in IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2023, pp. 1–6

    2023

  9. [19]

    Audio-visual event localization in unconstrained videos,

    Y . Tian, J. Shi, B. Li, Z. Duan, and C. Xu, “Audio-visual event localization in unconstrained videos,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 247–263

    2018

  10. [20]

    Dual-modality seq2seq net- work for audio-visual event localization,

    Y .-B. Lin, Y .-J. Li, and Y .-C. F. Wang, “Dual-modality seq2seq net- work for audio-visual event localization,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 2002–2006

    2019

  11. [21]

    Dual attention matching for audio- visual event localization,

    Y . Wu, L. Zhu, Y . Yan, and Y . Yang, “Dual attention matching for audio- visual event localization,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 6292–6300

    2019

  12. [22]

    Cross-modal relation- aware networks for audio-visual event localization,

    H. Xu, R. Zeng, Q. Wu, M. Tan, and C. Gan, “Cross-modal relation- aware networks for audio-visual event localization,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 3893– 3901

    2020

  13. [23]

    Audio- visual event localization via recursive fusion by joint co-attention,

    B. Duan, H. Tang, W. Wang, Z. Zong, G. Yang, and Y . Yan, “Audio- visual event localization via recursive fusion by joint co-attention,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 4013–4022

    2021

  14. [24]

    Cross-modal attention network for temporal inconsistent audio-visual event localization,

    H. Xuan, Z. Zhang, S. Chen, J. Yang, and Y . Yan, “Cross-modal attention network for temporal inconsistent audio-visual event localization,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 01, 2020, pp. 279–286

    2020

  15. [25]

    Positive sample propagation along the audio-visual event line,

    J. Zhou, L. Zheng, Y . Zhong, S. Hao, and M. Wang, “Positive sample propagation along the audio-visual event line,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 8436–8444

    2021

  16. [26]

    Discriminative cross-modality attention network for temporal inconsistent audio-visual event localization,

    H. Xuan, L. Luo, Z. Zhang, J. Yang, and Y . Yan, “Discriminative cross-modality attention network for temporal inconsistent audio-visual event localization,” IEEE Transactions on Image Processing , vol. 30, pp. 7878–7888, 2021

    2021

  17. [27]

    Audiovisual transformer with instance attention for audio-visual event localization,

    Y .-B. Lin and Y .-C. F. Wang, “Audiovisual transformer with instance attention for audio-visual event localization,” inProceedings of the Asian Conference on Computer Vision , 2020

    2020

  18. [28]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

    2014 arXiv

  19. [29]

    Eulernet: Adaptive feature interaction learning via euler’s formula for ctr prediction,

    Z. Tian, T. Bai, W. X. Zhao, J.-R. Wen, and Z. Cao, “Eulernet: Adaptive feature interaction learning via euler’s formula for ctr prediction,” in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , 2023, pp. 1376– 1385

    2023

  20. [30]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural informa- tion processing systems , vol. 25, 2012

    2012

  21. [31]

    Cnn architectures for large-scale audio classification,

    S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold et al. , “Cnn architectures for large-scale audio classification,” in 2017 ieee international conference on acoustics, speech and signal processing (ica...

    2017

  22. [32]

    Audio set: An ontology and human- labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human- labeled dataset for audio events,” in 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2017,...

    2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.