Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation

Ali Vosoughi; Chenliang Xu; Shijian Deng; Yapeng Tian; Yiyang Su

REVIEW

Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.11713 v1 pith:ID2LEUVH submitted 2023-10-18 cs.CV cs.SDeess.AS

Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation

Yiyang Su , Ali Vosoughi , Shijian Deng , Yapeng Tian , Chenliang Xu This is my paper

classification cs.CV cs.SDeess.AS

keywords separationsoundsinvisiblesoundvisibleaudio-visualavsa-sepscene-aware

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

The audio-visual sound separation field assumes visible sources in videos, but this excludes invisible sounds beyond the camera's view. Current methods struggle with such sounds lacking visible cues. This paper introduces a novel "Audio-Visual Scene-Aware Separation" (AVSA-Sep) framework. It includes a semantic parser for visible and invisible sounds and a separator for scene-informed separation. AVSA-Sep successfully separates both sound types, with joint training and cross-modal alignment enhancing effectiveness.

Separating Invisible Sounds Toward Universal Audiovisual Scene-Aware Sound Separation

discussion (0)