Pith. sign in

REVIEW 3 major objections 4 minor 62 references

Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Scene2Sound claims that any pre-trained 3D Gaussian Splatting world can be turned into a spatially consistent audio-visual scene, training-free, by using shared Gaussians as the anchor that keeps each sound attached to its object.

desk verdict A transparent, well-engineered system paper with real novelty in task and metrics, but the advertised GSM backbone does not carry the headline result — claims need to be matched to the evidence. read the letter →

arxiv 2608.00463 v1 pith:GHTB5AKA submitted 2026-08-01 cs.CV cs.MMcs.SD

Masaki Yoshida , Ren Togo , Takahiro Ogawa , Miki Haseyama This is my paper · ORCID
classification cs.CVcs.MMcs.SD
keywords 3DGaussianSplattingspatialaudiosoundscapegenerationauditorygroundingsetmatchingmulti-viewinstanceassociationobject-basedtraining-freepipeline
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Scene2Sound claims that a silent 3D Gaussian Splatting world can be given a coherent soundscape without any training or per-scene optimization, by grounding each sound in a specific object and anchoring it to a persistent 3D position. The paper's central bet is that the 3DGS representation itself supplies the correspondence cue: when two camera views see the same object, the same Gaussians render both detections, so Jaccard overlap over those Gaussian sets identifies the object across views. On 24 curated generated worlds and 81 real-world 360-degree reconstructions, the pipeline keeps audio quality competitive with per-viewpoint generators while staying spatially consistent: audio responds to listener motion (LMC +0.283 against a chance level of 0) and held-out views support the placed sources (CGC 0.259 vs 0.065 for a single-panorama baseline). A 19-participant study found the object-based rendering preferable to three spatial baselines on preference, spatial congruency, and semantic congruency. If correct, the result turns any pre-trained 3DGS scene into a navigable audio-visual world from the scene file alone.

What carries the argument

Gaussian set matching (GSM): Jaccard similarity $J(G_{k,m}, G_{k',m'}) = |G_{k,m}\cap G_{k',m'}|/|G_{k,m}\cup G_{k',m'}|$ between the sets of Gaussians that the renderer's tile metadata records as contributing to two detected regions. It does the work of cross-view instance association: along with a fixed threshold $J_{\min}=0.15$ and Union-Find clustering, it merges observations of one physical object into a single 3D source and keeps distinct objects separate, without learned features, per-Gaussian parameters, or per-scene optimization. Source positions are weighted centroids of the merged Gaussian sets, so the anchors inherit the scene's actual geometry.

What would settle it

Take a thin or highly reflective object, render it from two nearly orthogonal viewpoints, compute the Jaccard similarity of the Gaussian sets behind its segmentation masks, and check whether same-object pairs fall below the 0.15 threshold: if they do, the object would be placed as two sources and held-out CGC precision would drop, which would contradict the shared-primitive identity premise.

Watch

Extended reading notes

Core claim

The discovery is that persistent sound sources in a 3DGS world can be created by reading the rasterizer's own bookkeeping. When a segmentation model marks an object in a rendered view, the tile metadata records exactly which Gaussians contributed to those pixels; another view of the same object shares many of the same Gaussians, while a different object does not. Scene2Sound lifts per-view masks to Gaussian sets, merges sets whose Jaccard similarity exceeds 0.15 by Union-Find, and places each merged instance at the weighted centroid of its Gaussians. That gives each sound a stable 3D anchor that survives arbitrary listener motion, with audio synthesized per source by a text-to-audio model and spatialized by a standard object-based engine. The paper argues that this primitive-index shortcut is something multi-view images plus per-view depth cannot offer, because only 3DGS has a shared persistent set of primitives across views.

Load-bearing premise

The load-bearing premise is GSM's object-identity rule: observations of the same physical object from different views are exactly those whose tile-contributed Gaussian sets have Jaccard similarity above 0.15 — if a real object seen from steep angles, or thin or reflective geometry, shares fewer Gaussians than that, its instances fragment and its sound drifts per view.

Editorial extensions

If this is right

  • Any pre-trained 3DGS scene, whether generated or reconstructed, can receive a soundscape in about 222 seconds on one GPU, with no retraining or per-scene optimization.
  • Object-based audio attached to persistent 3D anchors can be re-spatialized at arbitrary listener poses, so navigation-consistent sound no longer needs a fixed viewpoint or panorama.
  • Per-viewpoint and single-panorama pipelines cannot pass both consistency axes: they either freeze the waveform (LMC 0) or change it without geometric cause; Scene2Sound reports LMC +0.283 and held-out grounding CGC 0.259.
  • Upstream errors are contained per source: a hallucinated VLM proposal is dropped when segmentation cannot ground it, and surviving sources are unaffected, so the pipeline degrades gracefully.
  • The proposed metrics LMC and CGC give the task a measurable definition of spatial consistency, validated against real multi-position recordings and a trained acoustic-synthesis model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Gaussian-set overlap trick is not specific to audio: it is a general way to fuse any multi-view 2D signal (labels, semantics, contact points) onto a pre-trained 3DGS world without optimization, so the same association module could serve other scene-authoring tasks.
  • Because LMC and CGC are defined on rendered audio and held-out views rather than on this pipeline's internals, they could become reusable checks for any generative audio-visual scene model that claims spatial consistency.
  • The real-scene results are limited to a single capture position, so the next evident bottleneck is data: multi-position real 360-degree captures paired with soundscape annotations would likely close the LMC gap from +0.068 toward +0.283.
  • A testable extension would use GSM as a change detector: static objects keep their Gaussian sets across time, so re-rendering at later states and re-matching could flag dynamic or moving sources without any tracking model.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces Scene2Sound, a training-free pipeline that takes a pretrained 3D Gaussian Splatting (3DGS) scene and produces an object-based soundscape: automatic viewpoint selection, VLM-based sound-event detection, text-to-audio synthesis, cross-view instance association via Gaussian set matching (GSM), and real-time object-based spatialization. The authors propose two spatial-consistency metrics: Listener-Motion Consistency (LMC), which checks that audio distance tracks listener displacement, and Cross-View Grounding Consistency (CGC), a held-out reprojection F1 that checks whether placed sources are supported by views not used for placement. Experiments on 24 generated scenes and 81 real-world 360-degree scenes compare Scene2Sound with eleven baselines, including a re-implemented SonoWorld. The paper reports that Scene2Sound preserves audio quality while achieving positive LMC and higher CGC than baselines, and a 19-participant listening study shows higher MOS for preference, spatial congruency, and semantic congruency.

Significance. The paper addresses a genuine gap: adding persistent, viewpoint-consistent sound to navigable 3DGS worlds. If the central claims hold, the training-free design and object-based representation are practically valuable, and the proposed metrics are a useful contribution. The paper deserves credit for validating LMC against two real multi-position datasets and a learning check (ViGAS moves LMC from 0 to +0.455), for a held-out CGC protocol with construct-validity corruptions, for a co-located-instance stress test (0/16,284 over-merges), and for committing to release the SoundscapePLY testbed and code. However, the current evidence does not cleanly demonstrate that GSM, the advertised backbone, causes the headline spatial-consistency gains, and several secondary claims outrun the measurements.

major comments (3)
  1. [Sec. V-D, Table V] The ablation 'w/o inst. assoc.' leaves CGC F1 essentially unchanged (0.2593 vs 0.2589) and only shifts the operating point from precision to recall (0.441/0.229 vs 0.302/0.316). The listener-axis contrast (+0.283 vs +0.228) is not a clean ablation because, per the table footnote, the variant row is scored with cached pre-adoption audio while the Scene2Sound row uses the adopted audio prompts and gain normalization. The threshold sweep in Appendix C further shows CGC rising monotonically with J_min up to 0.50 (0.159 to 0.286), suggesting the benefit may come from conservative merging rather than the shared-primitive-index cue per se. Consequently, neither spatial-consistency axis provides clean causal support for GSM as the source of the headline result. Please run the 'w/o inst. assoc.' variant at the final adopted configuration with matched audio prompts and gain normalization, and report LMC, CGC F1, precision, and recall.
  2. [Sec. V-F, App. F] The 19-participant subjective study was conducted at an earlier pipeline configuration (placement cap 3, J_min=0.05, earlier audio-prompt template), not at the final configuration (J_min=0.15, no cap, adopted prompts) used for all quantitative results. Since the subjective evaluation is presented as confirmation of the proposed system's perceptual benefit, it cannot validate the final GSM-based system; at most it validates the object-based rendering architecture under an earlier association policy. Please either rerun the study at the final configuration or explicitly limit the subjective claim to the rendering architecture and state that the association stage was not perceptually validated.
  3. [Sec. V-B, App. H] The real-world transfer evidence is weaker than the abstract's claim that Scene2Sound 'remains spatially consistent' on 3DGS scenes from real-world 360-degree captures. On D-SAV360, LMC is +0.068 over all 81 scenes and +0.120 on the 46 scenes with grounded point sources; the 35 scenes without grounded sources score exactly 0, and the single capture position precludes the cross-view association that the method's grounding axis depends on, so CGC is not computed in that regime. Please either temper the real-world spatial-consistency claim to 'weakly above chance on the listener axis' or provide a multi-position real-capture evaluation.
minor comments (4)
  1. [Sec. IV, Audio Quality] FAD is computed from 24 samples (one per scene) and the paper acknowledges that this is noisy, but no confidence intervals are reported for the FAD differences; given that the metric is used comparatively, bootstrap CIs around the FAD contrasts would make the 'preserves audio quality' claim easier to evaluate.
  2. [Sec. III-B, Eq. (2)] The distance weight w_d(d_ij) is only described as inverse-distance weighting; providing the explicit functional form in the main text or a specific equation number in the supplementary material would improve reproducibility.
  3. [Table V, App. C] The LMC values in the J_min sweep (+0.253 at the adopted column) predate the final rendering revision and differ from Table III's +0.283; the main text and appendix should be harmonized or have the discrepancy explicitly explained at one location.
  4. [App. H] The phrase 'cross-view association' for the three yaw-rotated panoramas from the same capture position may mislead readers; these are same-position rotations, not spatially distinct views, so the limitations of the single-position protocol should be stated at the first use of that phrase in the appendix.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the central pipeline is training-free and externally validated, with only minor metric-selection caveats.

full rationale

Scene2Sound's derivation chain is a composition of off-the-shelf foundation models and a geometric association rule (GSM); no parameter is fitted to the headline quantities and then reported as a prediction. The two spatial-consistency metrics are author-defined, but they carry external anchors: LMC is validated on Real Acoustic Fields and Replay-NV AS real recordings and on a trained-ViGAS learning check, and CGC uses a held-out-view reprojection protocol with construct-validity corruptions. Per-viewpoint baselines score LMC≈0 and the random-camera ablation collapses CGC, so the metrics are not definitionally equal to the method's output. The remaining issues are evaluation-protocol weaknesses rather than circular reductions: the J_min threshold is selected on a panel, the w/o-inst.-assoc. LMC row uses cached pre-adoption audio per Table V's footnote, and the VLM event inventory is built from all views so CGC is not fully held-out. All three are disclosed in the paper and none makes the central claim true by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical entities are postulated. The 'source_id', the object-based audio-event representation (audio signal plus 3D position), and the ambient bed are data-format choices standard in object-based audio, not invented physical objects. The free parameters that actually shape the headline results are J_min, the depth-gate tolerance, the viewpoint sampling constants, and the gain/normalization choices made after listening review; these are disclosed but several were tuned with knowledge of the reported metrics.

free parameters (6)
  • J_min (Jaccard matching threshold) = 0.15
    Selected on a fixed 8-scene panel with paired promotion gates; the Table VIII sweep shows CGC, precision, and LMC all peak or plateau near this value, so it is effectively tuned to the reported metrics (App. C.b, Eq. 3).
  • alpha_d (depth-consistency gate tolerance) = 1.5
    Fixed scale-relative tolerance for the one-sided background-removal gate; chosen by hand (App. C.a).
  • K, N (viewpoint count and ray count) = K=5, N=20
    Sampling constants chosen by hand for the coverage/quality trade-off (Sec. III-B).
  • ambient bed gain = 0.4
    Adopted after an author listening review that improved the listener axis and audio metrics (App. C.b).
  • per-label energy normalization = 1/sqrt(n)
    Design constant so repeated objects contribute their label's energy once rather than n times (Sec. III-F).
  • LMC tie tolerance epsilon = 10^-6 times median pairwise audio distance
    Metric parameter set relative to each evaluation set's distance scale; not fitted to the model (App. E.a).
assumptions (5)
  • domain assumption Navigable 3DGS scenes have a hollow structure: Gaussians form surfaces surrounding an open interior, so center-cast rays reach the scene shell.
    Underlies automatic viewpoint selection (Sec. III-B); the Discussion concedes failures on large or complex scenes make downstream modules miss sources or fail cross-view association.
  • domain assumption Observations of the same object from different views share Jaccard-overlapping Gaussian sets above 0.15, and distinct same-class instances do not.
    Core premise of Gaussian set matching (Sec. III-E, Eq. 3); checked only for over-merging on 5 scenes (16,284 pairs), not for under-merging or angle-extreme views.
  • domain assumption The VLM identifies sound events with roughly 93.5% proposal validity and 97% element coverage, and SAM3 grounds them.
    The whole soundscape inventory depends on Qwen2.5-VL-32B quality; swapping to the 7B model cuts detected sources to a third (Sec. V-E, App. D).
  • domain assumption Tile-Gaussian rasterizer metadata, queried under cubemap orientation, gives a consistent correspondence between 2D masks and 3D Gaussian sets after the one-sided depth gate.
    The lifting step relies on gsplat tile bookkeeping; background removal assumes tile overflow adds only farther primitives (Sec. III-E, App. C.a).
  • standard math Standard Jaccard, Union-Find, and weighted-centroid mathematics.
    Equations 3 to 5 are elementary set and geometry operations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds." pith.science (2026). https://pith.science/paper/GHTB5AKA

@misc{pith2026260800463,
  author       = {Pith},
  title        = {Pith review of: Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GHTB5AKA}},
  note         = {Machine review of arXiv:2608.00463}
}
read the original abstract

3D Gaussian Splatting (3DGS) turns captured or generated imagery into photorealistic 3D world simulations that users can freely explore, yet these worlds remain silent. Because existing audio generation methods condition on a single image or viewpoint, their sound is tied to that observation and cannot stay consistent while a listener moves. We introduce the task of generating a spatially consistent soundscape for a given 3DGS world through auditory grounding, identifying which objects in the world should emit sound and anchoring each to a persistent 3D position, and present Scene2Sound, a training-free framework built on this grounding. From the input world alone, our pipeline selects viewpoints that jointly cover the scene, identifies sound-emitting objects with a vision-language model, and associates the multi-view detections into 3D instances through Gaussian set matching, which measures the overlap between the Gaussian sets that render each detection. Each source then receives generated audio that a standard object-based audio engine spatializes in real time at arbitrary listener poses. We further propose two spatial-consistency metrics, one testing whether rendered audio responds consistently to listener motion and one testing whether the claimed sources are supported by views held out from their placement. On a curated set of generated 3DGS worlds and on 3DGS scenes generated from real-world 360-degree captures, Scene2Sound preserves the audio quality of strong per-viewpoint baselines while remaining spatially consistent where per-viewpoint and single-panorama pipelines do not, and a user study confirms the perceptual benefit. Project page: https://masaki-lmd.github.io/scene2sound/.

Figures

Figures reproduced from arXiv: 2608.00463 by the authors.

Figure 1
Figure 1. Soundscape generation for 3DGS scenes. Given a pre-trained 3DGS scene, Scene2Sound automatically infers what sounds should be present and where they originate, producing an object-based audio configuration. Combined with a user pose and an audio engine, this enables spatially consistent 3D audio that adapts as users navigate through the scene. a single photograph into a panorama and generates an audio￾visual scene a… view at source ↗
Figure 2
Figure 2. Framework overview. Our pipeline consists of three stages: (1) Scene Understanding with Viewpoint Selection renders multiple viewpoints from the 3DGS scene and infers sound events with their visual grounding using VLMs and segmentation models; (2) Audio Source Placement lifts 2D grounding regions into 3D via Gaussian set matching to produce spatially consistent instance associations; (3) Audio Source Generation synt… view at source ↗
Figure 3
Figure 3. Instance association for a fan observed from four viewpoints. Per [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Instance association results across four representative scenes: projected [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Subjective evaluation (MOS, 1–5 scale); error bars: standard error. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 56 canonical work pages

  1. [1]

    3D gaussian splatting for real-time radiance field rendering,

    B. Kerbl, G. Kopanas, T. Leimkuehler, and G. Drettakis, “3D gaussian splatting for real-time radiance field rendering,”ACM Trans. Graph., vol. 42, 2023

    work page 2023

  2. [2]

    DreamScene360: Unconstrained text-to-3D scene generation with panoramic gaussian splatting,

    S. Zhou, Z. Fan, D. Xuet al., “DreamScene360: Unconstrained text-to-3D scene generation with panoramic gaussian splatting,” inProc. ECCV, 2024

    work page 2024

  3. [3]

    WonderWorld: Interactive 3D scene generation from a single image,

    H.-X. Yu, H. Duan, C. Herrmannet al., “WonderWorld: Interactive 3D scene generation from a single image,” inProc. CVPR, 2025

    work page 2025

  4. [4]

    19, 2026

    World Labs, “Marble,” https://marble.worldlabs.ai/, 2025, accessed: Feb. 19, 2026

    work page 2025

  5. [5]

    AG2aussian: Anchor-graph structured gaussian splatting for instance-level 3D scene understanding and editing,

    Z. Wang, M. Li, and C. Tu, “AG2aussian: Anchor-graph structured gaussian splatting for instance-level 3D scene understanding and editing,” inProc. ICCV, 2025

    work page 2025

  6. [6]

    PhysSplat: Efficient physics simulation for 3D scenes via MLLM-guided gaussian splatting,

    H. Zhao, H. Wang, X. Zhaoet al., “PhysSplat: Efficient physics simulation for 3D scenes via MLLM-guided gaussian splatting,” inProc. ICCV, 2025

    work page 2025

  7. [7]

    R. M. Schafer,The Soundscape: Our Sonic Environment and the Tuning of the World. Rochester, VT, USA: Destiny Books, 1993

    work page 1993

  8. [8]

    Spatialized audio rendering for immersive virtual environments,

    M. Naef, O. Staadt, and M. Gross, “Spatialized audio rendering for immersive virtual environments,” inProc. VRST, 2002

    work page 2002

Show all 62 references
  1. [9]

    Sounding that object: Interactive object-aware image to audio generation,

    T. Li, B. Huang, X. Zhuanget al., “Sounding that object: Interactive object-aware image to audio generation,” inProc. ICML, 2025

    2025

  2. [10]

    SEE-2-SOUND: Zero-shot spatial environment-to-spatial sound,

    R. Dagli, S. Prakash, R. Wu, and H. Khosravani, “SEE-2-SOUND: Zero-shot spatial environment-to-spatial sound,” inProc. ACM SIGGRAPH Posters, 2025

    2025

  3. [11]

    OmniAudio: Generating spatial audio from 360- degree video,

    H. Liu, T. Luo, Q. Jianget al., “OmniAudio: Generating spatial audio from 360- degree video,” inProc. ICML, 2025. 14

    2025

  4. [12]

    Visual acoustic fields,

    Y . Li, H. Kim, F. Zhanet al., “Visual acoustic fields,”arXiv preprint arXiv:2503.24270, 2025

    2025 arXiv

  5. [13]

    Sonora: Human- AI co-creation of 3D audio worlds and its impact on anxiety and cognitive load,

    F. M. De La Torre, J. Hernandez, A. D. Wilson, and J. Amores, “Sonora: Human- AI co-creation of 3D audio worlds and its impact on anxiety and cognitive load,” inProc. CHI, 2025

    2025

  6. [14]

    SonoWorld: From one image to a 3D audio-visual scene,

    D. Jin, X. Chen, M. C. Lin, and R. Gao, “SonoWorld: From one image to a 3D audio-visual scene,” inProc. CVPR, 2026

    2026

  7. [15]

    Gaussian grouping: Segment and edit anything in 3D scenes,

    M. Ye, M. Danelljan, F. Yu, and L. Ke, “Gaussian grouping: Segment and edit anything in 3D scenes,” inProc. ECCV, 2024

    2024

  8. [16]

    Object-based 3D audio production for virtual reality using the audio definition model,

    C. Pike, R. Taylor, T. Parnell, and F. Melchior, “Object-based 3D audio production for virtual reality using the audio definition model,” inProc. AES Int. Conf. Audio for Virtual and Augmented Reality, 2016

    2016

  9. [17]

    Qwen2.5-VL technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tanget al., “Qwen2.5-VL technical report,”arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  10. [18]

    SAM 3: Segment anything with concepts,

    N. Carion, L. Gustafson, Y .-T. Hu, S. Debnath, R. Hu, D. Suris, C. Ryali, K. V . Alwala, H. Khedr, A. Huanget al., “SAM 3: Segment anything with concepts,” arXiv preprint arXiv:2511.16719, 2025

    2025 arXiv

  11. [19]

    Stable audio open,

    Z. Evans, J. D. Parker, C. Carret al., “Stable audio open,” inProc. ICASSP, 2025

    2025

  12. [20]

    AudioLDM 2: Learning holistic audio generation with self-supervised pretraining,

    H. Liu, Y . Yuan, X. Liuet al., “AudioLDM 2: Learning holistic audio generation with self-supervised pretraining,”IEEE/ACM Trans. Audio Speech Lang. Process., vol. 32, 2024

    2024

  13. [21]

    AudioLCM: Efficient and high-quality text-to- audio generation with minimal inference steps,

    H. Liu, R. Huang, Y . Liuet al., “AudioLCM: Efficient and high-quality text-to- audio generation with minimal inference steps,” inProc. ACM MM, 2024

    2024

  14. [22]

    FlashAudio: Rectified flow for fast and high- fidelity text-to-audio generation,

    H. Liu, J. Wang, R. Huanget al., “FlashAudio: Rectified flow for fast and high- fidelity text-to-audio generation,” inProc. ACL, 2025

    2025

  15. [23]

    TangoFlux: Super fast and faithful text to audio generation with flow matching and CLAP-ranked preference optimiza- tion,

    C.-Y . Hung, N. Majumder, Z. Konget al., “TangoFlux: Super fast and faithful text to audio generation with flow matching and CLAP-ranked preference optimiza- tion,” inProc. ICLR, 2026

    2026

  16. [24]

    Lumina-Next: Making Lumina-T2X stronger and faster with Next-DiT,

    L. Zhuo, R. Du, H. Xiaoet al., “Lumina-Next: Making Lumina-T2X stronger and faster with Next-DiT,”arXiv preprint arXiv:2406.18583, 2024

    2024 arXiv

  17. [25]

    Diff-Foley: Synchronized video-to-audio synthesis with latent diffusion models,

    S. Luo, C. Yan, C. Hu, and H. Zhao, “Diff-Foley: Synchronized video-to-audio synthesis with latent diffusion models,” inProc. NeurIPS, 2023

    2023

  18. [26]

    ThinkSound: Chain-of-thought reasoning in multimodal large language models for audio generation and editing,

    H. Liu, K. Luo, J. Wanget al., “ThinkSound: Chain-of-thought reasoning in multimodal large language models for audio generation and editing,” inProc. NeurIPS, 2025

    2025

  19. [27]

    PrismAudio: Decomposed chain-of-thoughts and multi-dimensional rewards for video-to-audio generation,

    H. Liu, K. Luo, W. Wanget al., “PrismAudio: Decomposed chain-of-thoughts and multi-dimensional rewards for video-to-audio generation,”arXiv preprint arXiv:2511.18833, 2025

    2025

  20. [28]

    MMAudio: Taming multimodal joint training for high-quality video-to-audio synthesis,

    H. K. Cheng, M. Ishii, A. Hayakawaet al., “MMAudio: Taming multimodal joint training for high-quality video-to-audio synthesis,” inProc. CVPR, 2025

    2025

  21. [29]

    ImmerseDiffusion: A gener- ative spatial audio latent diffusion model,

    M. Heydari, M. Souden, B. Conejo, and J. Atkins, “ImmerseDiffusion: A gener- ative spatial audio latent diffusion model,” inProc. ICASSP, 2025

    2025

  22. [30]

    Hearing anything anywhere,

    M. L. Wang, R. Sawata, S. Clarkeet al., “Hearing anything anywhere,” inProc. CVPR, 2024

    2024

  23. [31]

    A V-RIR: Audio-visual room impulse response estimation,

    A. Ratnarajah, S. Ghosh, S. Kumaret al., “A V-RIR: Audio-visual room impulse response estimation,” inProc. CVPR, 2024

    2024

  24. [32]

    Learning neural acoustic fields,

    A. Luo, Y . Du, M. Tarret al., “Learning neural acoustic fields,” inProc. NeurIPS, vol. 35, 2022

    2022

  25. [33]

    Acoustic volume rendering for neural impulse response fields,

    Z. Lan, C. Zheng, Z. Zheng, and M. Zhao, “Acoustic volume rendering for neural impulse response fields,” inProc. NeurIPS, vol. 37, 2024

    2024

  26. [34]

    Novel-view acoustic synthesis,

    C. Chen, A. Richard, R. Shapovalovet al., “Novel-view acoustic synthesis,” in Proc. CVPR, 2023

    2023

  27. [35]

    A V-NeRF: Learning neural fields for real-world audio-visual scene synthesis,

    S. Liang, C. Huang, Y . Tianet al., “A V-NeRF: Learning neural fields for real-world audio-visual scene synthesis,” inProc. NeurIPS, vol. 36, 2023

    2023

  28. [36]

    A V-GS: Learning material and geometry aware priors for novel view acoustic synthesis,

    S. Bhosale, H. Yang, D. Kanojiaet al., “A V-GS: Learning material and geometry aware priors for novel view acoustic synthesis,” inProc. NeurIPS, 2024

    2024

  29. [37]

    NeRAF: 3D scene infused neural radiance and acoustic fields,

    A. Brunetto, S. Hornauer, and F. Moutarde, “NeRAF: 3D scene infused neural radiance and acoustic fields,” inProc. ICLR, 2025

    2025

  30. [38]

    Object-based audio reproduction and the audio scene description format,

    M. Geier, J. Ahrens, and S. Spors, “Object-based audio reproduction and the audio scene description format,”Organised Sound, vol. 15, 2010

    2010

  31. [39]

    An audio-visual system for object- based audio: From recording to listening,

    P. Coleman, A. Franck, J. Francombeet al., “An audio-visual system for object- based audio: From recording to listening,”IEEE Trans. Multimedia, vol. 20, 2018

    2018

  32. [40]

    Segment any 3D gaussians,

    J. Cen, J. Fang, C. Yanget al., “Segment any 3D gaussians,” inProc. AAAI, 2025

    2025

  33. [41]

    OpenSplat3D: Open-vocabulary 3D instance segmentation using gaussian splatting,

    J. Piekenbrinck, C. Schmidt, A. Hermanset al., “OpenSplat3D: Open-vocabulary 3D instance segmentation using gaussian splatting,” inProc. CVPR Workshops, 2025

    2025

  34. [42]

    Object-aware gaussian splatting for robotic manipulation,

    Y . Li and D. Pathak, “Object-aware gaussian splatting for robotic manipulation,” inICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

    2024

  35. [43]

    GaussianCut: Interactive segmentation via graph cut for 3D gaussian splatting,

    U. Jain, A. Mirzaei, and I. Gilitschenski, “GaussianCut: Interactive segmentation via graph cut for 3D gaussian splatting,” inProc. NeurIPS, vol. 37, 2024

    2024

  36. [44]

    Training-free hierarchical scene understanding for gaussian splatting with superpoint graphs,

    S. Dai, Y . Qu, Z. Liet al., “Training-free hierarchical scene understanding for gaussian splatting with superpoint graphs,” inProc. ACM MM, 2025

    2025

  37. [45]

    Lifting by gaussians: A simple, fast and flexible method for 3D instance segmentation,

    R. Chacko, N. Haeni, E. Khaliullinet al., “Lifting by gaussians: A simple, fast and flexible method for 3D instance segmentation,” inProc. WACV, 2025

    2025

  38. [46]

    DCSEG: Decoupled 3D open-set segmentation using gaussian splatting,

    L. Wiedmann, L. Wiehe, and D. Rozenberszki, “DCSEG: Decoupled 3D open-set segmentation using gaussian splatting,” inProc. CVPR Workshops, 2025

    2025

  39. [47]

    Exploring CLIP for assessing the look and feel of images,

    J. Wang, K. C. Chan, and C. C. Loy, “Exploring CLIP for assessing the look and feel of images,” inProc. AAAI, 2023

    2023

  40. [48]

    gsplat: An open-source library for gaussian splatting,

    V . Ye, R. Li, J. Kerret al., “gsplat: An open-source library for gaussian splatting,” J. Mach. Learn. Res., vol. 26, 2025

    2025

  41. [49]

    Fr ´echet audio distance: A reference-free metric for evaluating music enhancement algorithms,

    K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi, “Fr ´echet audio distance: A reference-free metric for evaluating music enhancement algorithms,” inProc. Interspeech, 2019

    2019

  42. [50]

    PANNs: Large-scale pretrained audio neural networks for audio pattern recognition,

    Q. Kong, Y . Cao, T. Iqbalet al., “PANNs: Large-scale pretrained audio neural networks for audio pattern recognition,”IEEE/ACM Trans. Audio Speech Lang. Process., vol. 28, 2020

    2020

  43. [51]

    Correlation of Fr ´echet audio distance with human perception of environmental audio is embedding dependent,

    M. Tailleur, J. Lee, M. Lagrangeet al., “Correlation of Fr ´echet audio distance with human perception of environmental audio is embedding dependent,” inProc. EUSIPCO, 2024

    2024

  44. [52]

    Sound scene synthesis at the DCASE 2024 challenge,

    M. Lagrange, J. Lee, M. Tailleuret al., “Sound scene synthesis at the DCASE 2024 challenge,”arXiv preprint arXiv:2501.08587, 2025

    2024 arXiv

  45. [53]

    Clotho: An audio captioning dataset,

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An audio captioning dataset,” inProc. ICASSP, 2020

    2020

  46. [54]

    ImageBind: One embedding space to bind them all,

    R. Girdhar, A. El-Nouby, Z. Liuet al., “ImageBind: One embedding space to bind them all,” inProc. CVPR, 2023

    2023

  47. [55]

    Large-scale contrastive language-audio pretrain- ing with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhanget al., “Large-scale contrastive language-audio pretrain- ing with feature fusion and keyword-to-caption augmentation,” inProc. ICASSP, 2023

    2023

  48. [56]

    Tango 2: Aligning diffusion-based text-to-audio generations through direct preference optimization,

    N. Majumder, C.-Y . Hung, D. Ghosalet al., “Tango 2: Aligning diffusion-based text-to-audio generations through direct preference optimization,” inProc. ACM MM, 2024

    2024

  49. [57]

    Seeing and hearing: Open-domain visual-audio generation with diffusion latent aligners,

    Y . Xing, Y . He, Z. Tianet al., “Seeing and hearing: Open-domain visual-audio generation with diffusion latent aligners,” inProc. CVPR, 2024

    2024

  50. [58]

    I hear your true colors: Image guided audio generation,

    R. Sheffer and Y . Adi, “I hear your true colors: Image guided audio generation,” inProc. ICASSP, 2023

    2023

  51. [59]

    ViSAGe: Video-to-spatial audio generation,

    J. Kim, H. Yun, and G. Kim, “ViSAGe: Video-to-spatial audio generation,” in Proc. ICLR, 2025

    2025

  52. [60]

    D-SA V360: A dataset of gaze scanpaths on 360° ambisonic videos,

    E. Bernal-Berdun, D. Martin, S. Malpicaet al., “D-SA V360: A dataset of gaze scanpaths on 360° ambisonic videos,”IEEE Trans. Vis. Comput. Graph., 2023

    2023

  53. [61]

    Real acoustic fields: An audio-visual room acoustics dataset and benchmark,

    Z. Chenet al., “Real acoustic fields: An audio-visual room acoustics dataset and benchmark,” inProc. CVPR, 2024

    2024

  54. [62]

    FLUX.2 [klein] 9B,

    Black Forest Labs, “FLUX.2 [klein] 9B,” https://huggingface.co/black-forest-labs/ FLUX.2-klein-9B, 2026, Hugging Face model card, accessed 2026-02-15

    2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.