Pith. sign in

REVIEW 4 major objections 6 minor 48 references

Objects as Audio-Visual Modal Sound Fields

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read From a few photos and taps, an object's impact-sound field can be reconstructed, yielding accurate sounds at contact points that were never struck.

desk verdict AV-MSF is a genuinely new few-shot representation for object impact sound rendering, but the headline SOTA numbers in Table 1 are undermined by model selection on the ENV test metric. read the letter →

arxiv 2608.05145 v1 pith:UEN7XH6Q submitted 2026-08-05 cs.CV

Zisen Shao , Zihao Wei , Derong Jin , Ruohan Gao This is my paper
classification cs.CV
keywords impactsoundsynthesisaudio-visualrepresentationmodalfield3DGaussianSplattingfew-shotlearningcontactlocalizationobjecteditinglinearanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AV-MSF claims that an object's impact-sound field—how it sounds when tapped at any point on its surface—can be reconstructed from ordinary multi-view photos and only a handful of recorded taps. The method separates object-wide vibration properties (modal frequencies and decay rates, which are the same everywhere) from location-specific behavior (how strongly each vibration mode rings at a given contact point), and it predicts the location-specific part from visual features. This avoids both the cost and brittleness of physics-based inverse simulation and the data hunger of generative models. If the claim holds, realistic impact sounds for virtual objects can be synthesized from a few real taps, and the same representation directly supports localization of where an object was struck and text-driven editing of its material sound.

What carries the argument

The load-bearing object is Eq. (4), the modal synthesis model: $$s(x,t)=\sum_{i=1}^N g_i(x)$e^{{-d_i t}}$\sin(2\pi f_i t)+\sum_{i=1}^F \epsilon(m_i,t).$$ The frequencies $\{f_i\}$ and dampings $\{d_i\}$ are global and extracted by STFT plus log-linear regression; the neural gain field $G_\theta(x)$ supplies the only spatially varying terms, and a static filtered-noise term $\epsilon(m_i,t)$ absorbs non-modal environmental effects. The visual machinery that carries the few-shot generalization is a 3D Gaussian Splatting reconstruction whose per-Gaussian features come from DINOv2, refined by symmetry-aware pooling; a query point's gains are predicted by an MLP attending over cluster-level visual descriptors.

What would settle it

Record impact sounds at many surface points of an object with deliberately non-uniform material (e.g., a metal bowl with a rubber patch), estimate per-location damping via log-linear regression, and compare the spread of damping across locations to its mean: if the spread is comparable to the mean at several modes, the global-modal-plus-gain model in Eq. (4) cannot represent that object's sound field, and AV-MSF's rendering errors on the patched region should remain large regardless of training shots.

Watch

Extended reading notes

Core claim

The paper's central claim is that the Audio-Visual Modal Sound Field (AV-MSF) representation—built from a 3D Gaussian Splatting point cloud, dense DINOv2 visual features lifted into 3D, and modal parameters extracted from a few real impact recordings—captures an object's impact sound field well enough to render novel-position impact sounds more accurately than prior physics-based and data-driven baselines. Concretely, an impact at contact point $x$ is rendered as a sum of $N$ decaying sinusoids with object-global frequencies and dampings, gains $g_i(x)$ predicted by a neural field conditioned on local appearance, geometry offsets, and attention over region features, plus a learnable filtered-noise residual. On ObjectFolder Real and RealImpact, the method reports the best L1, L1-log, envelope, and CDPAM scores, and it retains substantial accuracy with as few as one or two training recordings.

Load-bearing premise

The load-bearing premise is that a real object's impact sound is well captured by a small set of object-wide vibration frequencies and decay rates plus per-location loudness of each mode; the paper itself notes this linear-modal assumption does not hold for all objects and that its datasets are limited to uniform materials.

Editorial extensions

If this is right

  • A single object's sound field can be reconstructed from about 4–8 recorded taps with little loss; even 1–2 taps still yield usable novel-position rendering, where data-driven models fail.
  • Rendering accuracy stops depending on nearest-neighbor training sounds: on non-symmetric objects AV-MSF beats the KNN baseline on all reported metrics.
  • The learned gain field is discriminative enough to localize a previously unheard impact by matching extracted gains against $G_\theta(x)$, outperforming DiffSound.
  • Text-guided material editing can shift modal frequencies toward a target material's cluster while preserving the spatial gain ratios dictated by linear modal analysis.
  • The visual prior transfers beyond sampled areas: training taps concentrated in one region (closest-point sampling) do not collapse performance, and more input views monotonically help.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One unstated extension is cross-object transfer: because the gains are predicted from DINOv2 features rather than object-specific coordinates, a gain field pretrained on many objects might be adapted to an unseen object with a single tap.
  • The cleanest way to isolate the visual prior would be to compare AV-MSF's predicted gains against modal analysis on a reconstructed mesh of the same object; agreement would confirm that the learned field recovers physical mode shapes rather than memorizing training locations.
  • The global-damping simplification suggests a testable refinement: allowing per-location damping would likely improve rendering on non-uniform objects, at the cost of fewer training taps being shared across the surface.
  • Because the failure cases sit near sharp geometric changes with no similar training taps, a curriculum that deliberately samples high-curvature regions, rather than farthest-point sampling, could improve the few-shot frontier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper introduces AV-MSF, an object-level audio-visual representation for impact sound synthesis. The method starts from multi-view RGB images, reconstructs a 3D Gaussian Splatting scene, lifts DINOv2 features into 3D, detects and applies geometric symmetries, extracts global modal frequencies and dampings from a few impact recordings, and learns a position-dependent neural gain field via an attention MLP. Impact sounds at novel contact locations are rendered as a sum of damped sinusoids plus a filtered-noise residual, and the parameters are optimized with a multi-scale STFT loss. Experiments on ObjectFolder Real and RealImpact compare against DiffSound, SonicGauss, KNN, and simple baselines, with additional ablations and demonstrations of contact localization and text-driven sound editing.

Significance. If the central claim holds, AV-MSF provides a notably sample-efficient and physically interpretable route to object impact sound rendering, combining a compact modal parameterization with visual priors. The paper is commendably explicit about its assumptions and limitations (Sec. 3.1 and Sec. 5), and the downstream applications, especially contact localization, are interesting and potentially useful. However, the headline state-of-the-art claim rests on an experimental protocol in which the model is selected on a test-set metric (Supp. B), and the margins over strong baselines on ObjectFolder Real are small. Until the selection protocol is corrected or clarified, the empirical contribution is not fully supported by the reported numbers.

major comments (4)
  1. [Supp. B / Table 1] The model-selection protocol in Supp. B is test-set selection on a reported metric. The text states that for each object the authors run both the single-damping and spatial-damping variants and choose the best model based on the ENV metric, and the same paragraph reports that learning rate, epoch count, cutoff weight, and cluster number are selected from ranges. ENV is one of the four metrics in Table 1, and the Table 1 margins over KNN on ObjectFolder Real are small (ENV 1.35e-4 vs 1.53e-4; L1 0.013 vs 0.014). This gives AV-MSF an oracle advantage that the KNN, DiffSound, and SonicGauss baselines do not receive, so the reported SOTA margins cannot be distinguished from selection artifact. Please select variants and hyperparameters on a held-out validation split, or report each variant separately and apply a fixed selection rule, and then compute all test metrics once.
  2. [Sec. 3.1 / Supp. B] The spatial-damping variant that can be selected in Supp. B is never defined. Sec. 3.1 only mentions that damping is 'optionally learnable as a spatially varying parameter,' but neither the main text nor Supp. A gives an equation, parameterization, initialization, or regularization for a spatially varying d_i(x). Since the comparison in Table 1 may be produced by this variant for some objects, the reader cannot reproduce the method or understand how model capacity changes across the selected variants. Please specify the spatial-damping parameterization and how it relates to the global damping extracted in Eq. (5).
  3. [Sec. 3.4 / Table 1] The modal parameter extraction is underspecified. Eq. (5) fits damping and gain for each candidate mode, but the text does not say how the number of modes N is chosen per object, how candidate peaks are selected across STFT bins, or how 'frequencies that consistently appear across recordings' is operationalized. The representation's capacity, and therefore the comparison with DiffSound and SonicGauss, depends directly on N and on the mode-selection rule. Please state the selection criterion (e.g., energy threshold, minimum number of recordings) and whether N is fixed or tuned; if tuned, the same validation-protocol concern as above applies.
  4. [Table 1 / Table 3] The main quantitative claim is based on aggregate means without any measure of variance or statistical significance. On ObjectFolder Real, the L1 difference between AV-MSF (0.013) and KNN (0.014) is small in absolute terms, and the non-symmetric subset in Table 3 shows a similarly small margin (0.0110 vs 0.0135). Please report per-object mean and median and a paired significance test, or otherwise justify why the aggregate means are robust under the current selection protocol.
minor comments (6)
  1. [Sec. 3.2] Eq. (4) is introduced by 'Following Eq. (20),' but Eq. (20) appears only in Supp. A.2; please refer to Eq. (3) in the main text or move the relevant equation to the main text.
  2. [Sec. 3.5] The clustering of Gaussian centers into K spatial groups is described as using 3D Euclidean distance, but the algorithm is not specified; please state whether this is k-means, its initialization, and how K is chosen beyond the range {128, 256, 512} in Supp. B.
  3. [Table 5 / Supp. D.2] The 'Generation' baseline in Table 5 is not defined in the main text; please describe it in the main text or at least refer explicitly to Supp. D.2 where the metric setup is given.
  4. [Supp. C.4 / Table 7] The few-shot study reports only L1; please report the other rendering metrics (L1 Log, ENV, CDPAM) so that the few-shot generalization claim can be assessed consistently with Table 1.
  5. [Sec. 5] The statement that 'existing real-world object impact sound datasets contain only objects with uniform materials' is an important scope limitation; consider stating it in the abstract or introduction as well.
  6. [Fig. 3] The spectrogram figures lack axis labels and units (time, frequency, and amplitude scale); please add them for interpretability.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the final model variant and hyperparameters are selected on the ENV metric that Table 1 then reports as the method's outcome (Supp. B), making the ENV SOTA margin an oracle selection; the modal-synthesis derivation itself is otherwise self-contained.

  1. fitted input called prediction [Supplementary Section B (Implementation Details); Table 1 (novel-position impact sound rendering)]
    "For each object, we run both the single-damping and spatial-damping variants, and choose the best model based on the ENV metric."

    ENV is one of the four metrics reported in Table 1 on the held-out evaluation split, and the same Supplementary paragraph selects learning rate, epoch count, cutoff weight, and cluster number from ranges. Selecting the final model by the same ENV metric that is then reported as the method's result means the reported ENV value is the minimum over the searched variants rather than an independent prediction of the representation. This is the fitted-input-called-prediction pattern: the discrete model choice is fit to the test metric, and the resulting ENV is then presented as evidence of state-of-the-art rendering.

full rationale

The core derivation is not circular. Equation (4) defines the impact sound as a sum of damped sinusoids plus a filtered-noise residual; the modal frequencies and dampings are extracted from the training recordings and refined, and the spatial gain field is trained with a warm-up loss against extracted gains plus an end-to-end multi-scale STFT loss on those same training recordings. Evaluation is performed at farthest-point-sampled held-out contact locations, so novel-position rendering is a genuine generalization test, not a reconstruction of the training targets. The physical assumptions (linear modal analysis, position-invariant damping in the base variant) are stated explicitly as assumptions rather than smuggled in as conclusions. The paper also compares against external or pretrained baselines on public datasets; the authors' co-authorship of some of those datasets and baselines is not load-bearing self-citation because the datasets are public and the comparisons are empirically grounded. The one load-bearing circularity is the model-selection rule in Supplementary Section B: choosing between the single-damping and spatial-damping variants, and selecting hyperparameters, by the ENV metric that Table 1 then reports as the method's outcome makes the ENV comparison an oracle selection rather than an honest prediction. This is a partial circularity and the score is 6; the remaining metrics and the representation's derivation retain independent content, and the issue is fixable by pre-registering a validation split or reporting each variant separately.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central representation rests on the linear modal analysis model (Eq. 4), which is a domain assumption from classic rigid-body sound synthesis. The empirical claim that DINOv2 features predict modal gains is a learning assumption validated only on two real-world datasets. The residual-noise model and the material-editing rescaling rule are pragmatic additions specific to this paper. No new physical entities are proposed; the audio-visual modal sound field is a representation, not an invented entity.

free parameters (8)
  • Number of modes N = Per object, selected by thresholding peak consistency across recordings
    N is determined by extracting consistently appearing peaks across recordings, but the threshold is a modeling choice that affects all rendered sounds.
  • Number of clusters K for region descriptors = Per object from {128, 256, 512}
    K controls the granularity of the region-level visual descriptors and is selected per object for efficiency and performance (Supp. B).
  • Modal frequencies {f_i} = Extracted from training recordings, then optimized end-to-end
    Global frequencies are estimated by log-linear regression from STFT sub-bands (Sec. 3.4) and then fine-tuned during training; they are fitted to the training audio.
  • Modal dampings {d_i} = Extracted and fine-tuned, or replaced by a spatial-damping variant
    Damping coefficients are averaged from per-recording estimates, then optimized; the paper runs both a global and a spatial variant and selects one per object.
  • Residual noise magnitudes {m_i} = Initialized from quiet segments, then optimized
    Per-band magnitudes of the filtered-noise residual are learnable and fitted to the training recordings (Sec. 3.4).
  • Damping variant (global vs spatial) = Selected per object based on ENV metric
    Supp. B states the authors run both variants and choose the better model using the ENV evaluation metric, introducing selection on an evaluation metric.
  • Neural field weights (attention MLP) = Optimized per object
    Weights of the attention-based gain-field MLP are fitted to the training recordings through the multi-scale STFT loss.
  • Training hyperparameters (learning rate, epochs, cutoff weight) = Selected per object from small discrete sets
    Learning rate, number of epochs, and cutoff loss weight are chosen from {1e-2, 5e-3, 1e-3}, {5000, 10000, 20000}, and {0.5, 1, 2} respectively (Supp. B), adding per-object tuning.
assumptions (5)
  • domain assumption The impact sound of an object is modeled as a sum of N damped modal vibrations with global frequencies and dampings, plus a filtered-noise residual (Eq. 4).
    This linear modal analysis assumption underlies the entire representation; it is stated in Sec. 3.1 and Sec. 3.2, and the paper notes it does not hold for all real objects.
  • domain assumption Modal frequencies and dampings are position-invariant, with only gains varying spatially (damping optionally learnable in a variant).
    Stated in Sec. 3.1 and used in the field definition in Sec. 3.2; the paper acknowledges damping can vary and makes it optionally learnable.
  • domain assumption DINOv2 visual features, lifted to 3D and symmetry-aligned, are a strong prior for predicting modal gains at novel locations.
    This is the core learning assumption motivating the method (Sec. 1 and Sec. 3.3/3.5); it is validated only empirically on two datasets.
  • ad hoc to paper A static filtered-noise residual captures unmodeled environmental effects such as low-frequency background noise and contact-force variation.
    The residual model in Eq. (6) is introduced to stabilize optimization; it is a pragmatic addition rather than derived from physical principles.
  • ad hoc to paper Material-parameter editing scales modal frequencies and dampings while leaving mode shapes unchanged, so gains rescale by a frequency ratio (Supp. A.2).
    This derivation is used for the sound-editing application; it relies on the generalized eigenproblem structure and ignores modal-mass normalization effects.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Objects as Audio-Visual Modal Sound Fields." pith.science (2026). https://pith.science/paper/UEN7XH6Q

@misc{pith2026260805145,
  author       = {Pith},
  title        = {Pith review of: Objects as Audio-Visual Modal Sound Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UEN7XH6Q}},
  note         = {Machine review of arXiv:2608.05145}
}
read the original abstract

While modern 3D reconstruction excels at modeling object geometry and appearance, it largely ignores the rich acoustic cues revealed through physical interaction. Object impact sounds convey material, stiffness, and structural properties that complement vision, yet existing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed from multi-view images and only a few impact sound recordings. AV-MSF builds on 3D Gaussian Splatting integrated with dense 3D visual feature to provide a strong geometry-aware prior, and represents the impact sound field using compact, physically meaningful modal parameters, enabling robust few-shot reconstruction. Experiments on two real-world datasets show that AV-MSF achieves state-of-the-art impact sound rendering, outperforming both physics-based and data-driven baselines. Furthermore, we demonstrate downstream applications enabled by our representation, including contact localization and object sound editing.

Figures

Figures reproduced from arXiv: 2608.05145 by the authors.

Figure 1
Figure 1. Left: We reconstruct the Audio-Visual Modal Sound Field (AV-MSF) from multi-view RGB observations and only a few impact recordings, leveraging the insight that visually similar object regions exhibit similar vibration patterns. Right: AV-MSF enables diverse applications, including novel-position impact sound synthesis, contact localization, and object sound editing. appearance from images, enabling downstream applic… view at source ↗
Figure 2
Figure 2. Overview of AV-MSF. Given multi-view image observations and few-shot impact recordings, our framework reconstructs an audio-visual modal sound field. (1) Visual Processing: We extract dense features with a pre-trained vision encoder and lift them into a 3D Gaussian Splatting (3DGS) representation to form a geometry-aware visual feature field. (2) Acoustic Processing: We extract modal parameters from a few reference … view at source ↗
Figure 3
Figure 3. Qualitative Spectrogram Comparisons. We visualize the rendered impact sounds from DiffSound [19], SonicGauss [42], and our method, alongside the ground truth. All spectrograms are generated on the settings of 20–20k Hz, the first second, scaled to (−1, 1). See Supp. for more examples [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Examples for Downstream Applications. Left [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Illustration of Objects in the ObjectFolder Real Dataset [12]. Moreover, for two impact locations x and y under the same direction d, the gain ratio ak(x) ak(y) = ϕk(x) ⊤dx ϕk(y)⊤dy is invariant to such parameter editing, since the frequency-dependent factor cancels ou…
Figure 6
Figure 6. Figure 6: Spectrogram Comparisons. We provide additional spectrogram compar￾isons between the baselines and our method. All spectrograms are generated using the same settings as in the main paper. In most cases, our method yields more accurate and sharper reconstructions. Two re…
Figure 7
Figure 7. Figure 7: Contact Localization Heatmaps. Predicted regions are shown with red indicating high likelihood and blue indicating low likelihood. Objects 7, 8, 9, and 10 show failure cases. and larger objects (Objects 2 and 3), impact sounds exhibit more diverse patterns that better …
Figure 8
Figure 8. Figure 8: CLAP Score Progression. Semantic alignment consistently improves during editing. – CLAP Score: Measures the semantic alignment between the edited impact sound and the target text prompt (higher is better). – UMAP Distance: Quantifies how accurately the edited frequenci…
Figure 9
Figure 9. Figure 9: Sound Editing in UMAP Space. Modal frequencies consistently shift to￾ward target material clusters post-editing [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 41 canonical work pages

  1. [1]

    In: ICCV (2021)

    Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., Srini- vasan, P.P.: Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In: ICCV (2021)

    2021

  2. [2]

    In: CVPR (2023)

    Clarke, S., Gao, R., Wang, M., Rau, M., Xu, J., Wang, J.H., James, D.L., Wu, J.: Realimpact: A dataset of impact sound fields for real objects. In: CVPR (2023)

    work page 2023

  3. [3]

    In: CoRL (2021)

    Clarke, S., Heravi, N., Rau, M., Gao, R., Wu, J., James, D., Bohg, J.: Diffimpact: Differentiable rendering and identification of impact sounds. In: CoRL (2021)

    work page 2021

  4. [4]

    In: ICCV (2025)

    Clarke, S., Wistreich, S., Ze, Y., Wu, J.: X-capture: An open-source portable device for multi-sensory learning. In: ICCV (2025)

    work page 2025

  5. [5]

    Presence (2007)

    Corbett, R., van den Doel, K., Lloyd, J.E., Heidrich, W.: Timbrefields: 3d interac- tive sound models for real-time audio. Presence (2007)

    work page 2007

  6. [6]

    SIGGRAPH (1999)

    Curless, B.: From range scans to 3d models. SIGGRAPH (1999)

    work page 1999

  7. [7]

    In: SIGGRAPH (2001)

    van den Doel, K., Kry, P.G., Pai, D.K.: Foleyautomatic: physically-based sound effects for interactive simulation and animation. In: SIGGRAPH (2001)

    work page 2001

  8. [8]

    In: CVPR (2025)

    Dou, Y., Oh, W., Luo, Y., Loquercio, A., Owens, A.: Hearing hands: Generating sounds from physical interactions in 3d scenes. In: CVPR (2025)

    work page 2025

Show all 48 references
  1. [9]

    In: CVPR (2024)

    Dou, Y., Yang, F., Liu, Y., Loquercio, A., Owens, A.: Tactile-augmented radiance fields. In: CVPR (2024)

    2024

  2. [10]

    CVPR (2025)

    Fan, J., Luan, F., Yang, J., Hasan, M., Wang, B.: Rng: Relightable neural gaus- sians. CVPR (2025)

    2025

  3. [11]

    In: CoRL (2021)

    Gao, R., Chang, Y.Y., Mall, S., Fei-Fei, L., Wu, J.: Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. In: CoRL (2021)

    2021

  4. [12]

    In: CVPR (2023)

    Gao, R., Dou, Y., Li, H., Agarwal, T., Bohg, J., Li, Y., Fei-Fei, L., Wu, J.: The objectfolder benchmark: Multisensory learning with neural and real objects. In: CVPR (2023)

    2023

  5. [13]

    In: CVPR (2022)

    Gao, R., Si, Z., Chang, Y.Y., Clarke, S., Bohg, J., Fei-Fei, L., Yuan, W., Wu, J.: Objectfolder 2.0: A multisensory object dataset for sim2real transfer. In: CVPR (2022)

    2022

  6. [14]

    In: ICCV (2007)

    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV (2007)

    2007

  7. [15]

    In: CVPR (2024)

    Guédon, A., Lepetit, V.: Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In: CVPR (2024)

    2024

  8. [16]

    In: CoRL (2025)

    Higuera, C., Sharma, A., Fan, T., Bodduluri, C.K., Boots, B., Kaess, M., Lambeta, M., Wu, T., Liu, Z., Hogan, F.R., et al.: Tactile beyond pixels: Multisensory touch representations for robot manipulation. In: CoRL (2025)

    2025

  9. [17]

    In: SIGGRAPH (2024)

    Huang, B., Yu, Z., Chen, A., Geiger, A., Gao, S.: 2d gaussian splatting for geo- metrically accurate radiance fields. In: SIGGRAPH (2024)

    2024

  10. [18]

    In: ACM SIGGRAPH 2016 Courses (2016)

    James, D.L., Langlois, T.R., Mehra, R., Zheng, C.: Physically based sound for computer animation and virtual environments. In: ACM SIGGRAPH 2016 Courses (2016)

    2016

  11. [19]

    In: SIGGRAPH (2024)

    Jin, X., Xu, C., Gao, R., Wu, J., Wang, G., Li, S.: Diffsound: Differentiable modal sound rendering and inverse rendering for diverse inference tasks. In: SIGGRAPH (2024)

    2024

  12. [20]

    TOG (2023)

    Kerbl, B., Kopanas, G., Leimkuehler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. TOG (2023)

    2023

  13. [21]

    RAL (2025) Objects as Audio-Visual Modal Sound Fields 17

    Lee, M., Yoo, U., Oh, J., Ichnowski, J., Kantor, G., Kroemer, O.: Sonicboom: Contact localization using array of microphones. RAL (2025) Objects as Audio-Visual Modal Sound Fields 17

    2025

  14. [22]

    In: SIGGRAPH (2000)

    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginz- ton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: SIGGRAPH (2000)

    2000

  15. [23]

    In: CoRL (2022)

    Li, H., Zhang, Y., Zhu, J., Wang, S., Lee, A.M., Xu, H., Adelson, E., Li, F.F., Gao, R., Wu, J.: See, hear, and feel: Smart sensory fusion for robotic manipulation. In: CoRL (2022)

    2022

  16. [24]

    In: ICML (2025)

    Li, T., Huang, B., Zhuang, X., Jia, D., Chen, J., Wang, Y., Chen, Z., Anu- manchipalli, G., Wang, Y.: Sounding that object: Interactive object-aware image to audio generation. In: ICML (2025)

    2025

  17. [25]

    Li, Y., Kim, H., Zhan, F., Qiu, R.Z., Ji, M., Shan, X., Zou, X., Liang, P., Pfister, H., Wang, X.: Visual acoustic fields (2025)

    2025

  18. [26]

    In: ICRA (2015)

    Luo, S., Mou, W., Althoefer, K., Liu, H.: Localizing the object contact through matching tactile features with visual map. In: ICRA (2015)

    2015

  19. [27]

    In: ICASSP (2021)

    Manocha, P., Jin, Z., Zhang, R., Finkelstein, A.: Cdpam: Contrastive learning for perceptual audio similarity. In: ICASSP (2021)

    2021

  20. [28]

    In: ECCV (2020)

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)

    2020

  21. [29]

    In: ACM SIGGRAPH/Eurographics Symposium on Computer Ani- mation (2002)

    O’Brien, J.F., Shen, C., Gatchalian, C.M.: Synthesizing sounds from rigid-body simulations. In: ACM SIGGRAPH/Eurographics Symposium on Computer Ani- mation (2002)

    2002

  22. [30]

    TMLR (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Mairal, ...

    2024

  23. [31]

    In: CVPR (2016)

    Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E.H., Freeman, W.T.: Visually indicated sounds. In: CVPR (2016)

    2016

  24. [32]

    In: SIGGRAPH (2001)

    Pai, D.K., Doel, K.v.d., James, D.L., Lang, J., Lloyd, J.E., Richmond, J.L., Yau, S.H.: Scanning physical interaction behavior of 3d objects. In: SIGGRAPH (2001)

    2001

  25. [33]

    Pang, B., Xu, C., Ren, J., Wang, G., Li, S.: Vibraverse: A large-scale geometry- acoustics alignment dataset for physically-consistent multimodal learning (2025)

    2025

  26. [34]

    Qi, Z., Yang, Y., Zhang, M., Xing, L., Wu, X., Wu, T., Lin, D., Liu, X., Wang, J., Zhao, H.: Tailor3d: Customized 3d assets editing and generation with dual-side images (2024)

    2024

  27. [35]

    TOG (2013)

    Ren, Z., Yeh, H., Lin, M.C.: Example-guided physically based modal sound syn- thesis. TOG (2013)

    2013

  28. [36]

    arXiv preprint arXiv:2505.04621 (2025)

    Richter-Powell, J., Torralba, A., Lorraine, J.: Score distillation sampling for audio: Source separation, synthesis, and beyond. arXiv preprint arXiv:2505.04621 (2025)

    2025 arXiv

  29. [37]

    In: CVPR (2016)

    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)

    2016

  30. [38]

    In: ICCV (2023)

    Sella, E., Fiebelman, G., Hedman, P., Averbuch-Elor, H.: Vox-e: Text-guided voxel editing of 3d objects. In: ICCV (2023)

    2023

  31. [39]

    In: NeurIPS (2020)

    Smith, E.J., Calandra, R., Romero, A., Gkioxari, G., Meger, D., Malik, J., Drozdzal, M.: 3D shape reconstruction from vision and touch. In: NeurIPS (2020)

    2020

  32. [40]

    In: CVPR (2023)

    Su, K., Qian, K., Shlizerman, E., Torralba, A., Gan, C.: Physics-driven diffusion models for impact sound synthesis from videos. In: CVPR (2023)

    2023

  33. [41]

    In: ICRA (2022)

    Suresh, S., Si, Z., Mangelson, J.G., Yuan, W., Kaess, M.: ShapeMap 3-D: Efficient shape mapping through dense touch and vision. In: ICRA (2022)

    2022

  34. [42]

    In: ACMMM (2025) 18 Z

    Wang, C., Li, H., Luo, Y.: Sonicgauss: Position-aware physical sound synthesis for 3d gaussian representations. In: ACMMM (2025) 18 Z. Shao et al

    2025

  35. [43]

    In: NeurIPS (2021)

    Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: NeurIPS (2021)

    2021

  36. [44]

    In: ICLR (2026)

    Ye, J., Xie, S., Zhao, R., Wang, Z., Yan, H., Zu, W., Ma, L., Zhu, J.: NANO3d: A training-free approach for efficient 3d editing without masks. In: ICLR (2026)

    2026

  37. [45]

    TMLR (2023)

    Yu, H.X., Guo, M., Fathi, A., Chang, Y.Y., Chan, E.R., Gao, R., Funkhouser, T., Wu, J.: Learning object-centric neural scattering functions for free-viewpoint relighting and scene composition. TMLR (2023)

    2023

  38. [46]

    TOG (2021)

    Zhang, X., Srinivasan, P.P., Deng, B., Debevec, P., Freeman, W.T., Barron, J.T.: Nerfactor: neural factorization of shape and reflectance under an unknown illumi- nation. TOG (2021)

    2021

  39. [47]

    In: NeurIPS (2017)

    Zhang, Z., Li, Q., Huang, Z., Wu, J., Tenenbaum, J.B., Freeman, W.T.: Shape and material from sound. In: NeurIPS (2017)

    2017

  40. [48]

    wood to ceramic

    Zhao, X., Srinivasan, P.P., Verbin, D., Park, K., Martin-Brualla, R., Henzler, P.: Illuminerf: 3d relighting without inverse rendering. In: NeurIPS (2024) Objects as Audio-Visual Modal Sound Fields 19 In this supplementary material, we first expand on our method formulations i...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.