REVIEW 4 major objections 4 minor 47 references
FillGauss: Fine-Grained Filling-Aware Impact Sound Generation for 3D Gaussian Splatting
T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Containers sound different when filled, and a 3D-aware diffusion model can synthesize that difference.
desk verdict Useful new dataset and a plausible fill-aware conditioning scheme, but the evaluation overreaches: the split may leak object identity and the SOTA claim trips over the paper's own FAD table. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the cross-modal attention fusion pair: context self-attention combines the positional embedding of the strike point with the text embedding of the physical state into a unified excitation context; then geometry cross-attention lets the 3D Gaussian features query that context, so each surface region selectively absorbs positional and damping semantics. This fused embedding conditions a diffusion-transformer denoiser, trained with Huber loss, that predicts the audio latent. The dataset's verified physical trends (dominant frequency decreasing with fill level, higher decay for rice than water, higher spectral centroid for steel) provide the prior that the model is
What would settle it
An object-disjoint evaluation: hold out all recordings of a set of containers and measure FAD on impacts of those unseen containers. If FillGauss's FAD degrades substantially compared to the random-split result, the claimed physical generalization is not established. A second check is to synthesize audio for the same 3D scan with two different fill descriptions and verify the dominant-frequency shift quantitatively.
Extended reading notes
Core claim
On its own terms, the central discovery is that internal filling state is a first-class acoustic condition: adding mass lowers resonance frequency according to the harmonic-oscillator relation f ∝ sqrt(k/m), granular fill suppresses resonance through high damping, and harder strikers shift energy to higher frequencies. FillGauss represents an object as 3D Gaussians, encodes the exact strike point with positional encoding, encodes fill and striker as natural-language text, and uses a self-attention-plus-cross-attention fusion to make the 3D surface query the localized physical context. The fused condition guides a pretrained latent audio diffusion model. The paper shows qualitative spectrogra
Load-bearing premise
The reported 80/20 train/test split is described only over samples, not over objects; if the same container appears in both training and testing at different fill levels, the model could memorize object-specific resonances rather than learn transferable physics.
Editorial extensions
If this is right
- If FillGauss works as claimed, interactive 3D scenes can add plausible impact sounds for containers whose fill state changes, without re-recording audio for every state.
- The dependence on a textual fill description means fill state can be controlled at inference time: the same 3D scan can sound hollow, water-filled, or rice-filled on demand.
- Because the model is conditioned on exact 3D strike coordinates, sounds can vary continuously with strike position across the surface, enabling positional audio feedback in virtual reality and robotic manipulation.
- The finding that fine-grained text prompts outperform hard-coded categorical embeddings suggests pretrained language models are an effective carrier for physical condition semantics.
- The sparsity experiment indicates that a compressed 3D Gaussian representation still yields acceptable audio, pointing to a practical route for real-time synthesis.
Reading between the lines
- If fill state is truly acoustically discriminable, then the audio itself could be used to infer the fill state of an unseen container, a task the paper does not attempt but its dataset enables.
- The paper's argument implies a failure mode for purely geometric 3D-aware models: they will average over fill states and produce a generic container sound; one could test this by comparing a hollow-only model on filled containers.
- The five-level discretization of fill could be extended to continuous regression or to dynamic processes like sloshing, which the paper lists as future work.
- A practical extension would be to train on object-disjoint splits to verify generalization beyond the 88 recorded containers.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a new task, fine-grained filling-aware impact sound generation, and contributes FillImpact, a dataset of over 5,000 recordings from 88 real containers with varied fill materials, fill levels, strikers, and strike positions, together with 3DGS reconstructions. It then proposes FillGauss, a latent-diffusion framework that fuses 3DGS geometric features, a 3D strike coordinate, and text-described physical state through self- and cross-attention to condition audio generation on the internal fill state. The paper reports physical acoustic analysis of the dataset, quantitative comparisons against SonicGauss and TangoFlux, subjective listening tests, ablations, and sparsity sensitivity experiments, claiming a new state-of-the-art for physically grounded cross-modal audio generation. The central novelty is conditioning on internal fill state rather than only surface geometry, and this is a promising direction.
Significance. If the empirical claims are sustained, the work would be a valuable contribution: it defines a concrete task, provides a controlled multimodal dataset with physical annotations, and proposes a principled architecture that combines 3DGS geometry with internal-state conditioning. The physical analysis of the dataset (e.g., frequency shift with fill level, distinct decay for granular vs. liquid fill, striker hardness effects) is a useful prior for future work, and the ablations show the contribution of each component. However, the central generalization claim to unseen 3D-scanned containers is currently unsupported because the train/test split is not stated to be object-disjoint. In addition, the headline state-of-the-art claim conflicts with the paper's own primary metric, and the subjective evaluation has internal inconsistencies. With a corrected evaluation protocol and appropriately qualified claims, the contribution would be solid and publishable.
major comments (4)
- [Section V.A (Implementation Details)] The 80/20 split is described only as a 'partition of FillImpact Dataset into an 80% training split and a 20% testing split,' with no statement that the 88 physical objects are split disjointly. Because each object appears in multiple recordings across fill levels, fill materials, strikers, and strike positions, a sample-level split will likely place the same object in both training and test sets. The 3DGS geometry stream in the cross-attention fusion (Eq. 4) can then act as a near-unique object fingerprint, letting the model retrieve object-specific resonances rather than predict them from geometry plus fill state. This would inflate every metric in Tables I–IV and invalidates the claim of generalization to unseen containers. Please report an object-disjoint split with per-object counts, and/or add a held-out-object evaluation.
- [Table I vs. Abstract] FAD is declared the primary objective metric (Section V.A), but TangoFlux FT achieves a better FAD (1.0627) than FillGauss (1.3500), and FillGauss is also worse than SonicGauss FT on KL sig (0.6977 vs. 0.5750). The unqualified 'state-of-the-art' claims in the Abstract and Section V.B are therefore not supported by the paper's own primary metric. Either restrict the claim to the best 3D-aware method, or provide a principled argument for why a worse FAD is acceptable (e.g., TangoFlux's lack of spatial grounding) and support it with a metric that privileges 3D conditioning.
- [Section V.A / Appendix C / Table II] There is an inconsistency in the reported number of participants: Section V.A states 8 participants, while Appendix C states 20. Additionally, Table II shows FillGauss MOS (4.43) slightly below SonicGauss ZS (4.50), yet FillGauss has a 69.6% win rate; this is not explained. Because the subjective results are central to the physical-grounding claims, the participant count must be corrected and the MOS/win-rate relationship interpreted. Statistical significance testing for the subjective comparisons should also be reported.
- [Abstract / Section III / Section IV.A] The task is described as requiring a 'continuous fill level' in the Abstract and IV.A, but Section III states the fill level is discretized into five states (level 0 = hollow). 'Fine-grained' conditioning and interpolation across arbitrary fill levels is not demonstrated with five discrete levels. Clarify the actual conditioning variable and, if continuous interpolation is claimed, add an experiment on unseen intermediate fill levels.
minor comments (4)
- [Table III] Table III lists only four attributes (Striker Mat., Pos., Fill Mat., Level), but the text says five attributes including container material are evaluated. Add the missing column and report its matching rate.
- [Section V.A] The metric 'KL sig' is not defined; specify how the KL divergence is computed and what 'sig' denotes. Also, FAD is described as 'reference-free,' which is misleading; FAD compares generated and reference feature distributions via VGGish.
- [Reproducibility] No code, checkpoints, or split metadata are released. If the object-disjoint split is adopted, release the object IDs and random seed for reproducibility.
- [Figure 4 / Section III] The exact normalization used for 'normalized dominant frequency' is not stated. Clarify whether it is per-object or per-level normalization, so the claimed physical trend can be reproduced.
Circularity Check
No significant circularity: the paper's empirical predictions are not equivalent by construction to their conditioning inputs.
full rationale
FillGauss is an empirical cross-modal generative model, not a derivation of physical laws from first principles. The physical principles cited (f∝sqrt(k/m), damping differences) are introduced from external acoustics literature (e.g., [24]) and used as post-hoc validation of the dataset and of generated spectrograms; they are not outputs derived from the model equations, and the model is not fitted to those laws. Fill-awareness is architecturally induced by conditioning on fill-level text and/or categorical embeddings, but the paper's quantitative and perceptual evaluations test whether the learned model actually uses that conditioning (e.g., Table III attribute matching and Table V sparsity sensitivity), so success is not guaranteed by construction. The candidate concern that the 80/20 split is described only as a sample partition (Section V.A) is a real evaluation-validity risk (possible object-level leakage), but it is not a circularity of the derivation: it concerns whether the test metrics measure generalization, not whether a prediction reduces to an input by definition or via self-citation. No load-bearing self-citation was found: the cited SonicGauss, TangoFlux, SplatFormer, and Stable Audio Open are external prior works, and no uniqueness theorem or ansatz is imported from the present authors' own prior work. Therefore no circular step can be exhibited, and the appropriate score is 0.
Assumptions & free parameters
free parameters (3)
- Fill-level discretization =
5 levels (0-4)
- Huber loss threshold δ =
1.0
- 3DGS sparsity voxel sizes =
{0.0, 0.02, 0.05, 0.08, 0.12}
assumptions (5)
- domain assumption Frozen pre-trained Audio VAE and text encoder preserve fine spectral detail needed for fill discrimination.
- domain assumption 3DGS surface geometry plus text condition is sufficient to determine impact sound; wall thickness, internal cavity shape, and wall coupling are not modeled.
- domain assumption The 80/20 split is instance-level, not object-disjoint.
- domain assumption The harmonic oscillator relation f ∝ sqrt(k/m) applies to filled containers with added mass.
- domain assumption Granular fill (rice) decays faster than liquid fill (water) and this is a stable ground-truth cue.
Cite this review
Pith. "Pith review of FillGauss: Fine-Grained Filling-Aware Impact Sound Generation for 3D Gaussian Splatting." pith.science (2026). https://pith.science/paper/23JEBDFP
@misc{pith2026260717773,
author = {Pith},
title = {Pith review of: FillGauss: Fine-Grained Filling-Aware Impact Sound Generation for 3D Gaussian Splatting},
year = {2026},
howpublished = {\url{https://pith.science/paper/23JEBDFP}},
note = {Machine review of arXiv:2607.17773}
}
read the original abstract
Synthesizing physically plausible impact sounds from visual observations remains a great challenge in multi-modal AI. Existing 3D-aware audio generation methods primarily model the surface geometry of hollow rigid bodies. However, they fundamentally overlook internal filling states, a critical physical factor that drastically modulates acoustic resonance and damping. To address this issue, we have defined a new task called Fine-Grained Filling-Aware Impact Sound Generation. As a foundational step, we first introduce the fine-grained fill-aware dataset (FillImpact), a pioneering multi-modal collection comprising over 5,000 rigorous acoustic recordings from 88 diverse real-world objects. It captures impact interactions with varying internal contents (i.e., water, rice), a continuous range of fill levels, and distinct striker materials. Furthermore, comprehensive acoustic analysis confirms that the collected data closely aligns with established physical laws governing acoustic resonance and damping, indicating its suitability for physically grounded modeling. Building on this dataset, we propose a novel generative framework (FillGauss) that integrates 3D Gaussian Splatting (3DGS) with internal state conditioning for sound generation. By fusing 3DGS geometric features, precise 3D spatial strike coordinates, and fine-grained textual physical conditions within a latent diffusion architecture, FillGauss enables position-aware, striker-aware, and filling-aware audio generation. Extensive experiments demonstrate that our approach could generate high-fidelity impact sounds that adhere to underlying physical principles, establishing a new state-of-the-art for physically grounded cross-modal audio generation.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
That sounds right: Auditory self-supervision for dynamic robot manipulation,
A. Thankaraj and L. Pinto, “That sounds right: Auditory self-supervision for dynamic robot manipulation,” inConference on Robot Learning. PMLR, 2023, pp. 1036–1049
2023
-
[2]
Resound: interactive sound rendering for dynamic virtual environments,
M. T. Taylor, A. Chandak, L. Antani, and D. Manocha, “Resound: interactive sound rendering for dynamic virtual environments,” inPro- ceedings of the 17th ACM international conference on Multimedia, 2009, pp. 271–280. 9
2009
-
[3]
Visually indicated sounds,
A. Owens, P. Isola, J. McDermott, A. Torralba, E. H. Adelson, and W. T. Freeman, “Visually indicated sounds,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2405– 2413
2016
-
[4]
Modal synthesis for vibrating objects,
K. Van Den Doel, “Modal synthesis for vibrating objects,” 2003
2003
-
[5]
Example-guided physically based modal sound synthesis,
Z. Ren, H. Yeh, and M. C. Lin, “Example-guided physically based modal sound synthesis,”ACM Transactions on Graphics (TOG), vol. 32, no. 1, pp. 1–16, 2013
2013
-
[6]
A dynamic analysis method of liquid-filled containers considering the fluid–structure interaction,
X. Fang, X. Bao, F. Yue, and Q. Zhao, “A dynamic analysis method of liquid-filled containers considering the fluid–structure interaction,” Applied Sciences, vol. 14, no. 7, p. 2688, 2024
2024
-
[7]
Dynamic analysis method of liquid-filled containers considering the fluid-structure interaction effect,
X. bing Fang, X. Bao, F. Yue, and Q. Zhao, “Dynamic analysis method of liquid-filled containers considering the fluid-structure interaction effect,” 2024
2024
-
[8]
Visual to sound: Generating natural sound for videos in the wild,
Y . Zhou, Z. Wang, C. Fang, T. Bui, and T. L. Berg, “Visual to sound: Generating natural sound for videos in the wild,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 3550–3558
2018
Show all 47 references
-
[9]
Action2sound: Ambient-aware generation of action sounds from egocentric videos,
C. Chen, P. Peng, A. Baid, Z. Xue, W.-N. Hsu, D. Harwath, and K. Grau- man, “Action2sound: Ambient-aware generation of action sounds from egocentric videos,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 277–295
2024
-
[10]
Sonicsense: Object perception from in-hand acoustic vibration,
J. Liu and B. Chen, “Sonicsense: Object perception from in-hand acoustic vibration,”arXiv preprint arXiv:2406.17932, 2024
2024 arXiv
-
[11]
Physics- driven diffusion models for impact sound synthesis from videos,
K. Su, K. Qian, E. Shlizerman, A. Torralba, and C. Gan, “Physics- driven diffusion models for impact sound synthesis from videos,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 9749–9759
2023
-
[12]
Diffsound: Discrete diffusion model for text-to-sound generation,
D. Yang, J. Yu, H. Wang, W. Wang, C. Weng, Y . Zou, and D. Yu, “Diffsound: Discrete diffusion model for text-to-sound generation,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1720–1733, 2023
2023
-
[13]
Learning neural acoustic fields,
A. Luo, Y . Du, M. Tarr, J. Tenenbaum, A. Torralba, and C. Gan, “Learning neural acoustic fields,”Advances in Neural Information Processing Systems, vol. 35, pp. 3165–3177, 2022
2022
-
[14]
Neraf: 3d scene infused neural radiance and acoustic fields,
A. Brunetto, S. Hornauer, and F. Moutarde, “Neraf: 3d scene infused neural radiance and acoustic fields,”arXiv preprint arXiv:2405.18213, 2024
2024
-
[15]
Soundspaces: Audio-visual navigation in 3d environments,
C. Chen and U. Jain, “Soundspaces: Audio-visual navigation in 3d environments,” inECCV, 2020
2020
-
[16]
Neural acoustic fields,
X. Li, “Neural acoustic fields,”ACM Transactions on Graphics, 2022
2022
-
[17]
3d gaussian splatting for real-time radiance field rendering,
B. Kerbl, G. Kopanas, and T. Leimk ¨uhler, “3d gaussian splatting for real-time radiance field rendering,” inSIGGRAPH, 2023
2023
-
[18]
Sonicgauss: 3d gaussian-based spatial sound generation,
X. Wang, “Sonicgauss: 3d gaussian-based spatial sound generation,” arXiv preprint, 2025
2025
-
[19]
Tangoflux: Diffusion-based text-to-audio generation,
Y . Hung, “Tangoflux: Diffusion-based text-to-audio generation,”arXiv preprint, 2024
2024
-
[20]
The theory of sound,
J. W. S. Rayleigh, “The theory of sound,” 1896
-
[21]
Dover, 1945
——,The Theory of Sound. Dover, 1945
1945
-
[22]
L. E. Kinsler,Fundamentals of Acoustics. Wiley, 2000
2000
-
[23]
The use of helmholtz resonance for measuring the volume of liquids and solids,
E. S. Webster and C. E. Davies, “The use of helmholtz resonance for measuring the volume of liquids and solids,”Sensors, vol. 10, no. 12, pp. 10 663–10 672, 2010
2010
-
[24]
N. H. Fletcher and T. D. Rossing,The Physics of Musical Instruments. Springer, 1998
1998
-
[25]
Granular solids, liquids, and gases,
H. M. Jaeger and S. R. Nagel, “Granular solids, liquids, and gases,” Reviews of Modern Physics, 1996
1996
-
[26]
Granular packings: nonlinear elasticity and sound prop- agation,
H. A. Makse, “Granular packings: nonlinear elasticity and sound prop- agation,”Physical Review Letters, 2004
2004
-
[27]
Sound-20k dataset,
Y . Zhou, “Sound-20k dataset,” inCVPR, 2018
2018
-
[28]
Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations,
R. Gao, “Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations,” inCoRL, 2021
2021
-
[29]
Objectfolder 2.0,
——, “Objectfolder 2.0,”IJRR, 2022
2022
-
[30]
Realimpact: A dataset for spatial impact sound fields,
X. Li, “Realimpact: A dataset for spatial impact sound fields,”IEEE Transactions on Multimedia, 2023
2023
-
[31]
Hunyuan3d: A large-scale 3d generation model,
L. et al., “Hunyuan3d: A large-scale 3d generation model,” 2024
2024
-
[32]
glTF 2.0 Specification,
Khronos 3D Formats Working Group, “glTF 2.0 Specification,” https: //github.com/KhronosGroup/glTF, 2021, accessed: 2026-03-14
2021
-
[33]
Point transformer,
H. Zhao, L. Jiang, J. Jia, P. H. Torr, and V . Koltun, “Point transformer,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 16 259–16 268
2021
-
[34]
Splatformer: Point transformer for robust 3d gaussian splatting,
Y . Chen, M. Mihajlovic, X. Chen, Y . Wang, S. Prokudin, and S. Tang, “Splatformer: Point transformer for robust 3d gaussian splatting,”arXiv preprint arXiv:2411.06390, 2024
2024 arXiv
-
[35]
The physics of granular materials,
H. M. Jaeger and S. R. Nagel, “The physics of granular materials,” Science, vol. 255, no. 5051, pp. 1523–1531, 1992
1992
-
[36]
Stable audio open,
Z. Evans, J. D. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable audio open,” inICASSP 2025-2025 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5
2025
-
[37]
Fr ´echet audio distance: A reference-free metric for evaluating music enhancement algorithms,
K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi, “Fr ´echet audio distance: A reference-free metric for evaluating music enhancement algorithms,” inInterspeech, 2019
2019
-
[38]
Cnn architectures for large-scale audio classification,
S. Hershey, S. Chaudhuri, D. P. W. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, and M. Slaney, “Cnn architectures for large-scale audio classification,” inICASSP, 2017
2017
-
[39]
On information and sufficiency,
S. Kullback and R. A. Leibler, “On information and sufficiency,”Annals of Mathematical Statistics, vol. 22, no. 1, pp. 79–86, 1951
1951
-
[40]
Improved techniques for training gans,
T. Salimans, I. Goodfellow, W. Zaremba, V . Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” inNeurIPS, 2016
2016
-
[41]
Recommendation p.800: Methods for subjective determination of transmission quality,
ITU-T, “Recommendation p.800: Methods for subjective determination of transmission quality,” 1996
1996
-
[42]
Crowdsourcing for speech quality evaluation,
F. Ribeiro, D. Florencio, C. Zhang, and M. Seltzer, “Crowdsourcing for speech quality evaluation,” inICASSP, 2011
2011
-
[43]
Decoupled weight decay regularization,
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” International Conference on Learning Representations (ICLR), 2019
2019
-
[44]
Scaling instruction-finetuned language models,
H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y . Tay, W. Fedus, Y . Li, X. Wang, M. Dehghani, S. Brahmaet al., “Scaling instruction-finetuned language models,”arXiv preprint arXiv:2210.11416, 2022
2022 arXiv
-
[45]
Exploring the limits of transfer learning with a unified text-to-text transformer,
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,”Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020. 10 APPENDIX A. Data Proc...
2020
-
[46]
SonicGauss Fine-Tuning:The SonicGauss framework is natively designed to synthesize sound conditioned on 3D Gaussian Splatting representations and spatial impact coor- dinates. Consequently, we directly utilize the 3DGS models, precise strike positions, and the corresponding gr...
-
[47]
To leverage our entire dataset for fine-tuning TangoFlux, we systematically transform all multimodal physical parameters into rich textual prompts
TangoFlux Fine-Tuning:Unlike models utilizing explicit 3D visual conditions, TangoFlux operates strictly as a text-to- audio generation framework. To leverage our entire dataset for fine-tuning TangoFlux, we systematically transform all multimodal physical parameters into rich...
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.