REVIEW 5 cited by
Random Weight Factorization Improves the Training of Continuous Neural Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Continuous neural representations have recently emerged as a powerful and flexible alternative to classical discretized representations of signals. However, training them to capture fine details in multi-scale signals is difficult and computationally expensive. Here we propose random weight factorization as a simple drop-in replacement for parameterizing and initializing conventional linear layers in coordinate-based multi-layer perceptrons (MLPs) that significantly accelerates and improves their training. We show how this factorization alters the underlying loss landscape and effectively enables each neuron in the network to learn using its own self-adaptive learning rate. This not only helps with mitigating spectral bias, but also allows networks to quickly recover from poor initializations and reach better local minima. We demonstrate how random weight factorization can be leveraged to improve the training of neural representations on a variety of tasks, including image regression, shape representation, computed tomography, inverse rendering, solving partial differential equations, and learning operators between function spaces.
Forward citations
Cited by 5 Pith papers
-
Learning Deformable Body Interactions With Adaptive Spatial Tokenization
A spatial-tokenization transformer predicts deformable-body interactions accurately and scales to meshes beyond 100,000 nodes where graph baselines run out of memory.
-
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries
Factorizing weights into D≥2 multiplicative factors and applying L2 weight decay induces a non-convex sparse L2/D penalty, and with tailored initialization and learning rates, achieves superior sparsity-accuracy tradeoffs.
-
Physics-informed neural networks for solving moving interface flow problems using the level set approach
A PirateNet-based physics-informed neural network solves level set interface transport benchmarks to low L2 error without upwind stabilization, though the 'state-of-the-art' claim is tied to in-sample hyperparameter tuning.
-
Fragment size density estimator for shrinkage-induced fracture based on a physics-informed neural network
A PINN surrogate, seeded with a generalized-gamma ansatz, maps fragmentation-model parameters (alpha,gamma) directly to the scaled fragment-size density and is shown to beat a finite-difference solver in speed while m...
-
Physics-informed machine learning surrogate for scalable simulation of thermal histories during wire-arc directed energy deposition
A data-free physics-informed neural network matches finite-element thermal histories of wire-arc directed energy deposition to about 7% relative L2 error, reporting up to 98.6% compute-time reduction versus a fine-mes...
Discussion (0). Continue with ORCID to comment.