GS-SCNet unifies 3D Gaussian Splatting with a disparity-guided semantic codec and direct Gaussian parameter prediction for efficient real-time 3D video communications with strong generalization.
hub
End-to-end optimized image compression
17 Pith papers cite this work. Polarity classification is still indexing.
17
Pith papers citing it
abstract
We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear filters and nonlinear activation functions. Unlike most convolutional neural networks, the joint nonlinearity is chosen to implement a form of local gain control, inspired by those used to model biological neurons. Using a variant of stochastic gradient descent, we jointly optimize the entire model for rate-distortion performance over a database of training images, introducing a continuous proxy for the discontinuous loss function arising from the quantizer. Under certain conditions, the relaxed loss function may be interpreted as the log likelihood of a generative model, as implemented by a variational autoencoder. Unlike these models, however, the compression model must operate at any given point along the rate-distortion curve, as specified by a trade-off parameter. Across an independent set of test images, we find that the optimized method generally exhibits better rate-distortion performance than the standard JPEG and JPEG 2000 compression methods. More importantly, we observe a dramatic improvement in visual quality for all images at all bit rates, which is supported by objective quality estimates using MS-SSIM.
hub tools
citation-role summary
background 1
method 1
citation-polarity summary
representative citing papers
SAD is a new explicit differentiable image representation based on soft anisotropic additively weighted Voronoi partitions that achieves higher PSNR and 4-19x faster training than Image-GS and Instant-NGP at matched bitrate.
NDGI compresses temporal lightmaps via neural feature maps and lightweight networks, delivering high-quality dynamic global illumination with low storage and modest real-time decompression cost.
Finite scalar quantization simplifies VQ-VAE latents by independently rounding a few dimensions to fixed levels, producing an equivalent-sized implicit codebook with competitive performance and no collapse.
A Mamba-based semantic video communication system with CSI-guided adaptive encoding and packet loss recovery achieves PSNR > 21 dB at -8 dB SNR and 90% packet loss in AWGN channels.
MambaRaw uses SSM-based context modeling with TileMambaBlock and EAR modules for efficient JPEG-guided 4K raw reconstruction, reporting 1.2-1.4 dB PSNR gains and 9% lower latency over baselines on Sony, Olympus, and Samsung datasets.
MoECodec replaces FFN layers with token-wise MoE plus stable routing and GShMLP experts to support multiple downstream tasks in a single image compression model.
ECC integrates hyperprior side information, channel-wise context, latent residual prediction, temporal modeling, and entropy skip into a learned entropy model, yielding 39.9% and 76.3% average BD-rate reductions on ViSQOL and PESQ over baselines.
Few-step generative models can be reformulated as lossy codecs in the reverse channel coding framework without retraining, yielding faster encoding/decoding on low-resolution image benchmarks.
RDVQ enables joint rate-distortion optimization for vector-quantized generative image compression via differentiable codebook distribution relaxation and an autoregressive entropy model.
FrequencyFormer co-designs a multi-scale DCT tokenizer, LUT-based near-sensor hardware, and modified MIPI communication to enable frequency-domain ViT inference with up to 128x data reduction and 230x lower communication energy.
Derives optimality constraints for nonnegative joint dictionary learning that explain observed SAE behaviors such as feature splitting, absorption, and dense antipodal features.
A practical learned image codec delivers 2.3-3x bitrate savings over AV1/VVC and 20-40% over prior learned codecs while encoding 12MP images in 230ms on iPhone.
SAMIC introduces semantic-aware Mamba blocks and SVD-based redundancy reduction to achieve efficient perceptual image compression with improved rate-distortion-perception tradeoffs.
ActDiff-VC partitions video into segments, transmits adaptive keyframes and budget-aware point trajectories, and reconstructs frames via conditional diffusion, reporting up to 64.6% bitrate reduction at matched NIQE on UVG and MCL-JCV.
A bilinear CNN that fuses features from a distortion-type classifier and an image classifier achieves superior BIQA performance on both synthetic and authentic distortion databases.
DinoLink uses saliency-aware token pruning plus residual vector quantization to cut V2X bitrate by 139x while reporting 32.8% mAP on nuScenes.
citing papers explorer
-
Generalizable 3D Gaussian Splatting enabled Semantic Coding for Real-Time Immersive Video Communications
GS-SCNet unifies 3D Gaussian Splatting with a disparity-guided semantic codec and direct Gaussian parameter prediction for efficient real-time 3D video communications with strong generalization.
-
Soft Anisotropic Diagrams for Differentiable Image Representation
SAD is a new explicit differentiable image representation based on soft anisotropic additively weighted Voronoi partitions that achieves higher PSNR and 4-19x faster training than Image-GS and Instant-NGP at matched bitrate.
-
Neural Dynamic GI: Random-Access Neural Compression for Temporal Lightmaps in Dynamic Lighting Environments
NDGI compresses temporal lightmaps via neural feature maps and lightweight networks, delivering high-quality dynamic global illumination with low storage and modest real-time decompression cost.
-
Finite Scalar Quantization: VQ-VAE Made Simple
Finite scalar quantization simplifies VQ-VAE latents by independently rounding a few dimensions to fixed levels, producing an equivalent-sized implicit codebook with competitive performance and no collapse.
-
-8 dB SNR + 90% Packet Loss: MamVSC -- CSI-Guided Semantic Mamba for Extreme-Robust Video Semantic Communication
A Mamba-based semantic video communication system with CSI-guided adaptive encoding and packet loss recovery achieves PSNR > 21 dB at -8 dB SNR and 90% packet loss in AWGN channels.
-
MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction
MambaRaw uses SSM-based context modeling with TileMambaBlock and EAR modules for efficient JPEG-guided 4K raw reconstruction, reporting 1.2-1.4 dB PSNR gains and 9% lower latency over baselines on Sony, Olympus, and Samsung datasets.
-
MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts
MoECodec replaces FFN layers with token-wise MoE plus stable routing and GShMLP experts to support multiple downstream tasks in a single image compression model.
-
Benchmarking Neural Speech Compression from a Rate-Distortion Perspective
ECC integrates hyperprior side information, channel-wise context, latent residual prediction, temporal modeling, and entropy skip into a learned entropy model, yielding 39.9% and 76.3% average BD-rate reductions on ViSQOL and PESQ over baselines.
-
Few-step Generative Models as Lossy Compression
Few-step generative models can be reformulated as lossy codecs in the reverse channel coding framework without retraining, yielding faster encoding/decoding on low-resolution image benchmarks.
-
Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression
RDVQ enables joint rate-distortion optimization for vector-quantized generative image compression via differentiable codebook distribution relaxation and an autoregressive entropy model.
-
FrequencyFormer: A Co-Designed Sensor-to-Processor Pipeline for Frequency-Domain Vision Transformer Inference
FrequencyFormer co-designs a multi-scale DCT tokenizer, LUT-based near-sensor hardware, and modified MIPI communication to enable frequency-domain ViT inference with up to 128x data reduction and 230x lower communication energy.
-
How Optimality Structures Sparse Dictionaries: A Theory for Understanding SAE Representations
Derives optimality constraints for nonnegative joint dictionary learning that explain observed SAE behaviors such as feature splitting, absorption, and dense antipodal features.
-
What Matters in Practical Learned Image Compression
A practical learned image codec delivers 2.3-3x bitrate savings over AV1/VVC and 20-40% over prior learned codecs while encoding 12MP images in 230ms on iPhone.
-
SAMIC: A Lightweight Semantic-Aware Mamba for Efficient Perceptual Image Compression
SAMIC introduces semantic-aware Mamba blocks and SVD-based redundancy reduction to achieve efficient perceptual image compression with improved rate-distortion-perception tradeoffs.
-
Active Sampling for Ultra-Low-Bit-Rate Video Compression via Conditional Controlled Diffusion
ActDiff-VC partitions video into segments, transmits adaptive keyframes and budget-aware point trajectories, and reconstructs frames via conditional diffusion, reporting up to 64.6% bitrate reduction at matched NIQE on UVG and MCL-JCV.
-
Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network
A bilinear CNN that fuses features from a distortion-type classifier and an image classifier achieves superior BIQA performance on both synthetic and authentic distortion databases.
-
DinoLink: A Token-Centric Representation Compression Framework for Bandwidth-Constrained Collaborative V2X Perception
DinoLink uses saliency-aware token pruning plus residual vector quantization to cut V2X bitrate by 139x while reporting 32.8% mAP on nuScenes.