Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

FRAPPE autoencoder uses projection pursuit to sort latent channels by importance for zero-overhead variable-rate CPU image compression.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 09:17 UTC pith:DIVTAD6U

load-bearing objection FRAPPE presents a parallel projection pursuit encoder for residual prediction that claims natural channel ordering for variable-rate coding and strong CPU performance at low rates, but the abstract supplies no supporting details or ablations. the 2 major comments →

arxiv 2605.28992 v1 pith:DIVTAD6U submitted 2026-05-27 eess.IV

FRAPPE: Full Input, Residual Output Autoencoding with Projection Pursuit Encoder

Dan Jacobellis , Neeraja J. Yadwadkar This is my paper
classification eess.IV
keywords image compressionautoencoderprojection pursuitvariable-rate codingperceptual qualityreal-time encodingresidual predictionCPU codec
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces FRAPPE, an autoencoding framework in which the full input predicts the residual output through a projection pursuit encoder. This design produces an embarrassingly parallel DAG of independent projections that naturally orders latent channels by importance. The ordering supports variable-rate coding simply by selecting subsets of channels, without retraining or extra mechanisms. Applied to RGB images as FRAPPE-Image, the method reaches higher perceptual quality than AVIF near 0.1 bits per pixel while running 47 times faster on CPU, enabling real-time 1080p encoding at 30 frames per second.

Core claim

FRAPPE maps the full input to residual output via a Projection Pursuit Encoder whose objective sorts latent channels by importance. This yields zero-overhead variable-rate coding through an embarrassingly parallel analysis path, in contrast to sequential RNN-based or RVQ-style codecs. The resulting FRAPPE-Image codec demonstrates a superior rate-distortion-complexity trade-off, delivering higher perceptual quality than AVIF at high compression ratios with far lower encoding time on standard CPU hardware.

What carries the argument

The Projection Pursuit Encoder, a DAG of independent input projections that predicts residual output from full input and orders channels by importance for variable-rate selection.

Load-bearing premise

The projection pursuit encoder's objective will naturally produce latent channels sorted by importance that support effective zero-overhead variable-rate coding across bitrates without post-hoc adjustments or additional mechanisms.

What would settle it

A test in which selecting successive numbers of latent channels fails to produce a smooth, monotonically improving rate-distortion curve without any retraining would falsify the claim that the channels are ordered by importance in a way that enables zero-overhead variable-rate coding.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Variable-rate operation requires only channel selection after a single training run.
  • Encoding proceeds through independent parallel projections, supporting CPU execution at real-time rates.
  • At approximately 0.1 bpp the codec exceeds AVIF perceptual quality while using 47 times less encoding time.
  • The same framework can be applied to other media types that benefit from residual prediction.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Resource-constrained platforms such as wearables or robotics sensors could perform high-ratio compression locally without GPU hardware.
  • Cloud offloading of perception tasks becomes more practical when bandwidth is limited and encoding must occur on the device.
  • The parallel structure may combine with existing standards to create hybrid codecs that switch between learned and conventional modes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces FRAPPE, a novel autoencoding framework that employs a Projection Pursuit Encoder to predict the residual output from the full input. This design is claimed to naturally sort the latent channels by importance, enabling zero-overhead variable-rate coding through simple truncation. The authors apply this to create FRAPPE-Image, an RGB image codec, and report that at approximately 0.1 bpp it achieves higher perceptual quality than AVIF while being 47 times faster in encoding, supporting real-time 1080p 30fps on CPU only. Code and models are made available.

Significance. If the performance claims are substantiated, this work could have substantial impact on compression for edge devices and real-time applications by providing a parallel, GPU-free, bitrate-adaptive codec that improves upon current standards in the rate-distortion-complexity space. The open-sourcing of code and pre-trained models is a strength that supports reproducibility and further research.

major comments (2)
  1. [Abstract] The claim that 'FRAPPE's encoding objective naturally sorts latent channels by importance, allowing zero-overhead variable-rate coding' (abstract) is load-bearing for the variable-rate results and the 0.1 bpp performance. No formal description of the objective function or empirical validation that the ordering is stable across images (allowing truncation to achieve the reported rates without post-hoc adjustments) is supplied; if this property does not hold, the headline comparison to AVIF rests on an unverified mechanism.
  2. [Evaluation] The evaluation of FRAPPE-Image against AVIF at high compression ratios asserts superior perceptual quality and 47x speedup but supplies no details on the exact perceptual metric, test dataset, training procedure, or ablation confirming that the 0.1 bpp point is obtained solely via channel truncation (no retraining or side information). This undermines assessment of the central claim.
minor comments (2)
  1. The GitHub link for code and models is provided but lacks a specific commit hash or version tag to ensure exact reproducibility of the reported results.
  2. Notation for the projection pursuit encoder (e.g., how the residual is defined relative to the full input) could be introduced with an equation early in the manuscript for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the major comments point-by-point below and will revise the manuscript accordingly to provide the requested formal descriptions, empirical validations, and evaluation details.

read point-by-point responses

  1. Referee: [Abstract] The claim that 'FRAPPE's encoding objective naturally sorts latent channels by importance, allowing zero-overhead variable-rate coding' (abstract) is load-bearing for the variable-rate results and the 0.1 bpp performance. No formal description of the objective function or empirical validation that the ordering is stable across images (allowing truncation to achieve the reported rates without post-hoc adjustments) is supplied; if this property does not hold, the headline comparison to AVIF rests on an unverified mechanism.

    Authors: We acknowledge the need for greater rigor on this central claim. The Projection Pursuit Encoder minimizes a residual reconstruction objective that successively captures the largest remaining variance components in the input, which mathematically induces an importance ordering on the latent channels (similar to PCA but applied to the residual prediction task). We will add a formal description of this objective function, including the mathematical formulation, to Section 3. We will also add an empirical validation subsection demonstrating that the channel ordering is stable across images by reporting the per-channel contribution to reconstruction error on a held-out validation set, confirming that simple truncation achieves the target rates without post-hoc adjustments or retraining. revision: yes

  2. Referee: [Evaluation] The evaluation of FRAPPE-Image against AVIF at high compression ratios asserts superior perceptual quality and 47x speedup but supplies no details on the exact perceptual metric, test dataset, training procedure, or ablation confirming that the 0.1 bpp point is obtained solely via channel truncation (no retraining or side information). This undermines assessment of the central claim.

    Authors: We agree that these details must be explicitly stated for proper assessment. The perceptual metric used is LPIPS; the test sets are the Kodak dataset plus a 100-image subset of CLIC; training used a combination of ImageNet and DIV2K with the procedure described in Section 4; and the 0.1 bpp point is obtained solely by truncating the ordered latent channels with no retraining or side information. We will expand Section 5 to include these specifics and add an ablation table confirming the truncation-only variable-rate mechanism, including rate-distortion curves for different truncation levels. revision: yes

Circularity Check

0 steps flagged

No significant circularity in FRAPPE derivation

full rationale

The paper introduces FRAPPE as a new autoencoding framework whose projection pursuit objective is claimed to sort latent channels by importance as a direct consequence of the stated architecture (full-input residual-output via parallel independent projections). This property is presented as following from the design choices rather than being presupposed by the variable-rate coding result or fitted to it. No self-citations, self-definitional loops, fitted inputs renamed as predictions, or ansatzes imported via prior author work appear in the provided claims or abstract. The central performance claims rest on empirical evaluation against external codecs rather than reducing to internal definitions by construction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on the effectiveness of a newly proposed encoder architecture whose benefits are demonstrated empirically rather than derived from first principles; many standard DNN training assumptions are implicit.

free parameters (1)
  • projection pursuit encoder hyperparameters
    Architecture depth, projection dimensions, and training schedule are chosen to achieve the reported channel sorting and rate-distortion performance.
axioms (1)
  • domain assumption Residual prediction from the full input via independent projections is feasible and yields naturally ordered latents
    This is the core modeling choice invoked to justify zero-overhead variable-rate coding.

pith-pipeline@v0.9.1-grok · 5806 in / 1326 out tokens · 37510 ms · 2026-06-29T09:17:37.064978+00:00 · methodology

0 comments
read the original abstract

Media compression standards have reached a plateau in terms of the rate-distortion-complexity trade-off, limiting the ability to offload expensive AI perception to the cloud in applications like robotics, wearables, and remote sensing. DNN-based codecs improve compression efficiency, but at a cost: they cannot easily adapt to large changes in available bitrate, and real-time encoding requires expensive, power-hungry GPUs that prohibit use on low-cost or resource-constrained platforms. To address these limitations, we propose a novel autoencoding framework (FRAPPE) that uses the Full input to predict the Residual output via a Projection Pursuit Encoder. FRAPPE's encoding objective naturally sorts latent channels by importance, allowing zero-overhead variable-rate coding. Unlike RNN-based learned codecs, whose encoder consumes the previous reconstruction's residual, or RVQ-style codecs, whose codebooks must be applied sequentially, FRAPPE's analysis path is an embarrassingly parallel DAG of independent input projections. Using FRAPPE, we build a variable-rate RGB image codec (FRAPPE-Image), and evaluate its rate-distortion-complexity trade-off against standard image codecs. At high compression ratios (approx. 0.1 bpp) FRAPPE-Image provides higher perceptual quality than AVIF with 47 times faster encoding, making it capable of real-time 1080p, 30fps CPU-only encoding. Our code and pre-trained models are available: https://github.com/UT-SysML/FRAPPE .

Figures

Figures reproduced from arXiv: 2605.28992 by Dan Jacobellis, Neeraja J. Yadwadkar.

Figure 1
Figure 1. Figure 1: Consolidated encoder weights of FRAPPE-Image, one row per scale [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Progressive reconstructions of kodim22 as the transmitted channel count n is varied. All n panels share the same encoder weights; only the truncated channel count and matching merged-decoder snapshot differ. The bottom-right panel is the uncoded reference. Bits-per-pixel measurements are JPEG-LS-coded. CN multiply–accumulates per sample regardless of patch size—a patch of size p D s requires CpD s MACs but… view at source ↗
Figure 3
Figure 3. Figure 3: Rate-distortion and encoding-throughput comparison on Kodak for JPEG and AVIF (both via Pillow), mbt2018 (via CompressAI), WaLLoC, and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SEAOTTER: Sensor Embedded Autoencoding with One-Time Transcode for Efficient Reconstruction

    eess.IV 2026-06 unverdicted novelty 5.0

    SEAOTTER pairs a frozen sensor autoencoder with a learnable JPEG color/quantization transcode to deliver 200:1 compression, 7x faster encoding and 3.5x faster decoding than AVIF while improving ImageNet accuracy and r...

Reference graph

Works this paper leans on

33 extracted references · 2 canonical work pages · cited by 1 Pith paper · 1 internal anchor

  1. [1]

    VVC complexity and software implementation analysis,

    F. Bossen, K. S ¨uhring, A. Wieckowski, and S. Liu, “VVC complexity and software implementation analysis,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 10, pp. 3765–3778, 2021

    2021

  2. [2]

    Mcucoder: Adaptive bitrate learned video compression for IoT devices,

    A. Hojjat, J. Haberer, and O. Landsiedel, “Mcucoder: Adaptive bitrate learned video compression for IoT devices,” inDAGM German Confer- ence on Pattern Recognition. Springer, 2025, pp. 123–138

    2025

  3. [3]

    Learned compression for com- pressed learning,

    D. Jacobellis and N. J. Yadwadkar, “Learned compression for com- pressed learning,” in2025 Data Compression Conference (DCC). IEEE, 2025

    2025

  4. [4]

    LiVeAction: Lightweight, versatile, and asymmetric codec design for real-time operation,

    D. Jacobellis and N. J. Yadwadkar, “LiVeAction: Lightweight, versatile, and asymmetric codec design for real-time operation,” inIEEE Data Compression Conference (DCC), 2026, in press. [Online]. Available: https://ut-sysml.github.io/liveaction

    2026

  5. [5]

    Variable Rate Image Compression with Recurrent Neural Networks

    G. Toderici, S. M. O’Malley, S. J. Hwang, D. Vincent, D. Minnen, S. Baluja, M. Covell, and R. Sukthankar, “Variable rate image compres- sion with recurrent neural networks,”arXiv preprint arXiv:1511.06085, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  6. [6]

    Full resolution image compression with recurrent neural networks,

    G. Toderici, D. Vincent, N. Johnston, S. Jin Hwang, D. Minnen, J. Shor, and M. Covell, “Full resolution image compression with recurrent neural networks,” inProceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 5306–5314

    2017

  7. [7]

    SoundStream: An end-to-end neural audio codec,

    N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “SoundStream: An end-to-end neural audio codec,”IEEE/ACM Transac- tions on Audio, Speech, and Language Processing, vol. 30, pp. 495–507, 2022

    2022

  8. [8]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023

    2023

  9. [9]

    High- fidelity audio compression with improved RVQGAN,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High- fidelity audio compression with improved RVQGAN,” inNeurIPS, 2023

    2023

  10. [10]

    Forward stagewise additive modeling,

    T. Hastie, R. Tibshirani, and J. Friedman, “Forward stagewise additive modeling,” inThe Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., ser. Springer Series in Statistics. New York: Springer, 2009, ch. 10.2, pp. 389–392

    2009

  11. [11]

    Matching pursuits with time-frequency dictionaries,

    S. G. Mallat and Z. Zhang, “Matching pursuits with time-frequency dictionaries,”IEEE Transactions on signal processing, vol. 41, no. 12, pp. 3397–3415, 1993

    1993

  12. [12]

    Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition,

    Y . C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, “Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition,” inProceedings of 27th Asilomar conference on signals, systems and computers. IEEE, 1993, pp. 40–44

    1993

  13. [13]

    The recurrent cascade-correlation architecture,

    S. Fahlman, “The recurrent cascade-correlation architecture,”Advances in neural information processing systems, vol. 3, 1990

    1990

  14. [14]

    Greedy layer- wise training of deep networks,

    Y . Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer- wise training of deep networks,”Advances in neural information pro- cessing systems, vol. 19, 2006

    2006

  15. [15]

    The principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling features,

    F. W. Young, Y . Takane, and J. de Leeuw, “The principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling features,”Psychometrika, vol. 43, no. 2, pp. 279–281, 1978

    1978

  16. [16]

    The gifi system of descriptive multivariate analysis,

    G. Michailidis and J. De Leeuw, “The gifi system of descriptive multivariate analysis,”Statistical Science, pp. 307–336, 1998

    1998

  17. [17]

    Canonical correlation analysis: A general parametric significance-testing system

    T. R. Knapp, “Canonical correlation analysis: A general parametric significance-testing system.”Psychological Bulletin, vol. 85, no. 2, p. 410, 1978

    1978

  18. [18]

    Chapter 16: Canonical correlation analysis,

    W. K. H ¨ardle and L. Simar, “Chapter 16: Canonical correlation analysis,” inApplied multivariate statistical analysis. Springer, 2015

    2015

  19. [19]

    A fast fixed-point algorithm for independent component analysis,

    A. Hyv ¨arinen and E. Oja, “A fast fixed-point algorithm for independent component analysis,”Neural computation, vol. 9, no. 7, pp. 1483–1492, 1997

    1997

  20. [20]

    Fast and robust fixed-point algorithms for independent component analysis,

    A. Hyvarinen, “Fast and robust fixed-point algorithms for independent component analysis,”IEEE transactions on Neural Networks, vol. 10, no. 3, pp. 626–634, 1999

    1999

  21. [21]

    A projection pursuit algorithm for exploratory data analysis,

    J. Friedman and J. Tukey, “A projection pursuit algorithm for exploratory data analysis,”IEEE Transactions on Computers, vol. 23, no. 9, pp. 881– 890, 1974

    1974

  22. [22]

    Projection pursuit regression,

    T. Hastie, R. Tibshirani, and J. Friedman, “Projection pursuit regression,” inThe Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., ser. Springer Series in Statistics. New York: Springer, 2009, ch. 11.2, pp. 389–392

    2009

  23. [23]

    Projection pursuit regression,

    J. H. Friedman and W. Stuetzle, “Projection pursuit regression,”Journal of the American statistical Association, vol. 76, no. 376, pp. 817–823, 1981

    1981

  24. [24]

    A ConvNet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” inCVPR, 2022

    2022

  25. [25]

    Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video,

    K. Sharifi and A. Leon-Garcia, “Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 5, no. 1, pp. 52–56, 1995

    1995

  26. [26]

    The LOCO-I lossless image compression algorithm: Principles and standardization into JPEG- LS,

    M. J. Weinberger, G. Seroussi, and G. Sapiro, “The LOCO-I lossless image compression algorithm: Principles and standardization into JPEG- LS,”IEEE Transactions on Image Processing, vol. 9, no. 8, pp. 1309– 1324, 2000

    2000

  27. [27]

    Lsdir: A large scale dataset for image restoration,

    Y . Li, K. Zhang, J. Liang, J. Cao, C. Liu, R. Gong, Y . Zhang, H. Tang, Y . Liu, D. Demandolxet al., “Lsdir: A large scale dataset for image restoration,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1775–1787

    2023

  28. [28]

    Adan: Adaptive nesterov momentum algorithm for faster optimizing deep models,

    X. Xie, P. Zhou, H. Li, Z. Lin, and S. Yan, “Adan: Adaptive nesterov momentum algorithm for faster optimizing deep models,”IEEE Trans- actions on Pattern Analysis and Machine Intelligence, vol. 46, no. 12, pp. 9508–9520, 2024

    2024

  29. [29]

    Joint autoregressive and hierarchical priors for learned image compression,

    D. Minnen, J. Ball ´e, and G. D. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” inNeurIPS, 2018

    2018

  30. [30]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004

    2004

  31. [31]

    Image quality assessment: Unifying structure and texture similarity,

    K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Image quality assessment: Unifying structure and texture similarity,”IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 5, pp. 2567– 2581, 2020

    2020

  32. [32]

    Towards practical real-time neural video compression,

    Z. Jia, B. Li, J. Li, W. Xie, L. Qi, H. Li, and Y . Lu, “Towards practical real-time neural video compression,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 12 543–12 552

    2025

  33. [33]

    Improvements of the BD-PSNR model,

    G. Bjøntegaard, “Improvements of the BD-PSNR model,” ITU-T SG16/Q6, Document VCEG-AI11, Berlin, Germany, Tech. Rep., 2008. APPENDIXA REGIME-LOCALIZEDBJONTEGAARD-DELTAANALYSIS Tables I and II summarize the operating points of Fig. 3 from two complementary viewpoints, extended with three additional CPU-only baselines: JPEG XL (libjxl, effort=7), LiVeAction ...

    work page arXiv 2008