REVIEW 6 major objections 6 minor 1 cited by
Cross-architecture universal feature coding via distribution alignment
T0 review · 6 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that a single learned codec can compress CNN and Vision Transformer features for image classification after aligning their formats and value distributions, beating the architecture-specific baseline.
desk verdict A plausible first step on cross-architecture feature coding, but the headline claim of beating architecture-specific codecs is not yet supported by the experiments. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a two-step distribution alignment that makes heterogeneous features look alike to a downstream entropy model. Format alignment reinterprets each CNN feature map location as a token spanning all channels, matching the token structure of ViTs; value alignment then clips extremes and maps both feature families into a shared [0,1] range, with a shifted lower bound for ResNet features to push their dominant mass toward DINOv2's. This preprocessing lets an unmodified Hyperprior model with one input channel serve as the unified codec, trained on a mixed dataset whose DINOv2-to-ResNet sampling ratio is scheduled by bitrate.
What would settle it
Run the same fixed alignment constants and codec on a held-out distribution such as Places365 or on different backbone pairs (for example Swin Transformer and ConvNeXt), and compare against separately trained architecture-specific codecs; if the unified codec no longer matches or beats those baselines, the universality claim fails.
Extended reading notes
Core claim
The central claim is that cross-architecture universal feature coding is feasible: a single Hyperprior-based codec, once the two feature families are made statistically and structurally comparable, can compress ResNet-50 C5 features (2048×7×7) and DINOv2 layer-40 features (257×1536) for image classification with better rate-accuracy performance than a codec tailored to large-model features. Format alignment reshapes CNN feature maps into (H×W)×N tokens, treating each spatial location as a token whose dimensions are the channels, and leaves ViT tokens as M×L. Value alignment truncates DINOv2 features to [-5,5], ResNet features to [0,5], then normalizes both to [0,1], with ResNet's lower bound shifted to DINOv2's minimum so the two distributions occupy similar regions. Training mixes both feature types with sampling ratios that depend on the target bitrate. The paper reports consistent accuracy gains across bitrates for ResNet features and at low and high bitrates for DINOv2 features, with a small loss at medium bitrates.
Load-bearing premise
The load-bearing premise is that the hand-set alignment constants tuned on the ImageNet test distribution—truncation ranges, normalization bounds, and sampling ratios—generalize without retuning, and that treating CNN channels as equivalent to ViT token dimensions is valid.
Editorial extensions
If this is right
- A single codec can serve mixed CNN/ViT deployments, removing the need for separate architecture-specific feature codecs.
- Truncation range is an effective rate-accuracy control: wider ranges help at moderate-to-high bitrates, while narrower ranges can seriously degrade accuracy, especially for ResNet features.
- Training-data sampling ratio between feature types is a practical lever: favoring DINOv2 at high bitrates and ResNet at low bitrates improves the overall rate-accuracy trade-off.
- The same alignment recipe can in principle be carried to other feature-extraction backbones beyond ResNet-50 and DINOv2.
- Replacing two codecs with one reduces storage and deployment overhead in collaborative intelligence systems.
Reading between the lines
- The alignment constants were chosen on the same ImageNet test distribution used for evaluation; generalization to other datasets or backbones is untested, so 'universal' should be read as demonstrated for this particular pair of models and this task.
- The semantic claim that CNN channels correspond to ViT token dimensions is assumed rather than proven; a direct probe or decoding experiment could test whether spatial-token alignment preserves task-relevant information across backbones.
- A data-driven version that learns truncation bounds and normalization parameters from a small calibration set could replace the hand-set constants and likely transfer better.
- The comparison to a single baseline leaves open whether separately trained, equally tuned architecture-specific codecs would close the gap; testing that pair would isolate the benefit of unification.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces cross-architecture universal feature coding (CAUFC), proposing a single Hyperprior-based codec that compresses both CNN and ViT features. The method first aligns formats by reshaping CNN activations from N×H×W into (H×W)×N 2D tokens, then aligns value ranges through truncation and a shifted normalization designed to make ResNet features resemble DINOv2 features. The codec is trained on mixed ResNet-50 and DINOv2 features from ImageNet, with bitrate-dependent sampling ratios and λ values. Experiments on image classification report rate-accuracy comparisons against the large-model feature codec of [18] for both feature types, claiming superior overall performance. The paper also includes an ablation on truncation ranges for DINOv2 features. The central claim is that, after these alignments, one codec can outperform an architecture-specific baseline for both ResNet-50 and DINOv2 features.
Significance. The problem formulation is timely and well motivated: real systems do mix CNN and Transformer backbones, and a universal feature codec would be practically useful. The proposed alignment steps are simple and intuitively reasonable, and the paper demonstrates, at least visually, that the shifted normalization makes the two feature distributions more similar. The paper is also honest about some limitations, explicitly noting that the DINOv2 medium-bitrate result is worse than baseline and attributing part of the gain to training strategy. However, the current evidence is insufficient to support the strong claim of superiority over an architecture-specific baseline. The comparison against [18] is not a clean architecture-specific baseline for CNN features, training protocol is not held fixed, and all alignment constants are selected on the same ImageNet test distribution used for evaluation. The evaluation covers only one task, one CNN, one ViT, and one test dataset, with no error bars. With additional controlled experiments, the central idea could be valuable, but as presented the claim is not yet established.
major comments (6)
- [Sec. 3.2, 3.3, Fig. 5] The claim that the proposed method beats an architecture-specific baseline is not supported by the current comparison. For ResNet features, the baseline [18] is designed for large-model/Transformer features, not for CNN feature coding; existing CNN feature codecs require 3D inputs and are not used. Both methods are forced into the same 2D token format with shifted normalization, so the reported ResNet gains may reflect the baseline's format mismatch rather than an advantage of the proposed alignment. Please compare against a CNN-specific codec operating on the native 2048×7×7 activation tensor, or at minimum against a 2D-adapted codec trained separately on ResNet features in the same preprocessed format, with the same training schedule and termination criteria.
- [Sec. 3.3, Sec. 2.3] The attribution of the improvement to the proposed method is confounded by training protocol. The DINOv2 comparison uses the same preprocessing and Hyperprior architecture for both methods, and the paper explicitly credits the DINOv2 gains to 'optimized training strategies, particularly in terms of training data sampling and learning rate-based training termination criteria.' These are choices outside the proposed alignment pipeline (the sampling-ratio schedule is introduced in Sec. 2.3 as a training choice, not as part of format or value alignment). Please include an ablation that holds the training schedule and sampling ratio fixed and adds the alignment components one at a time, or report the baseline retrained under exactly the proposed schedule without alignment.
- [Sec. 2.2, Sec. 3.1, Sec. 3.3] The 'universal' claim is currently untested across data distributions. The truncation ranges ([-5,5] for DINOv2 and [0,5] for ResNet), the shifted-normalization lower bound (set to the DINOv2 minimum), and the λ-to-sampling-ratio schedule (1:1, 1:2, 1:3, 1:5) are all hand-chosen on the ImageNet feature statistics used for both training and evaluation. No held-out feature distribution is used to select or validate these constants. To support the central generalization claim, please evaluate the same fixed constants on additional architectures (e.g., a different ViT or CNN) or on an out-of-domain dataset such as Places365, and report whether the rate-accuracy ordering is preserved.
- [Sec. 3.3, Fig. 5] The evidence is quantitatively thin: there are no error bars, repeated runs, or integrated rate-accuracy metrics (e.g., BD-rate or area under the curve), and the DINOv2 comparison shows a medium-bitrate regime where the proposed method is marginally worse than baseline. Please report multiple training runs with standard deviations and provide a quantitative summary of the gain rather than relying on visual inspection of two curves. Also specify the size of the test set and the number of feature points per bitrate point, so that the significance of the accuracy differences can be assessed.
- [Sec. 2.1, Fig. 2] The semantic equivalence assumption underlying the proposed tokenization is load-bearing but not directly verified. The paper asserts that each CNN channel corresponds to a semantic attribute in the same way that each ViT token dimension does, and that sampling one attribute from each channel therefore preserves semantic alignment. This is plausible but not demonstrated. Please provide a quantitative or experimental check, for example by comparing the proposed channel-to-token reshaping against a flattening or a random permutation of channels in terms of downstream classification accuracy, or by measuring cross-architecture similarity between the resulting token dimensions.
- [Sec. 3.4] The ablation in Sec. 3.4 studies truncation ranges for DINOv2 features but does not isolate the contributions of the three main components (format alignment, value alignment, and the shifted-normalization lower bound) in the final system. The rate-accuracy comparison in Fig. 4 is run with 'baseline Hyperprior models' and does not control for the training schedule. Please add a full ablation table, e.g., rows for (i) no alignment, (ii) format alignment only, (iii) standard normalization only, (iv) shifted normalization, (v) truncation only, and (vi) the full method, with the training protocol fixed across rows.
minor comments (6)
- [Figures 2-7] Several figures contain garbled text, with sequences such as '/uni00000013/uni00000015...' appearing in captions and axes. Please regenerate the figures with readable labels, including axis titles for the rate-accuracy plots.
- [Sec. 2.2] The phrase 'lowerandupper' is missing a space; it should read 'lower and upper'.
- [Sec. 2.3, Sec. 3.1] It is not stated which feature type corresponds to the first and second number in the 1:1, 1:2, 1:3, and 1:5 sampling ratios. Please state explicitly whether the ratio is DINOv2:ResNet or ResNet:DINOv2.
- [Sec. 3.2] The test dataset is described only as 'the classification dataset proposed in [18]'. Please give the number of test images and the class composition, and clarify whether the original accuracies 86.4% and 100% are computed on clean images or under the same distortion conditions.
- [Sec. 2.3] The loss function L = BPFP + λ×||X − X_hat||^2 is not fully specified. Please define BPFP precisely (e.g., expected bits per feature point over the batch) and state how X and X_hat are preprocessed before the distortion term is computed.
- [References] Reference [18] is an arXiv preprint by a largely overlapping set of authors. If a peer-reviewed version exists, it should be cited; otherwise, more detail on the baseline's architecture and training should be provided so that the comparison is reproducible.
Circularity Check
No significant circularity: the rate-accuracy comparisons are empirical and not forced by the preprocessing definitions or by self-citation.
full rationale
The paper's claimed derivation chain is an empirical pipeline (Sec. 2): CNN features are reshaped to a 2D token format, both feature types are truncated and normalized, and a Hyperprior codec is trained with a mixed dataset. No equation in the paper defines the reported classification accuracy or bitrate in terms of the alignment constants, and no predicted quantity is constructed from a fitted parameter of the same quantity. The truncation bounds [-5,5] and [0,5], the shifted-normalization lower bound, and the lambda-to-sampling-ratio schedule (Secs. 2.2, 2.3, 3.1) are hand-set hyperparameters; Sec. 3.4 explicitly ablates alternative truncation ranges and reports tradeoffs, which is the behavior of an empirical choice rather than a hidden re-injection of the result. The main legitimate caveats are not circular: Sec. 3.2 compares against the authors' own prior large-model codec [18] instead of a true CNN-specific 3D codec, so the ResNet 'architecture-specific baseline' label is inaccurate; and Sec. 3.3 attributes part of the DINOv2 gain to training-data sampling and termination criteria rather than to the alignment itself. Both are baseline-fairness and attribution concerns that could invalidate the generalization of the claim, but they do not make the result equivalent to its inputs by construction. Self-citation to [18] for the benchmark and test set is standard practice; the measured rate-accuracy curves are externally falsifiable and are not implied by that citation.
Assumptions & free parameters
free parameters (5)
- DINOv2 truncation range =
[-5,5]
- ResNet truncation range =
[0,5]
- Shifted normalization lower bound =
minimum of DINOv2 features
- Sampling ratio schedule =
1:1, 1:2, 1:3, 1:5 for lambda 0.001, 0.003, 0.005, 0.01
- Lambda values =
0.001, 0.003, 0.005, 0.01
assumptions (3)
- domain assumption CNN channels correspond to semantic attributes analogous to ViT token dimensions, so reshaping (H,W,C) to (H*W,C) preserves semantic structure.
- domain assumption A single Hyperprior codec with one input/output channel can model both DINOv2 and ResNet feature distributions after alignment.
- domain assumption The test dataset from [18] is representative for measuring rate-accuracy trade-offs in cross-architecture feature coding.
Cite this review
Pith. "Pith review of Cross-architecture universal feature coding via distribution alignment." pith.science (2026). https://pith.science/paper/KTKVEJYX
@misc{pith2026250612737,
author = {Pith},
title = {Pith review of: Cross-architecture universal feature coding via distribution alignment},
year = {2026},
howpublished = {\url{https://pith.science/paper/KTKVEJYX}},
note = {Machine review of arXiv:2506.12737}
}
read the original abstract
Feature coding has become increasingly important in scenarios where semantic representations rather than raw pixels are transmitted and stored. However, most existing methods are architecture-specific, targeting either CNNs or Transformers. This design limits their applicability in real-world scenarios where features from both architectures coexist. To address this gap, we introduce a new research problem: cross-architecture universal feature coding (CAUFC), which seeks to build a unified codec that can effectively compress features from heterogeneous architectures. To tackle this challenge, we propose a two-step distribution alignment method. First, we design the format alignment method that unifies CNN and Transformer features into a consistent 2D token format. Second, we propose the feature value alignment method that harmonizes statistical distributions via truncation and normalization. As a first attempt to study CAUFC, we evaluate our method on the image classification task. Experimental results demonstrate that our method achieves superior rate-accuracy trade-offs compared to the architecture-specific baseline. This work marks an initial step toward universal feature compression across heterogeneous model architectures.
Forward citations
Cited by 1 Pith paper
-
Compressed Feature Quality Assessment: Dataset and Baselines
The first compressed feature quality assessment benchmark is released, and three standard similarity metrics are shown to correlate inconsistently with task-level semantic distortion.
Reference graph
Works this paper leans on
-
[18]
have built benchmarks and proposed codecs tailored to such features, highlighting the importance and uniqueness of large model feature coding. Despite the extensive literature on feature coding, most existing methods are architecture-specific, targeting either convolutional neural networks (CNNs) or Vision Transform- ers (ViTs). However, real-world applic...
work page Pith review arXiv 2025
-
[1]
INTRODUCTION In recent years, coding for machines has received growing at- tention, which can be categorized into two branches: pixel coding and feature coding. Pixel coding compresses and re- constructs the original pixel data for downstream tasks [1, 2, 3, 4], while feature coding [5, 6, 7, 8] focuses on compress- ing and reconstructing intermediate fea...
-
[2]
THE PROPOSED METHOD 2.1. Feature Format Alignment We begin by introducing the feature extraction mechanisms in CNNs and Transformers, as illustrated in Fig. 1. In CNNs, the entire input imageXis processed through a series of con- volutional layers. Each layer containsNlearnable kernels Ki, each of which performs a localized weighted sum over a receptive f...
-
[3]
Training Details We construct our training set by randomly selecting 10,000 images from ImageNet
EXPERIMENTS 3.1. Training Details We construct our training set by randomly selecting 10,000 images from ImageNet. These images are processed through both ResNet50 and DINOv2 to generate the original features, which are then preprocessed using our proposed truncation and normalization methods. /uni00000017/uni00000013/uni00000013 /uni00000015/uni00000013/...
-
[4]
We proposed a two-step align- ment strategy to unify heterogeneous feature representations
CONCLUSION In this paper, we presented a cross-architecture universal fea- ture coding method capable of compressing features from both CNNs and Transformers. We proposed a two-step align- ment strategy to unify heterogeneous feature representations. Experimental results demonstrate that our method achieves superior rate-accuracy performance compared to t...
-
[5]
Video feature compression for machine tasks,
Kiran Misra, Tianying Ji, Andrew Segall, and Frank Bossen, “Video feature compression for machine tasks,” inICME, 2022, pp. 1–6
work page 2022
-
[6]
Scalable image coding for humans and machines,
Hyomin Choi and Ivan V Baji´c, “Scalable image coding for humans and machines,”IEEE Transactions on Image Processing, vol. 31, pp. 2739–2754, 2022
work page 2022
-
[7]
Non-semantics suppressed mask learning for unsuper- vised video semantic compression,
Yuan Tian, Guo Lu, Guangtao Zhai, and Zhiyong Gao, “Non-semantics suppressed mask learning for unsuper- vised video semantic compression,” inICCV, 2023, pp. 13564–13576
work page 2023
Show all 24 references
-
[8]
Image coding for machine via analytics-driven appearance re- dundancy reduction,
Xuelin Shen, Haoqiao Ou, and Wenhan Yang, “Image coding for machine via analytics-driven appearance re- dundancy reduction,” inICIP. IEEE, 2024, pp. 1883– 1889
2024
-
[9]
HMFVC: a human–machine friendly video compression scheme,
Zhimeng Huang, Chuanmin Jia, Shanshe Wang, and Si- wei Ma, “HMFVC: a human–machine friendly video compression scheme,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 4, pp. 3757–3769, 2025
2025
-
[10]
Deep feature compres- sion for collaborative object detection,
Hyomin Choi and Ivan V . Baji´c, “Deep feature compres- sion for collaborative object detection,” inICIP, 2018, pp. 3743–3747
2018
-
[11]
Multi-task learning with compressible features for collaborative in- telligence,
Saeed Ranjbar Alvar and Ivan V . Baji ´c, “Multi-task learning with compressible features for collaborative in- telligence,” inICIP, 2019, pp. 1705–1709
2019
-
[12]
Bit allocation for multi-task collaborative intelligence,
Saeed Ranjbar Alvar and Ivan V . Baji´c, “Bit allocation for multi-task collaborative intelligence,” inICASSP, 2020, pp. 4342–4346
2020
-
[13]
Toward scalable image feature compression: A content-adaptive and diffusion- based approach,
Sha Guo, Zhuo Chen, Yang Zhao, Ning Zhang, Xiao- tong Li, and Lingyu Duan, “Toward scalable image feature compression: A content-adaptive and diffusion- based approach,” inACM MM, 2023, p. 1431–1442
2023
-
[14]
Call for proposals on feature compression for video coding for machines,
WG2, “Call for proposals on feature compression for video coding for machines,” vol. ISO/IEC JTC 1/SC 29/WG 2, no. N282, April 2023
2023
-
[15]
End-to-end learnable multi-scale feature com- pression for VCM,
Yeongwoong Kim, Hyewon Jeong, Janghyun Yu, Youn- hee Kim, Jooyoung Lee, Se Yoon Jeong, and Hui Yong Kim, “End-to-end learnable multi-scale feature com- pression for VCM,”IEEE Transactions on Circuits and Systems for Video Technology, pp. 1–1, 2023
2023
-
[16]
Overview of the MPEG-CDVS standard,
Ling-Yu Duan, Vijay Chandrasekhar, Jie Chen, Jie Lin, Zhe Wang, Tiejun Huang, Bernd Girod, and Wen Gao, “Overview of the MPEG-CDVS standard,”IEEE Trans- actions on Image Processing, vol. 25, no. 1, pp. 179– 194, 2016
2016
-
[17]
Learned image compression for both humans and machines via dynamic adaptation,
Lingyu Zhu, Binzhe Li, Riyu Lu, Peilin Chen, Qi Mao, Zhao Wang, Wenhan Yang, and Shiqi Wang, “Learned image compression for both humans and machines via dynamic adaptation,” inICIP. IEEE, 2024, pp. 1788– 1794
2024
-
[19]
IMOFC: identity-level metric optimized feature compression for identification tasks,
Changsheng Gao, Yiheng Jiang, Siqi Wu, Yifan Ma, Li Li, and Dong Liu, “IMOFC: identity-level metric optimized feature compression for identification tasks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 2, pp. 1855–1869, 2025
2025
-
[20]
MSFC: Deep feature compres- sion in multi-task network,
Zhicong Zhang, Mengyang Wang, Mengyao Ma, Jiahui Li, and Xiaopeng Fan, “MSFC: Deep feature compres- sion in multi-task network,” inICME, 2021, pp. 1–6
2021
-
[21]
DINOv2: Learning robust visual features without supervision,
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fer- nandez, Daniel Haziza, Francisco Massa, Alaaeldin El- Nouby, et al., “DINOv2: Learning robust visual features without supervision,”arXiv preprint arXiv:2304.07193, 2023
2023 arXiv
-
[22]
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,” arXiv preprint arXiv:2501.12948, 2025
2025 arXiv
-
[23]
Feature coding in the era of large models: Dataset, test conditions, and benchmark,
Changsheng Gao, Yifan Ma, Qiaoxi Chen, Yenan Xu, Dong Liu, and Weisi Lin, “Feature coding in the era of large models: Dataset, test conditions, and benchmark,” arXiv preprint arXiv:2412.04307, 2024
2024 arXiv
-
[24]
Variational im- age compression with a scale hyperprior,
Johannes Ball ´e, David C. Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston, “Variational im- age compression with a scale hyperprior,”ArXiv, vol. abs/1802.01436, 2018
2018 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.