Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Neural Coding Is Not Always Semantic: Toward the Standardized Coding Workflow in Semantic Communications

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that neural feature compression is not semantic coding, and that a tokenize–reorganize–quantize pipeline can transmit images with 10–30 tokens per image better than a learned feature baseline.

desk verdict Useful conceptual framework for semantic coding, but the headline empirical claim is confounded by the generative decoder and shouldn't be taken at face value. read the letter →

arxiv 2505.18637 v3 pith:SKBKCNR2 submitted 2025-05-24 cs.IT math.IT

Hai-Long Qin , Jincheng Dai , Sixian Wang , Xiaoqi Qin , Shuo Shao , Kai Niu , Wenjun Xu , Ping Zhang This is my paper · ORCID
classification cs.ITmath.IT
keywords semanticcommunicationcodingneuraltokenizationtokenreorganizationcontextualmodelingjointsource-channelwirelessimagetransmission
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Semantic communication has leaned on deep neural networks to extract and compress features from raw data, but this paper argues that such 'neural coding' is not the same as coding semantics: it captures statistical patterns without grounding them in concepts or context. The paper proposes a standardized three-step procedure it calls semantic coding — tokenization, similarity-based reorganization, and optional quantization — that turns raw data into a compact set of semantic tokens meant to represent the meaning behind the data. The payoff, if correct, is a unified coding workflow: one general semantic representation serves both human-facing reconstruction and machine-facing recognition with only light task-specific adaptation at the receiver, and wireless transmission of just 10–30 tokens per image can beat a conventional learned feature-compression baseline. The article is an attempt to establish a working definition, not just a new network architecture, so that semantic communication can be discussed, compared, and standardized around the idea of conveying meanings rather than features.

What carries the argument

The load-bearing object is the semantic coding pipeline with its three operations. Tokenization uses an attention-based encoder to turn raw data into discrete tokens, which are the basic learnable units of representation. Reorganization is the step that makes representation compact: tokens are grouped and similar ones are matched and merged based on embedding-space similarity (implemented as bipartite soft matching with eight merges per transformer layer), reducing 196 tokens to roughly 10–30 while aiming to preserve each concept in the image. Optional quantization then maps the merged tokens to discrete codes so that one code can stand for one concept, which also aids robustness and compatibility with generative receivers. The workflow additionally decouples a 'general' semantic encoder from task-specific decoders, so the same compact representation can be conditioned by a generator for reconstruction or a discriminator for recognition.

What would settle it

Take two objects whose token embeddings are close but whose semantics differ (e.g., a wolf and a husky, or two fine-grained bird species), transmit each at 10 tokens through the same pipeline, and measure whether a receiver fine-tuned to distinguish them still can; if the merged representation cannot separate the two categories, the paper's premise that embedding-space proximity preserves semantic content fails for that input class.

Watch

Extended reading notes

Core claim

The central claim is that deep feature extraction ('neural coding') is necessary but not sufficient for semantic communication, and that the missing piece is a dedicated semantic representation stage. The paper defines semantic coding as tokenization (decomposing raw data into discrete learnable tokens), reorganization (merging nearby tokens in an embedding space, e.g., via bipartite soft matching, so that semantically related tokens collapse into one representative token), and optional quantization (mapping tokens to discrete codebook entries). In the reported case study, a pretrained visual transformer's 196 tokens are reduced to 10 or 30 tokens per image, fed through a joint source-channel coding transmitter over an AWGN channel, and reconstructed by a generative receiver; the paper reports better PSNR and FID than a neural-coding baseline across all tested channel bandwidth ratios, with a particularly large gap at low rates. The result is meant to show that compact, context-aware semantic representations, not merely compressed features, are the right unit of transmission for semantic communication.

Load-bearing premise

The whole pipeline rests on the assumption that tokens which are close together in embedding space really do mean the same thing, so merging them loses no task-relevant meaning; if similar-looking tokens can stand for different concepts, the compact representation silently drops information the receiver needs.

Editorial extensions

If this is right

  • If semantic coding works as claimed, a semantic communication system can be built on top of a pretrained attention-based tokenizer without retraining the tokenizer, since reorganization and quantization are applied to its outputs.
  • Reducing an image to 10–30 tokens means the transmitted rate is set by the number of semantic concepts, not by the resolution of the image, so low-rate regimes become the natural operating point.
  • The same general semantic representation can feed both human-centric tasks (via generative receivers) and machine-centric tasks (via discriminative receivers), with only minimal fine-tuning at the receiver side.
  • Quantized sparse codes are expected to be more robust to channel noise and adversarial perturbations than continuous feature vectors, because a few discrete codes carry the essential concepts.
  • The reported throughput and accuracy numbers suggest that merging tokens is also a cheap way to speed up inference, with reduced token counts and only a modest drop in classification accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If embedding-space proximity really tracks semantic synonymy, then the ideal number of tokens for an image should scale with the number of distinct concepts in the scene, which points to variable-rate semantic coding rather than fixed 10- or 30-token budgets.
  • The same reorganization step could be applied to other modalities (audio, video, point clouds) as long as a suitable attention-based tokenizer exists, turning semantic coding into a generic multimodal front-end for wireless transmission.
  • A direct way to stress-test the paper's thesis is to measure downstream task performance (e.g., fine-grained classification) on reconstructed images: if a 10-token representation preserves accuracy much better than a 196-token one on some tasks but not others, the semantic-preservation assumption is task-dependent.
  • The paper's comparison is against a specific neural-coding baseline; a stronger test of the claim that semantic coding is not just better compression would be to compare against an equally optimized learned codec at matched rates and distortions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper argues that general neural coding, i.e., feature extraction and compressive coding with deep networks, is not sufficient for semantic communication because it captures statistical patterns rather than semantic meaning. The authors propose 'semantic coding' as a standardized workflow consisting of tokenization, reorganization, and optional quantization, and they position it as a pre-processing module before compressive and transmissive coding. They outline a transmitter-side general representation and receiver-side task-specific decoding, and they report a case study on wireless image transmission (Section III.D) in which a ViT/ToMe tokenization pipeline with 10-30 tokens per image and a Stable Diffusion decoder is claimed to outperform a deep JSCC baseline across all tested CBR regimes.

Significance. The conceptual contribution is a useful clarification: separating entropy-oriented feature compression from context-aware semantic representation could help structure future semantic communication systems. The paper explicitly marks the reorganization mechanism as a hypothesis, which is honest, and it provides a concrete albeit limited demonstration. If the framework and the case study were rigorously validated, the paper would provide a valuable reference architecture for the field. However, the current empirical evidence is not sufficient to establish the central claim that this token-based pipeline outperforms neural coding, because the comparison is confounded and under-specified.

major comments (4)
  1. [Section III.D, Fig. 5] The comparison between the proposed semantic-coding system and the neural-coding baseline is confounded by the decoder design. The semantic pipeline uses a Stable Diffusion generator for content recovery, while the baseline [14] is a non-generative JSCC decoder, so any improvement in FID or perceived realism could be attributed to the generative prior rather than to semantic tokenization and reorganization. The paper does not provide a control experiment using the same generator on neural-coding features or a non-generative decoder on semantic tokens. Without such matched decoders, the reported 'performs better across all CBR regimes' does not isolate the contribution of semantic coding.
  2. [Section III.D, Fig. 5] The rate and bit-budget matching between the two systems is not specified. The paper does not state how 10 or 30 tokens are quantized (if at all), how token vectors are dimensioned, or how many channel symbols each CBR value corresponds to for each system. Since CBR is defined as the ratio of transmitted signal dimension k to original dimension m (footnote 4), the claim that semantic coding is better 'across all CBR regimes' is only meaningful if both systems spend the same number of channel symbols per source pixel; the manuscript provides no evidence for this.
  3. [Section II.B.3] The load-bearing assumption that 'proximate token embeddings maintain high contextual relevance and typically represent similar meanings or concepts' is presented as a hypothesis ('It is hypothesized that token reorganization works due to its concise contextual representation in concept categorization'). The entire semantic coding pipeline depends on this assumption, but the case study provides no direct validation for downstream tasks, no ablation showing that similarity-based merging preserves task-relevant semantics, and no analysis of when it fails. This needs either a formal argument or substantially more empirical support.
  4. [Section III.D] The experimental reporting is incomplete: the dataset is not named ('large-scale image datasets'), no error bars, standard deviations, or multiple random seeds are reported, and the PSNR/FID values are not presented numerically for each CBR level. The visual comparison is shown for a single rate (0.0417 cpp). As a result, the claimed advantage cannot be assessed for statistical significance or reproducibility.
minor comments (4)
  1. [Section III.D] The throughput claim (524 images/s, Top-1 74.06%) appears to be on a classification task, but the dataset and hardware are not specified, and the comparison is to vanilla ViT-B/16 rather than to the neural-coding baseline. Please clarify.
  2. [Section III.D] The sentence 'The overall JSCC framework for wireless transmission is trained end-to-end on large-scale image datasets' is ambiguous: it should state whether the JSCC encoder and decoder are trained on semantic tokens or on raw images, and how the off-the-shelf ViT/ToMe tokenizer is integrated with a trainable framework.
  3. [Section II.B.3] The example of reducing 196 tokens to 10-30 tokens is illustrative, but it would be helpful to state how the number of tokens is chosen per image and whether it is adaptive or fixed.
  4. [Section III.B and Table I] For reproducibility, it would be helpful to cite the exact implementations of the metrics and the generator; the current references are to surveys and to the original model papers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a framework proposal with a stipulative definition, an empirical case study, and only background-level self-citations.

full rationale

The paper is a framework/tutorial-style contribution rather than a derivation chain. Semantic coding is explicitly introduced as a 'standardized definition' (Section I.B) with an operational recipe of tokenization, reorganization, and optional quantization, and the key semantic assumption is flagged as a hypothesis: 'It is hypothesized that token reorganization works due to its concise contextual representation in concept categorization' (Section II.B.3). No parameter is fitted to a subset of data and then renamed as a prediction; the Fig. 5 claim is an empirical comparison, not a quantity derived from earlier equations. The comparison may be confounded (Stable Diffusion generator on the semantic arm vs. a non-generative JSCC decoder on the neural-coding arm), but that is an experimental-validity concern, not a circular reduction. Self-citations [7], [11], [13] are used for quantization forms, nonlinear-transform JSCC, and knowledge-base priors; they are supporting modules, and the paper's central conceptual claim does not reduce to them. The 'semantic' label is stipulated rather than imported from an author-uniqueness theorem, so the conclusion is not equivalent to its inputs by construction.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The framework rests on several domain assumptions about tokenization and semantic similarity that are asserted rather than proved. The case study uses hand-chosen hyperparameters and no formal guarantees. There are no physically invented entities; the abstraction of 'general semantic representation' is a conceptual construct without independent experimental evidence.

free parameters (4)
  • Number of semantic tokens = 30 or 10
    Chosen by hand for the case study to compress 224x224 images. Justified by reference to [6] rather than optimized in this paper.
  • Matching ratio r (number of tokens merged per layer) = 8
    Adopted from ToMe [5] with no tuning described. Controls the aggressiveness of token consolidation.
  • CBR rate levels = [0.0208, 0.0417, 0.0625, 0.0833, 0.125]
    Discrete points at which performance is evaluated, chosen by the authors for the rate-sweep.
  • SNR = 1 dB
    Fixed channel condition for all experiments, chosen without reported justification.
assumptions (5)
  • domain assumption Semantic meaning of data can be captured by discrete tokens at variable granularities.
    Assumed in Section II.B.1: 'Token is the basic learnable element for data representation... capturing semantic meaning at variable granularities.'
  • domain assumption Similarity in token embedding space corresponds to semantic relatedness, so merging nearby tokens preserves semantics.
    Assumed in Section II.B.3: 'proximate token embeddings maintain high contextual relevance and typically represent similar meanings or concepts.'
  • domain assumption A task-independent general semantic representation exists and can be extracted before task-specific decoding.
    Core premise stated in Sections I.B and V.A: decouple general semantic encoding from user-specific interpretation.
  • domain assumption Attention mechanisms are a necessary condition for flexible tokenization.
    Section II.B.2 states: 'incorporating attention mechanisms (a necessary condition, e.g., multi-head self-attention) into tokenization'.
  • domain assumption Frozen pre-trained ViT features contain sufficient semantic structure for token merging to retain task-relevant information.
    The case study uses a vanilla ViT-B/16 with ToMe and no additional training, assuming the features are semantically meaningful enough for the communication task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Coding Is Not Always Semantic: Toward the Standardized Coding Workflow in Semantic Communications." pith.science (2026). https://pith.science/paper/SKBKCNR2

@misc{pith2026250518637,
  author       = {Pith},
  title        = {Pith review of: Neural Coding Is Not Always Semantic: Toward the Standardized Coding Workflow in Semantic Communications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SKBKCNR2}},
  note         = {Machine review of arXiv:2505.18637}
}
read the original abstract

Semantic communication, leveraging advanced deep learning techniques, emerges as a new paradigm that meets the requirements of next-generation wireless networks. However, current semantic communication systems, which employ neural coding for feature extraction from raw data, have not adequately addressed the fundamental question: Is general feature extraction through deep neural networks sufficient for understanding semantic meaning within raw data in semantic communication? This article is thus motivated to clarify two critical aspects: semantic understanding and general semantic representation. This article presents a standardized definition on semantic coding, an extensive neural coding scheme for general semantic representation that clearly represents underlying data semantics based on contextual modeling. With these general semantic representations obtained, both human- and machine-centric end-to-end data transmission can be achieved through only minimal specialized modifications, such as fine-tuning and regularization. This article contributes to establishing a commonsense that semantic communication extends far beyond mere feature transmission, focusing instead on conveying compact semantic representations through context-aware coding schemes.

Figures

Figures reproduced from arXiv: 2505.18637 by the authors.

Figure 1
Figure 1. Overview of the coding workflow in semantic communication systems. Notations: Solid single arrows represent data flow, while dashed double arrows represent evaluation. T and G denote semantic tokenizer and detokenizer; A and S represent analysis and synthesis transforms; E and D indicate transmissive encoding and decoding with modulation and demodulation. W represents the wireless channel, Q denotes quantization, an… view at source ↗
Figure 2
Figure 2. Semantic coding for compact representation (towards dimensionality reduction) vs. Compressive coding for feature compression (towards entropy optimization). Notations: dim(·) specify data dimensionality, and H(·) define information entropy. Bold lowercase letters with tildes denote quantized vectors (e.g., z˜). The dashed box represents optional operations. Compressive coding implements data compression through thre… view at source ↗
Figure 3
Figure 3. Illustration of extracting general semantic representations. Raw data in source space is transformed into tokens via tokenization, and these obtained tokens in token embedding space are then reorganized into compact semantic tokens constituting the so-called semantic latent space through similarity-based consolidation. In this process, similar tokens are aggregated together, indicating high semantic relevance. Optio… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The technical workflow of semantic coding enabled semantic communication. Notations: JSCE and JSCD denote joint source-channel encoding and decoding respectively. They constitute joint source-channel coding as an implementation of transmissive coding for real-time high…
Figure 5
Figure 5. Figure 5: Results for end-to-end wireless image transmission with semantic coding. “Recon.”, “Neu. Cod.”, “Sem. Cod.” and “w/” denote “Reconstructions”, “Neural Coding”, “Semantic Coding” and “with” respectively. A higher PSNR or lower FID score indicates better performance. Fea…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rethinking Multi-User Communication in Semantic Domain: Enhanced OMDMA by Shuffle-Based Orthogonalization and Diffusion Denoising

    cs.IT 2025-07 conditional novelty 6.0 of 10

    Randomly permuting each user's JSCC features turns multi-user interference into roughly Gaussian noise, letting a single pretrained encoder and diffusion denoiser serve all users.

Reference graph

Works this paper leans on

15 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [14]

    Deep joint source- channel coding for wireless image transmission,

    E. Bourtsoulatze, D. B. Kurka, and D. G ¨und¨uz, “Deep joint source- channel coding for wireless image transmission,” IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 3, pp. 567–579, 2019

    2019

  2. [1]

    Recent contributions to the mathematical theory of com- munication,

    W. Weaver, “Recent contributions to the mathematical theory of com- munication,” ETC: A Review of General Semantics , pp. 261–281, 1953

    work page 1953

  3. [2]

    Semantic communications for future internet: Fundamentals, applications, and challenges,

    W. Yang, H. Du, Z. Q. Liew, W. Y . B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Shen, and C. Miao, “Semantic communications for future internet: Fundamentals, applications, and challenges,” IEEE Commun. Surv. Tuts. , vol. 25, no. 1, pp. 213–250, 2022

    work page 2022

  4. [3]

    Nonlinear transform coding,

    J. Ball ´e, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” IEEE J. Sel. Topics Signal Process. , vol. 15, no. 2, pp. 339–353, 2020

    work page 2020

  5. [4]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2021

    2021

  6. [5]

    Token merging: Your vit but faster,

    D. Bolya, C. Y . Fu, X. Dai, P. Zhang, C. Feichtenhofer, and J. Hoffman, “Token merging: Your vit but faster,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023

    work page 2023

  7. [6]

    An image is worth 32 tokens for reconstruction and generation,

    Q. Yu, M. Weber, X. Deng, X. Shen, D. Cremers, and L. C. Chen, “An image is worth 32 tokens for reconstruction and generation,” in Proc. Int. Conf. Neural Inf. Process. Syst. (NeurIPS) , 2024

    work page 2024

  8. [7]

    Deep generative modeling reshapes compression and transmission: From efficiency to resiliency,

    J. Dai, X. Qin, S. Wang, L. Xu, K. Niu, and P. Zhang, “Deep generative modeling reshapes compression and transmission: From efficiency to resiliency,” IEEE Wireless Commun., vol. 31, no. 4, pp. 48–56, 2024

    work page 2024

Show all 15 references
  1. [8]

    Perceptual image quality assessment: a survey,

    G. Zhai and X. Min, “Perceptual image quality assessment: a survey,” Sci. China Inf. Sci. , vol. 63, pp. 1–52, 2020

    2020

  2. [9]

    Rethinking lossy compression: The rate- distortion-perception tradeoff,

    Y . Blau and T. Michaeli, “Rethinking lossy compression: The rate- distortion-perception tradeoff,” in Proc. Int. Conf. Mach. Learn. (ICML) . PMLR, 2019, pp. 675–685

    2019

  3. [10]

    Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models,

    J. Yao, B. Yang, and X. Wang, “Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2025, pp. 15703–15712

    2025

  4. [11]

    Nonlinear transform source-channel coding for semantic communications,

    J. Dai, S. Wang, K. Tan, Z. Si, X. Qin, K. Niu, and P. Zhang, “Nonlinear transform source-channel coding for semantic communications,” IEEE J. Sel. Areas Commun. , vol. 40, no. 8, pp. 2300–2316, 2022

    2022

  5. [12]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2022, pp. 10684–10695

    2022

  6. [13]

    Knowl- edge base enabled semantic communication: A generative perspective,

    J. Ren, Z. Zhang, J. Xu, G. Chen, Y . Sun, P. Zhang, and S. Cui, “Knowl- edge base enabled semantic communication: A generative perspective,” IEEE Wireless Commun., vol. 31, no. 4, pp. 14–22, 2024

    2024

  7. [15]

    A unified multi- task semantic communication system for multimodal data,

    G. Zhang, Q. Hu, Z. Qin, Y . Cai, G. Yu, and X. Tao, “A unified multi- task semantic communication system for multimodal data,” IEEE Trans. Commun., vol. 72, no. 7, pp. 4101–4116, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.