Semantic Communications for Digital Signals via Carrier Images

Dong Li; Zhigang Yan

REVIEW 5 major objections 5 minor 12 references

Semantic Communications for Digital Signals via Carrier Images

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a semantic-communication scheme that encodes a digital bit stream into the mask locations of an image and uses a pre-trained Masked Autoencoder to recover both the image and the bits, with a sparse encoding module to…

desk verdict A novel idea for digital-signal SemCom, but the missing bit-to-mask mapping and non-invertible algorithms undermine the central claim. read the letter →

arxiv 2412.07173 v2 pith:F7MNG3AL submitted 2024-12-10 eess.SP

Zhigang Yan , Dong Li This is my paper · ORCID

classification eess.SP

keywords semanticcommunicationdigitalsignaltransmissionmaskedautoencodercarrierimagesparseencodingmasktokensjointsource-channelcodingbiterrorrate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that digital signals, which carry no semantic features, can still be transmitted through a semantic communication system by using an image as a carrier. The idea is to convert the binary data into a pattern of masked patches on the image and let a pre-trained Masked Autoencoder (MAE) reconstruct both the image and, from the mask locations, the original bits. A sparse encoding module is added at the transmitter so the mask information is sent as a sparse matrix rather than fixed-length mask tokens, cutting transmission overhead. Simulations over AWGN and Rayleigh fading channels are used to argue that the approach keeps image quality and bit error rate acceptable even at high mask ratios, and that its transmission cost falls below existing image-only semantic communication schemes when the same binary data is added.

What carries the argument

The binary-to-mask mapping is the carrier mechanism: a message of $L_b$ bits selects which of the $N$ image patches are masked, and the set of masked indices is transmitted as a sparse matrix along with the latent representation of the unmasked patches. The Masked Autoencoder (MAE) — an asymmetric encoder-decoder trained to fill in masked patches from visible ones — provides the reconstruction engine, since it is robust to where the masks sit. The sparse encoding module chooses the smaller of the set of masked or unmasked indices to transmit, and a masking-plus-argsort routine at the receiver rebuilds the full mask token sequence from that sparse set.

What would settle it

Take a fixed image and a binary message, map it to mask positions by the paper's unspecified rule, transmit the sparse matrix over the exact AWGN channel used in the simulations with no extra error protection, and count how often any index in the sparse matrix is corrupted; the scheme's bit error rate cannot fall below the index error rate, so a measurement showing index errors far larger than the simulated BER would falsify the claim that the digital data survives at the reported reliability.

Watch

Extended reading notes

Core claim

The central claim is that a binary stream can be mapped to the locations of masked patches in an image, and that a pre-trained Masked Autoencoder used as a semantic encoder/decoder can recover both the original image and the digital bits from the unmasked patches' latent representations plus a sparse encoding of the mask indices. Because the MAE was designed to reconstruct images from visible patches alone, the approach needs no new joint source-channel coding structure for the digital data; the bits ride along in the mask pattern. The paper further claims that encoding the mask tokens as a sparse matrix, which is restored at the receiver, improves bit error rate over sending mask tokens directly and reduces the number of transmitted bits, with total overhead decreasing as the binary message grows longer.

Load-bearing premise

The entire digital-data recovery depends on the masked-patch indices being reconstructed at the receiver without error, yet the paper never specifies how the bits are actually mapped to mask positions nor what channel coding protects those indices, so the reported bit error rate is not tied to any concrete transmission scheme.

Editorial extensions

If this is right

If the approach works as claimed, any pre-trained MAE can serve as a semantic codec for digital data without retraining, so digital signals can be added to existing image-oriented semantic communication links at near-zero extra cost.
Longer binary messages imply higher mask ratios, and within the MAE's 10 to 80 percent mask-ratio range the transmitted bits decrease as the message grows, making the overhead curve decrease with payload size.
The sparse-matrix encoding is claimed to give lower bit error rate than transmitting mask tokens directly, implying that index-set sparsity itself is a useful source of error resilience.
Because the mask pattern carries the data, the image and the digital message share the same physical transmission, so a single channel use delivers both, which no current image-only semantic communication model does.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not state this, but the digital capacity is bounded by the number of distinct mask patterns the image admits, roughly $\log_2 \binom{N}{k}$ bits for $k$ masked patches, which sets an upper limit that could be tested against the reported overhead numbers.
The scheme is effectively a steganographic side channel: the digital payload is recoverable by anyone who knows the patch indexing and mask convention, so it may serve covert or watermarking applications, though the paper does not discuss security.
A natural extension the paper leaves implicit is to add error-correcting codes over the index set or to learn the mask generation rather than using a fixed binary mapping; either could improve the bit error rate at low SNR without changing the MAE backbone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short take: this is a real idea with load-bearing holes. The novelty—using MAE mask locations as a side channel for digital bits in SemCom—is new relative to the cited literature, and the sparse-encoding module to compress mask tokens is a reasonable attempt to reduce overhead. It deserves credit for reusing a pre-trained MAE without redesigning the codec, and the simulations do show the image path behaving sensibly under AWGN and Rayleigh channels.

But the paper as written does not support the headline claim that digital signals can be reliably transmitted without significant overhead. Three soft spots, in increasing order of severity. First, the overhead analysis in Sec. III-C is internally inconsistent: the abstract says "without significant communication overheads," while Sec. III-B admits mask-token transmission is a significant cost, and the formula (N - Lb)*Le + Lm is presented as decreasing with Lb without modeling how Lm grows with the bit string. Second, and load-bearing, the bit-to-mask mapping is never specified. Section III-A only says masked positions are "determined by the indices of '1's," with no padding rule, no length prefix, no combinatorial code. Without that, the reported BER is not tied to any concrete transmission. Third, the pseudocode in Algorithms 1 and 2 is not invertible as written. Algorithm 1 stores the minority-class indices (either 0s or 1s), but Algorithm 2 unconditionally sets mask[:, sparse_matrix] = 1. If the minority class was the zeros, the receiver reconstructs the complement of the true mask—a bitwise inversion of the digital signal. There are also undefined variables (`ids_restore`) and an off-by-one in the length recovery, so the algorithm cannot be implemented from the text. No channel-coding or error protection for the index list is described, and since a single index error propagates through the argsort, the BER curves in Fig. 4 are not reproducible.

The reader's REJECT verdict is fair. The idea is salvageable—specify an invertible binary-to-mask code, define the index transmission scheme, and either fix the overhead accounting or drop the "no overhead" wording. But as is, the central claim is unsupported.

I'd send this to a serious referee if I were an editor, because the core concept is novel enough to warrant expert guidance, but I wouldn't accept it in this form. For a reading group, it's a good case study in how a neat hack can hide a missing system. I wouldn't cite it yet.

Recommendation: reject in current form, invite resubmission after the mapping and channel scheme are made concrete and the algorithms actually invert.

Referee Report

5 major / 5 minor

Summary. The manuscript proposes a semantic-communication scheme in which a binary digital signal is mapped into the positions of masked patches of a carrier image, and a pre-trained masked autoencoder (MAE) is used to reconstruct the image while the mask pattern carries the digital payload. To reduce the overhead of sending the mask pattern, it adds a sparse encoding module that transmits the minority index set together with the latent representation of the unmasked patches. Simulations report PSNR/MS-SSIM for the image and BER for the binary data over AWGN and Rayleigh channels, and tables compare transmission bits and computation time against DeepJSCC-V, SwinJSCC, and a traditional MAE baseline.

Significance. The idea of embedding digital bits in the mask pattern of an MAE is original and potentially attractive, since MAE is designed to tolerate varying mask patterns and using a pre-trained model avoids rebuilding a semantic codec. The overhead comparison in Table I and the computation-time study in Table II are also useful as starting points. However, the central claim of reliable joint transmission without significant overhead is not established: the bit-to-mask mapping is not defined, the encoding and decoding algorithms are not inverses of each other in the high-mask-ratio regime that the paper emphasizes, and no physical-layer transmission scheme for the sparse index list is given. These are load-bearing gaps, not merely presentation issues.

major comments (5)

[Abstract; Section III.A; Section III.B] The central assertion that the scheme achieves joint transmission 'without incurring significant communication overheads' (Abstract) and 'without additional communication costs' (Section III.A) is contradicted by Section III.B, which states that the original MAE 'still carries a significant overhead in transmitting M' and introduces the sparse encoding module 'to reduce the transmission costs.' The paper cannot simultaneously claim no overhead and then acknowledge that the mask tokens are a significant overhead; this internal inconsistency is load-bearing because overhead reduction is one of the stated contributions.
[Section III.A] The bit-to-mask mapping is never specified. The text says only that 'masked patch locations are determined by the indices of "1"s in the binary sequence,' and then immediately says 'the "1"s represent unmasked patches,' which is ambiguous. It does not explain how an arbitrary binary stream of length Lb is converted into the N-bit mask (e.g., fixed-length blocks, padding, or a combinatorial code), nor how the receiver knows Lb or the total number of patches. Since the entire digital payload is carried by this mapping, the BER results in Fig. 4 cannot be evaluated or reproduced from the description.
[Section III.B, Algorithms 1 and 2] Algorithms 1 and 2 are not mutually inverse. Algorithm 1 (lines 7–11) sets sparse_matrix to the complement of the larger of the two sets ids_0 and ids_1, i.e., it transmits whichever class—masked or unmasked—is the minority. Algorithm 2 (line 5) unconditionally assigns 1 to the columns sparse_matrix, treating it as the masked set. When the masked class is the majority (the high-mask-ratio regime that the paper emphasizes), Algorithm 1 transmits the unmasked indices, and Algorithm 2 then marks the unmasked patches as masked, yielding the complement of the transmitted mask and hence a bitwise-inverted digital signal. No flag or side information distinguishes the two cases. In addition, Algorithm 1 uses the undefined variable `ids_restore` and a spurious `-1` in `len_keep`, and Algorithm 2 repeats the same `-1` in computing L, so the length recovery is incoherent.
[Section IV.A and Fig. 4] No transmission scheme is specified for the sparse index list or the latent representation. The system model in Section II describes a complex baseband channel (Eq. (2)), but the simulations give no modulation, constellation, channel coding, or error protection for the transmitted indices and latent vectors, nor how an SNR value translates into discrete index errors. This matters because the receiver's argsort operations are globally sensitive: a single erroneous index can reorder the entire mask and destroy all payload bits. As written, the BER curves in Fig. 4 are not reproducible and are not connected to the claimed physical-layer model.
[Section III.C] The overhead expression is dimensionally inconsistent and unjustified. If Le and Lm denote total numbers of bits for gen(Sv_im) and M, then multiplying Le by the number of unmasked patches is incorrect; if they are per-patch or per-token quantities, that is not stated. The subsequent simplification assumes Lb = Mr N and treats Lm as fixed, then concludes that adding more binary data reduces the total transmitted bits, but no analysis shows how Lm scales with the mask pattern or what happens when Lb exceeds N. Since overhead reduction is a headline claim, this analysis must be made precise.

minor comments (5)

[Throughout] The symbol M is used to mean both the mask tokens and the masked-patch indices; these are different objects in an MAE decoder, and the paper should disambiguate them.
[Algorithm 2] The output M in Algorithm 2 is an index array obtained by argsort, but the text calls it 'recovered mask tokens'; the paper does not explain how the MAE decoder obtains the learned mask-token embeddings at the recovered indices.
[Table I] The baseline name 'SwimJSCC' should be 'SwinJSCC' to match reference [11].
[Section IV.A] The description of the loss functions says MAE with GAN is used for the proposed framework, but no training details, model weights, or code are provided, making it hard to replicate the reported curves.
[Section III.A] The phrase 'without the additional transmission costs' in the description of Eq. (4) conflicts with the later sparse-coding discussion; the paper should consistently distinguish between the image-only overhead and the digital-signal overhead.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper explicitly encodes digital bits as mask locations and recovers bits by inverting that map, which is a modulation design rather than a hidden derivation; robustness evidence is external [9], and the only self-citation is background.

full rationale

The paper's derivation chain is self-contained against external benchmarks and does not fit any parameter to its target result. The central mechanism is an explicit code design: the digital bit stream is mapped to masked-patch indices, the MAE processes the resulting masked image, and the receiver recovers the bits by inverting that mask mapping, as stated in Section III-A ('the masked patch locations are determined by the indices of “1”s in the binary sequence' and 'the receiver reverses the index sequence of the masked patches to recover the original data'). Recovering an encoded payload from its own encoding is how modulation works, not circularity. The claimed robustness to high mask ratios is borrowed from the external MAE paper [9], and the MAE-with-GAN baseline is external [12]; neither is a self-citation. The only self-citation, Ref. [6] by co-author Z. Yan, appears in the introduction as background on federated learning and is not load-bearing. The transmission-overhead comparison is a direct accounting identity (more binary bits map to more masked patches, reducing the number of encoded unmasked patches), not a fitted result. The manuscript does have serious correctness and reproducibility gaps: the bit-to-mask mapping is unspecified, Algorithms 1 and 2 are not invertible as written, and no channel-coding scheme for the sparse index list is given, so the reported BER is not tied to a concrete transmission scheme. However, these are completeness defects rather than circularity, and no equation, table, or claim reduces to its own input by construction.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. The sparse encoding module is a software component, not an invented physical entity. The free parameters are design choices, and the axioms are standard domain assumptions plus an unstated mapping assumption.

free parameters (2)

mask ratio M_r = Varies in simulations (e.g., around 50%)
Tunable design parameter controlling the trade-off between image quality and data capacity; chosen by hand in experiments, not fitted to data.
digital signal length Lb = 100, 120, 150 bits in Table I
Simulation input specifying how many bits are embedded; not fitted, but the method's overhead and BER depend on it.

assumptions (3)

domain assumption Pre-trained MAE is robust to varying mask distributions without retraining.
The paper relies on MAE's ability to reconstruct images at mask ratios up to 80%, an empirical property cited from the MAE paper [9] and not re-derived here.
domain assumption The wireless channel is modeled as AWGN or Rayleigh fading with known noise variance.
Section II assumes Y = HX + N with N ~ CN(0, sigma^2 I), a standard model in the literature.
ad hoc to paper A binary stream of length Lb can be mapped one-to-one to a subset of mask locations without significant redundancy.
The paper never specifies this mapping, yet the overhead analysis and BER results depend on it. This is an implicit ad hoc assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Semantic Communications for Digital Signals via Carrier Images." pith.science (2026). https://pith.science/paper/F7MNG3AL

@misc{pith2026241207173,
  author       = {Pith},
  title        = {Pith review of: Semantic Communications for Digital Signals via Carrier Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F7MNG3AL}},
  note         = {Machine review of arXiv:2412.07173}
}

read the original abstract

Most of current semantic communication (SemCom) frameworks focus on the image transmission, which, however, do not address the problem on how to deliver digital signals without any semantic features. This paper proposes a novel SemCom approach to transmit digital signals by using the image as the carrier signal. Specifically, the proposed approach encodes the digital signal as a binary stream and maps it to mask locations on an image. This allows binary data to be visually represented, enabling the use of existing model, pre-trained Masked Autoencoders (MAE), which are optimized for masked image reconstruction, as the SemCom encoder and decoder. Since MAE can both process and recover masked images, this approach allows for the joint transmission of digital signals and images without incurring significant communication overheads. In addition, considering the mask tokens transmission encoded by the MAE still faces extra costs, we design a sparse encoding module at the transmitter to encode the mask tokens into a sparse matrix, and it can be recovered at the receiver. Thus, this approach simply needs to transmit the latent representations of the unmasked patches and a sparse matrix, which further reduce the transmission overhead compared with the original MAE encoder. Simulation results show that the approach maintains reliable transmission even in a high mask ratio of images.

Figures

Figures reproduced from arXiv: 2412.07173 by the authors.

**Figure 2.** The PSNR performance of image reconstruction under d [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 5.** The PSNR and BER performance of image and binary data tr [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages

[1]

Research on semantic communication based on joint control mechanism of shallow and deep neural network,

W. Xie et al. , “Research on semantic communication based on joint control mechanism of shallow and deep neural network,” Chinese Journal of Electronics , early access, doi: 10.23919/cje.2023.00.278

work page doi:10.23919/cje.2023.00.278 2023
[2]

Deep Learning Enabled Semantic Communication Sys- tems,

H. Xie et al. , “Deep Learning Enabled Semantic Communication Sys- tems,” IEEE Trans. Signal Process. , vol. 69, pp. 2663–2675, 2021

work page 2021
[3]

Selection-Based Image Generation for Semantic Com- munication Systems,

C. Liang et al. , “Selection-Based Image Generation for Semantic Com- munication Systems,” IEEE Commun. Lett. , vol. 28, no. 1, pp. 34–38, 2024

work page 2024
[4]

Explainable Artiﬁcial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges towar d responsible AI,

A. Barredo Arrieta et al. , “Explainable Artiﬁcial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges towar d responsible AI,” Information Fusion , vol. 58, pp. 82–115, 2020

work page 2020
[5]

Computation-Efﬁcient Backscatter-Blessed MEC With User Reciprocity,

B. Gu et al. , “Computation-Efﬁcient Backscatter-Blessed MEC With User Reciprocity,” IEEE Trans. V eh. Technol., vol. 73, no. 6, pp. 9026– 9031, 2024

work page 2024
[6]

Performance Analysis for Resource Constrained De- centralized Federated Learning Over Wireless Networks,

Z. Y an et al. , “Performance Analysis for Resource Constrained De- centralized Federated Learning Over Wireless Networks,” IEEE Trans. Commun., vol. 72, no. 7, pp. 4084–4100, 2024

work page 2024
[7]

Image Generation with Multimodule Semantic Feature-Aided Selection for Semantic Communications,

C. Liang et al. , “Image Generation with Multimodule Semantic Feature-Aided Selection for Semantic Communications,” arXiv preprint arXiv:2411.17428, 2024

arXiv 2024
[8]

Deep Learning for Joint Source-Channel Coding of Text,

N. Farsad et al. , “Deep Learning for Joint Source-Channel Coding of Text,” in Proc. IEEE International Conference on Acoustics, Speech a nd Signal Processing (ICASSP) , pp. 2326–2330, 2018

work page 2018

Show all 12 references

[9]

Masked Autoencoders Are Scalable Vision Learners,

K. He et al. , “Masked Autoencoders Are Scalable Vision Learners,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Re cogni- tion (CVPR) , pp. 16000–16009, June 2022

2022
[10]

Predictive and Adaptive Deep Coding for Wireless Image Transmission in Semantic Communication,

W. Zhang et al. , “Predictive and Adaptive Deep Coding for Wireless Image Transmission in Semantic Communication,” IEEE Trans. Wireless Commun., vol. 22, no. 8, pp. 5486–5501, 2023

2023
[11]

SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding,

K. Y ang et al. , “SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding,” Trans. Cognit. Commun. Networking , early access, doi: 10.1109/TCCN.2024.3424842

2024
[12]

Masked Auto-Encoders Meet Generative Adversarial Networks and Beyond,

Z. Fei et al. , “Masked Auto-Encoders Meet Generative Adversarial Networks and Beyond,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 24449–24459, June 2023

2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Research on semantic communication based on joint control mechanism of shallow and deep neural network,

W. Xie et al. , “Research on semantic communication based on joint control mechanism of shallow and deep neural network,” Chinese Journal of Electronics , early access, doi: 10.23919/cje.2023.00.278

work page doi:10.23919/cje.2023.00.278 2023

[2] [2]

Deep Learning Enabled Semantic Communication Sys- tems,

H. Xie et al. , “Deep Learning Enabled Semantic Communication Sys- tems,” IEEE Trans. Signal Process. , vol. 69, pp. 2663–2675, 2021

work page 2021

[3] [3]

Selection-Based Image Generation for Semantic Com- munication Systems,

C. Liang et al. , “Selection-Based Image Generation for Semantic Com- munication Systems,” IEEE Commun. Lett. , vol. 28, no. 1, pp. 34–38, 2024

work page 2024

[4] [4]

Explainable Artiﬁcial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges towar d responsible AI,

A. Barredo Arrieta et al. , “Explainable Artiﬁcial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges towar d responsible AI,” Information Fusion , vol. 58, pp. 82–115, 2020

work page 2020

[5] [5]

Computation-Efﬁcient Backscatter-Blessed MEC With User Reciprocity,

B. Gu et al. , “Computation-Efﬁcient Backscatter-Blessed MEC With User Reciprocity,” IEEE Trans. V eh. Technol., vol. 73, no. 6, pp. 9026– 9031, 2024

work page 2024

[6] [6]

Performance Analysis for Resource Constrained De- centralized Federated Learning Over Wireless Networks,

Z. Y an et al. , “Performance Analysis for Resource Constrained De- centralized Federated Learning Over Wireless Networks,” IEEE Trans. Commun., vol. 72, no. 7, pp. 4084–4100, 2024

work page 2024

[7] [7]

Image Generation with Multimodule Semantic Feature-Aided Selection for Semantic Communications,

C. Liang et al. , “Image Generation with Multimodule Semantic Feature-Aided Selection for Semantic Communications,” arXiv preprint arXiv:2411.17428, 2024

arXiv 2024

[8] [8]

Deep Learning for Joint Source-Channel Coding of Text,

N. Farsad et al. , “Deep Learning for Joint Source-Channel Coding of Text,” in Proc. IEEE International Conference on Acoustics, Speech a nd Signal Processing (ICASSP) , pp. 2326–2330, 2018

work page 2018