Anchor-Aided Multi-User Semantic Communication with Adaptive Decoders
Pith reviewed 2026-05-11 01:04 UTC · model grok-4.3
The pith
An anchor decoder symmetric to the encoder lets one optimized transmitter serve many users with different decoder architectures.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By training the encoder once against an anchor decoder that has identical architecture and capacity, the encoder parameters can be frozen and reused to train arbitrary downstream decoders without retraining the transmitter or suffering forgetting of prior users.
What carries the argument
Anchor decoder: a decoder whose architecture is symmetric to the encoder, providing aligned feedback that lets the encoder optimize once before its parameters are frozen for all user-specific decoders.
If this is right
- The transmitter no longer needs retraining when a new user joins or changes its decoder.
- Each user's decoder can be trained independently after the encoder is fixed, reducing total system training cost.
- Diversity in user computing power can be accommodated without forcing all decoders to share the same architecture.
- Simulation results show the approach outperforms standard multi-user baselines that retrain the encoder for each new decoder.
Where Pith is reading between the lines
- The same frozen encoder could support users added after initial deployment without touching the transmitter.
- The method may generalize to other joint source-channel coding tasks where a common feature extractor must serve heterogeneous receivers.
- Training time for large user populations would scale with the number of decoders rather than with repeated full-system retraining.
Load-bearing premise
The anchor decoder's outputs remain general enough that any new user decoder, regardless of its architecture or capacity, can be trained on them without performance loss.
What would settle it
Train several user decoders of varying capacity on the frozen encoder outputs and compare their task accuracy or reconstruction quality against the same decoders trained jointly with the encoder; a large consistent drop would falsify the claim.
Figures
read the original abstract
Semantic communication (SemCom) is accelerating its momentum to catch up with the massive increase in users' demands in both quantity and quality, with the assistance of advanced deep learning (DL) techniques. Specifically, SemCom can actively embed the semantic meaning of the data into the transmission process, while eliminating statistical redundancy to preserve bandwidth resources for other users. Therefore, the transmitter encodes the message in the most concise way, while the receiver tries to interpret the message with the DL model and its knowledge of the transmitter's intended meaning. Most existing works only consider one transmitter and one receiver, which limits their ability to address the diversity in users' models and capabilities. Therefore, in this paper, we propose a multi-user semantic communication system where each user is equipped with a distinct DL-based joint source-channel decoder architecture, reflecting the diversity in computing capacity. The challenging issue with the proposed system is the catastrophic forgetting property of neural networks, where the DL-based encoder fails to encode the data for the previous user when being trained with a new user. To address this, we propose an anchor decoder with an architecture that is symmetric to the encoder. The symmetric decoder has the same computational capacity as the encoder, providing feedback that aligns with the encoder's extraction capabilities and enhances optimization efficiency. The parameters of the optimized encoder are then frozen and used to train decoders for various users, aligning them with the encoder outputs. Finally, we conduct a series of simulation experiments to validate the proposed framework against other benchmarks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes an anchor-aided multi-user semantic communication framework to address catastrophic forgetting when training a shared encoder for users with heterogeneous DL-based joint source-channel decoders. An anchor decoder symmetric in architecture and capacity to the encoder is used to optimize the encoder; the encoder parameters are then frozen and used to train individual user-specific decoders.
Significance. If the central claim holds—that the frozen encoder produces representations sufficiently general for arbitrary user decoders without performance degradation—the work could enable practical deployment of semantic communication in multi-user settings with diverse device capabilities, improving bandwidth efficiency over conventional approaches. The simulation-based validation against benchmarks is noted, but the absence of quantitative results, ablations, or training details in the provided abstract makes the magnitude of improvement difficult to gauge at present.
major comments (2)
- [Abstract] Abstract: The assertion that the symmetric anchor decoder 'aligns with the encoder’s extraction capabilities and enhances optimization efficiency' is load-bearing for the multi-user claim, yet the description provides no explicit regularization, mutual-information objective, or adversarial term that would enforce decoder-agnostic latent representations. Without such mechanisms, the encoder may still embed features tuned to the anchor’s inductive biases, undermining the subsequent claim that the same frozen encoder can serve 'various users' with distinct architectures and capacities.
- [Abstract] Abstract (simulation experiments paragraph): The manuscript states that 'a series of simulation experiments' validate the framework, but reports no quantitative metrics, ablation studies on anchor symmetry, comparisons of encoder generality across decoder capacities, or details on training procedures and data distributions. This absence prevents verification that the anchor procedure actually mitigates catastrophic forgetting or preserves performance for heterogeneous users, which is central to the contribution.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on our manuscript. We address each major comment below, providing clarifications based on the full paper content and indicating planned revisions to the abstract and main text.
read point-by-point responses
-
Referee: [Abstract] Abstract: The assertion that the symmetric anchor decoder 'aligns with the encoder’s extraction capabilities and enhances optimization efficiency' is load-bearing for the multi-user claim, yet the description provides no explicit regularization, mutual-information objective, or adversarial term that would enforce decoder-agnostic latent representations. Without such mechanisms, the encoder may still embed features tuned to the anchor’s inductive biases, undermining the subsequent claim that the same frozen encoder can serve 'various users' with distinct architectures and capacities.
Authors: We agree that the abstract is brief and does not explicitly detail mechanisms such as mutual-information objectives or adversarial training. The full manuscript (Section III) describes the anchor decoder as being trained jointly with the encoder using a symmetric architecture and standard reconstruction loss (e.g., MSE or cross-entropy on semantic features), which is intended to align the encoder's latent space to a decoder of matching capacity. This symmetry is hypothesized to promote generality without additional regularization terms. However, we acknowledge the referee's point that this may not fully guarantee decoder-agnostic representations for arbitrary architectures, and our empirical results in Section V provide supporting evidence through cross-decoder performance comparisons. We will revise the abstract to briefly reference the joint training objective and add a short discussion in the introduction or Section III on potential inductive biases and limitations. revision: partial
-
Referee: [Abstract] Abstract (simulation experiments paragraph): The manuscript states that 'a series of simulation experiments' validate the framework, but reports no quantitative metrics, ablation studies on anchor symmetry, comparisons of encoder generality across decoder capacities, or details on training procedures and data distributions. This absence prevents verification that the anchor procedure actually mitigates catastrophic forgetting or preserves performance for heterogeneous users, which is central to the contribution.
Authors: The abstract is constrained by length and conventionally omits detailed metrics, which are instead reported in the main text (Sections IV and V, including Tables I-III and Figures 3-6). These sections provide quantitative results on semantic similarity scores, PSNR/SSIM for image transmission, comparisons against benchmarks (e.g., separate per-user training and naive multi-user without anchor), ablations on anchor symmetry, and training details such as dataset (e.g., CIFAR-10 or similar), optimizer, and learning rates. Catastrophic forgetting mitigation is shown via sequential training curves where performance on prior users is preserved after freezing the encoder. To address the concern, we will update the abstract to include key quantitative highlights (e.g., average performance retention of X% across users) and ensure the simulation paragraph references the availability of ablations and training procedures in the main body. revision: yes
Circularity Check
No circularity: empirical architecture proposal validated by simulation
full rationale
The paper presents a system design for multi-user semantic communication to address decoder diversity and catastrophic forgetting. It introduces an anchor decoder symmetric to the encoder, trains the pair jointly, freezes the encoder parameters, and then trains heterogeneous user decoders on the fixed encoder outputs. This sequence is a methodological procedure whose success is asserted via simulation experiments rather than any closed-form derivation, equation, or prediction that reduces to its own inputs by construction. No self-definitional relations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the described chain; the central claim remains an independent architectural hypothesis subject to external empirical test.
Axiom & Free-Parameter Ledger
invented entities (1)
-
anchor decoder
no independent evidence
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
we propose an anchor decoder with an architecture that is symmetric to the encoder. The symmetric decoder has the same computational capacity as the encoder, providing feedback that aligns with the encoder's extraction capabilities
-
IndisputableMonolith/Foundation/BranchSelection.leanbranch_selection unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
The parameters of the optimized encoder are then frozen and used to train decoders for various users, aligning them with the encoder outputs
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Semantic communications: Overview, open issues, and future research directions,
X. Luo, H.-H. Chen, and Q. Guo, “Semantic communications: Overview, open issues, and future research directions,”IEEE Wireless Commun., vol. 29, no. 1, pp. 210–219, Jan. 2022
work page 2022
-
[2]
L. X. Nguyen, A. D. Raha, P. S. Aung, D. Niyato, Z. Han, and C. S. Hong, “A contemporary survey on semantic communications: Theory of mind, generative ai, and deep joint source-channel coding,”IEEE Commun. Surveys Tuts., Early Access, 2025
work page 2025
-
[3]
L. X. Nguyen, Y . L. Tun, Y . K. Tun, M. N. H. Nguyen, C. Zhang, Z. Han, and C. S. Hong, “Swin transformer-based dynamic semantic communication for multi-user with different computing capacity,”IEEE Trans. Veh. Technol., vol. 73, no. 6, pp. 8957–8972, Jun. 2024
work page 2024
-
[4]
Semantic communications for future internet: Fundamentals, applications, and challenges,
W. Yang, H. Du, Z. Q. Liew, W. Y . B. Lim, Z. Xiong, D. Niyato, X. Chi, X. Shen, and C. Miao, “Semantic communications for future internet: Fundamentals, applications, and challenges,”IEEE Commun. Surveys Tuts., vol. 25, no. 1, pp. 213–250, First-Quarter 2023
work page 2023
-
[5]
C. E. Shannon,The mathematical theory of communication, (and recent contributions to the mathematical theory of communication), W. Weaver. University of Illinois Press Champaign, IL, Jul. 1949
work page 1949
-
[6]
Deep joint source- channel coding for wireless image transmission,
E. Bourtsoulatze, D. B. Kurka, and D. G ¨und¨uz, “Deep joint source- channel coding for wireless image transmission,”IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 3, pp. 567–579, Sep. 2019
work page 2019
-
[7]
Deep learning for joint source- channel coding of text,
N. Farsad, M. Rao, and A. Goldsmith, “Deep learning for joint source- channel coding of text,” inProc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), pp. 2326–2330, AB, Canada, Apr. 2018
work page 2018
-
[8]
Deep learning enabled semantic communication systems,
H. Xie, Z. Qin, G. Y . Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,”IEEE Trans. Signal Process., vol. 69, pp. 2663–2675, Apr. 2021
work page 2021
-
[9]
A robust deep learning enabled semantic communication system for text,
X. Peng, Z. Qin, D. Huang, X. Tao, J. Lu, G. Liu, and C. Pan, “A robust deep learning enabled semantic communication system for text,” inProc. IEEE Glob. Commun. Conf., Rio de Janeiro, Brazil, Dec. 2022
work page 2022
-
[10]
Toward semantic communications: Deep learning-based image semantic coding,
D. Huang, F. Gao, X. Tao, Q. Du, and J. Lu, “Toward semantic communications: Deep learning-based image semantic coding,”IEEE J. Sel. Areas Commun., vol. 41, no. 1, pp. 55–71, Jan. 2023
work page 2023
-
[11]
Deepjscc-f: Deep joint source-channel coding of images with feedback,
D. B. Kurka and D. G ¨und¨uz, “Deepjscc-f: Deep joint source-channel coding of images with feedback,”IEEE J. Sel. Areas Inf. Theory, vol. 1, no. 1, pp. 178–193, May 2020
work page 2020
-
[12]
Witt: A wireless image transmission transformer for semantic communications,
K. Yang, S. Wang, J. Dai, K. Tan, K. Niu, and P. Zhang, “Witt: A wireless image transmission transformer for semantic communications,” inProc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), Rhodes Island, Greece, Jun. 2023
work page 2023
-
[13]
Semantic-preserved communication system for highly efficient speech transmission,
T. Han, Q. Yang, Z. Shi, S. He, and Z. Zhang, “Semantic-preserved communication system for highly efficient speech transmission,”IEEE J. Sel. Areas Commun., vol. 41, no. 1, pp. 245–259, Jan. 2023
work page 2023
-
[14]
Deep learning enabled semantic communications with speech recognition and synthesis,
Z. Weng, Z. Qin, X. Tao, C. Pan, G. Liu, and G. Y . Li, “Deep learning enabled semantic communications with speech recognition and synthesis,”IEEE Trans. Wirel. Commun., vol. 22, no. 9, pp. 6227–6240, Sep. 2023
work page 2023
-
[15]
Semantic communication systems for speech transmission,
Z. Weng and Z. Qin, “Semantic communication systems for speech transmission,”IEEE J. Sel. Areas Commun., vol. 39, no. 8, pp. 2434– 2444, Aug. 2021
work page 2021
-
[16]
Wireless image transmission using deep source channel coding with attention modules,
J. Xu, B. Ai, W. Chen, A. Yang, P. Sun, and M. Rodrigues, “Wireless image transmission using deep source channel coding with attention modules,”IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 4, pp. 2315–2328, May. 2021
work page 2021
-
[17]
A robust image semantic communication system with multi-scale vision transformer,
X. Peng, Z. Qin, X. Tao, J. Lu, and K. B. Letaief, “A robust image semantic communication system with multi-scale vision transformer,” IEEE J. Sel. Areas Commun., vol. 43, no. 4, pp. 1278–1291, Jan. 2025
work page 2025
-
[18]
Non-orthogonal multiple access enhanced multi-user semantic communication,
W. Li, H. Liang, C. Dong, X. Xu, P. Zhang, and K. Liu, “Non-orthogonal multiple access enhanced multi-user semantic communication,”IEEE Trans. Cogn. Commun. Netw., vol. 9, no. 6, pp. 1438–1453, Dec. 2023
work page 2023
-
[19]
Task-oriented multi-user semantic communications,
H. Xie, Z. Qin, X. Tao, and K. B. Letaief, “Task-oriented multi-user semantic communications,”IEEE J. Sel. Areas Commun., vol. 40, no. 9, pp. 2584–2597, Sep. 2022
work page 2022
-
[20]
Rethinking generative semantic communication for multi- user systems with large language models,
W. Yang, Z. Xiong, S. Mao, T. Q. S. Quek, P. Zhang, M. Debbah, and R. Tafazolli, “Rethinking generative semantic communication for multi- user systems with large language models,”IEEE Wirel. Commun., pp. 1–9, Early Access, 2025. 10
work page 2025
-
[21]
Optimizing multi-user semantic communica- tion via transfer learning and knowledge distillation,
L. X. Nguyen, K. Kim, Y . Lin Tun, S. Salman Hassan, Y . Kyaw Tun, Z. Han, and C. S. Hong, “Optimizing multi-user semantic communica- tion via transfer learning and knowledge distillation,”IEEE Commun. Lett., vol. 29, no. 1, pp. 90–94, Jan. 2025
work page 2025
-
[22]
A lite distributed semantic communication system for internet of things,
H. Xie and Z. Qin, “A lite distributed semantic communication system for internet of things,”IEEE J. Sel. Areas Commun., vol. 39, no. 1, pp. 142–153, Jan. 2021
work page 2021
-
[23]
Economics of semantic communication system in wireless powered internet of things,
Z. Q. Liew, Y . Cheng, W. Y . B. Lim, D. Niyato, C. Miao, and S. Sun, “Economics of semantic communication system in wireless powered internet of things,” inProc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), Singapore, May 2022, pp. 8637–8641
work page 2022
-
[24]
Generative semantic communication: Diffusion models beyond bit recovery,
E. Grassucci, S. Barbarossa, and D. Comminiello, “Generative semantic communication: Diffusion models beyond bit recovery,” ArXiv:2306.04321, 2023
work page internal anchor Pith review arXiv 2023
-
[25]
Generative al-aided joint training-free secure semantic communications via multi-modal prompts,
H. Du, G. Liu, D. Niyato, J. Zhang, J. Kang, Z. Xiong, B. Ai, and D. I. Kim, “Generative al-aided joint training-free secure semantic communications via multi-modal prompts,” inProc. IEEE Int. Conf. Acoust. Speech Signal Process., Seoul, South Korea, Apr. 2024
work page 2024
-
[26]
Y . Liu, H. Du, D. Niyato, J. Kang, Z. Xiong, S. Mao, P. Zhang, and X. Shen, “Cross-modal generative semantic communications for mobile aigc: Joint semantic encoding and prompt engineering,”IEEE Trans. Mob. Comput., vol. 23, no. 12, pp. 14 871–14 888, Dec. 2024
work page 2024
-
[27]
Semantics-native commu- nication via contextual reasoning,
H. Seo, J. Park, M. Bennis, and M. Debbah, “Semantics-native commu- nication via contextual reasoning,”IEEE Trans. Cogn. Commun. Netw., vol. 9, no. 3, pp. 604–617, Jun. 2023
work page 2023
-
[28]
F. Zhao, Y . Sunet al., “Enhancing reasoning ability in semantic communication through generative ai-assisted knowledge construction,” IEEE Commun. Lett., vol. 28, no. 4, pp. 832–836, Apr. 2024
work page 2024
-
[29]
Neuro-symbolic causal reasoning meets signaling game for emergent semantic communications,
C. K. Thomas and W. Saad, “Neuro-symbolic causal reasoning meets signaling game for emergent semantic communications,”IEEE Trans. Wireless Commun., vol. 23, no. 5, pp. 4546–4563, May 2024
work page 2024
-
[30]
Real-time semantic communications with a vision transformer,
H. Yoo, T. Jung, L. Dai, S. Kim, and C.-B. Chae, “Real-time semantic communications with a vision transformer,” inProc. IEEE Int. Conf. Commun. Workshops (ICC Workshops), Seoul, South Korea, May 2022
work page 2022
-
[31]
Z. Lyu, G. Zhu, J. Xu, B. Ai, and S. Cui, “Semantic communications for image recovery and classification via deep joint source and channel coding,”IEEE Trans. Wireless Commun., vol. 23, no. 8, pp. 8388–8404, Aug. 2024
work page 2024
-
[32]
Deepma: End-to-end deep multiple access for wireless image transmission in semantic communication,
W. Zhang, K. Bai, S. Zeadally, H. Zhang, H. Shao, H. Ma, and V . C. M. Leung, “Deepma: End-to-end deep multiple access for wireless image transmission in semantic communication,”IEEE Trans. Cogn. Commun. Netw., vol. 10, no. 2, pp. 387–402, Apr. 2024
work page 2024
-
[33]
Joint source-channel coding for channel-adaptive digital semantic communications,
J. Park, Y . Oh, S. Kim, and Y .-S. Jeon, “Joint source-channel coding for channel-adaptive digital semantic communications,”IEEE Trans. Cogn. Commun. Netw., vol. 11, no. 1, pp. 75–89, Feb. 2025
work page 2025
-
[34]
In-band full-duplex system for semantic communication,
M. Shi, H. Liang, C. Dong, X. Xu, R. Meng, and P. Zhang, “In-band full-duplex system for semantic communication,”IEEE Internet Things J., Early Access, 2025
work page 2025
-
[35]
Multiuser content-style adaptive semantic communication for image transmission,
M. Song, N. Ma, H. Liang, C. Dong, W. Li, J. Chen, Y . Lin, and P. Zhang, “Multiuser content-style adaptive semantic communication for image transmission,”IEEE Internet Things J., vol. 12, no. 17, pp. 36 063–36 078, Sep. 2025
work page 2025
-
[36]
Enhancing the reliability of multi-user image semantic communication in wireless networks,
Z. Huang, Y . Jia, J. Yan, W. Wen, L. Liang, X. Liu, and X. Wu, “Enhancing the reliability of multi-user image semantic communication in wireless networks,”IEEE Internet Things J., Early Access, 2025
work page 2025
-
[37]
Adaptive deep joint source- channel coding for one-to-many wireless image transmission,
L. Luo, Z. He, J. Wu, H. Guo, and C. Zhu, “Adaptive deep joint source- channel coding for one-to-many wireless image transmission,”IEEE Trans. Broadcast., vol. 71, no. 3, pp. 914–929, Sep. 2025
work page 2025
-
[38]
DMCE: Diffusion model channel enhancer for multi-user semantic communication systems,
Y . Zeng, X. He, X. Chen, H. Tong, Z. Yang, Y . Guo, and J. Hao, “DMCE: Diffusion model channel enhancer for multi-user semantic communication systems,” inProc. IEEE Int. Conf. Commun. (ICC), CO, USA, Jun. 2024
work page 2024
-
[39]
Deep joint source channel coding with semantic precoder for multi-task multi-user mimo systems,
W. Jiang, W. Chen, and B. Ai, “Deep joint source channel coding with semantic precoder for multi-task multi-user mimo systems,” inProc. IEEE 26th Int. Workshop Signal Process. Artif. Intell. Wireless Commun. (SPAWC), Surrey, UK, Jul. 2025
work page 2025
-
[40]
Adaptive federated deep learning-based semantic communication in the social internet of things,
W. Tan, Y . Wang, L. Liu, X. Wang, and T. Ding, “Adaptive federated deep learning-based semantic communication in the social internet of things,”IEEE Internet Things J., vol. 12, no. 17, pp. 34 921–34 930, Sep. 2025
work page 2025
-
[41]
Overcoming catastrophic forgetting in neural networks,
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,”Proc. Natl. Acad. Sci., vol. 114, no. 13, pp. 3521–3526, Dec. 2017
work page 2017
-
[42]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., NV , Jun. 2016, pp. 770–778
work page 2016
-
[43]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” inProc. Int. Conf. Learn. Represent. (ICLR), San Diego, CA, Sep. 2014
work page 2014
-
[44]
Ntire 2017 challenge on single image super-resolution: Dataset and study,
E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” inProc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Honolulu, HI, Jul. 2017
work page 2017
-
[45]
Kodak lossless true color image suite,
R. Franzen, “Kodak lossless true color image suite,”source: http://r0k. us/graphics/kodak, vol. 4, no. 2, Nov. 1999
work page 1999
-
[46]
Multiscale structural similarity for image quality assessment,
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” inProc. IEEE 37th Asilomar Conf. Signals Syst. Comput., vol. 2, Pacific Grove, CA, Nov. 2003
work page 2003
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.