REVIEW 5 major objections 6 minor 1 cited by
Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation
T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Molar claims a sequential recommender should fuse IDs after, not before, the multimodal LLM builds item representations.
desk verdict Molar's post-alignment is a real new variant in LLM sequential recommendation, but the reported numbers can't be checked until the evaluation protocol is specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the post-alignment contrastive loss $L_{align}$ (Eq. 4), a symmetric InfoNCE-style objective that aligns each content-based user embedding $E^{con}_u$ from the multimodal LLM path with the ID-based user embedding $E^{id}_u$ from a traditional sequential recommender, trained jointly with a pointwise BCE next-item loss $L_{bce}$. This is what lets Molar keep the MLLM's multimodal semantics while importing collaborative-filtering structure at the user-embedding level rather than at the token level.
What would settle it
Run Molar and an early-fusion baseline (CoLLM or HLLM) on a dataset where item text and images are near-duplicate or uninformative (e.g., anonymized purchases with generic titles) so IDs carry most of the signal; if Molar's NDCG@10 gain over early fusion vanishes or reverses, the post-alignment advantage is not about alignment timing but about content quality. A cheaper check is to ablate the alignment loss while holding everything else fixed and see whether the remaining content-only model still beats early fusion.
Extended reading notes
Core claim
Molar's central claim is that collaborative-filtering signals should be injected into an LLM-based sequential recommender after, not before, the LLM has built its content representations. The framework first fine-tunes a multimodal LLM (Qwen2-VL) as a Multimodal Item Representation Model, extracting an item embedding from a special [Cur_Item] token over a description that fuses text, image, and structured attributes. A Dynamic User Embedding Generator, also built on the LLM backbone, consumes the sequence of item embeddings and emits a content-based user embedding from a [User] token. A traditional sequential model (SASRec or DuoRec) trained on the same interaction sequences emits an ID-based user embedding, and a symmetric contrastive loss pulls the two user representations together. The experiments report that this design outperforms LLM baselines that concatenate ID embeddings with text early in the LLM input (CoLLM, HLLM), content-only variants, and classical sequential models on all three datasets, with the largest relative gains on NDCG-type metrics.
Load-bearing premise
The load-bearing premise is that aligning content-based and ID-based user embeddings after the multimodal LLM has processed the content preserves collaborative-filtering signals better than feeding ID tokens into the LLM early; if that late-alignment benefit does not generalize beyond these three datasets and this training setup, Molar's edge over CoLLM and HLLM would disappear.
Editorial extensions
If this is right
- Molar reports consistent improvements over the strongest baselines on all three datasets, e.g., over 7% relative gains in NDCG/Recall on MovieLens, meaning late ID alignment is a viable alternative to early fusion in LLM recommenders.
- The decoupled design (item encoder MIRM, user encoder DUEG) means user histories are compressed to embeddings rather than long token sequences, reducing inference cost relative to feeding full histories to an LLM.
- The post-alignment module can use any traditional sequential model; stronger models such as DuoRec yield better final performance, so the method scales with improvements in ID-based recommenders.
- Multimodal input (image + text) outperforms either modality alone on PixelRec, indicating the MLLM fusion is the source of a substantial part of the gain.
Reading between the lines
- The paper argues that early ID fusion 'can cause LLMs to learn shortcuts,' but it does not measure shortcut behavior directly; a natural test is to compare how much Molar's content and ID user embeddings diverge on users with long vs cold-start histories.
- If the post-alignment benefit is really about preserving collaborative structure, then Molar should transfer to non-LLM user encoders and to other sequence tasks (e.g., click-through prediction), providing a cheap way to retrofit multimodal understanding into existing recommender stacks.
- The reported gains are on full-corpus evaluation of next-item prediction; on candidate-ranking or re-ranking settings where negative sampling differs, the relative advantage may shrink, which the paper does not test.
- Molar's advantage likely depends on item content being genuinely informative; on datasets where IDs dominate (e.g., anonymous sessions with little metadata), the content path may add noise and the alignment loss would then dilute ID signal.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Molar, a multimodal LLM-based sequential recommendation framework that decouples item and user modeling. A Multimodal Item Representation Model (MIRM), built on Qwen2-VL, encodes each item's text and image into a compact embedding; a Dynamic User Embedding Generator (DUEG) builds a user embedding from the sequence of item embeddings; and a post-alignment contrastive loss aligns this content-based user embedding with an ID-based user embedding produced by a traditional sequential model such as SASRec. Training combines a point-wise BCE next-item loss with the alignment loss. Experiments on Amazon, PixelRec, and MovieLens report consistent improvements over traditional, content-based, and LLM-based baselines, with ablations on DUEG choice, input modalities, post-alignment models, fine-tuning data, and MLLM backbone.
Significance. If the reported results are reproducible, the paper makes a useful empirical contribution: it demonstrates that a post-alignment architecture, in which ID-based collaborative signals are aligned with content-based user embeddings only after multimodal LLM processing, can outperform early-fusion LLM recommenders such as CoLLM and HLLM. The decoupled design also has a practical efficiency motivation, and the ablation study is fairly complete, covering fine-tuning objectives, modality inputs, DUEG variants, and post-alignment models. The claimed gains are meaningful (roughly 4-8% relative over strong LLM baselines). However, the paper's central claim is entirely empirical, and the evaluation protocol is currently underspecified to the point that the headline numbers cannot be independently verified; this is the main barrier to accepting the paper. The authors also provide an anonymous code link, which is a positive step, but without a commit hash or configuration details it does not yet constitute reproducible artifacts.
major comments (5)
- [§4.1.1 and Table 2] The evaluation protocol is not fully specified. The text states that NDCG and Recall are 'evaluated on the full amount of data,' but it never states the candidate item set used for ranking. On Amazon, with 301,312 items, full-ranking evaluation with a 2B-parameter MLLM is computationally prohibitive, so the evaluation almost certainly uses sampled negatives, a truncated candidate set, or an approximate nearest-neighbor index; none of these is described. The reported metrics are therefore not independently verifiable, and unless the baselines use the identical candidate-generation process and seed, cross-method comparisons in Table 2 are not valid. This is load-bearing because the paper's central claim is that Molar 'consistently outperforms all baseline models across three datasets.' The authors must specify the exact candidate set construction, including the number of negatives, the sampling distribution, the seed, and whether the same protocol is applied to every baseline.
- [Table 2 and §4.2] The paper reports 'p-value << 0.05' in Table 2 but provides no number of runs, no standard deviations, no confidence intervals, and no description of the statistical test used. A significance claim of this strength requires at least three to five independent runs with seeds stated, and ideally paired tests across baselines. Without this information, the reader cannot distinguish real improvements from noise, especially for the small absolute differences on PixelRec (e.g., N@10 0.0197 vs. 0.0189 for HLLM). Please report run counts, standard deviations, and the exact test.
- [§3.4 and Appendix B] The DUEG architecture is described only as removing the word embedding layer from the MLLM and adding a [User] token. This is insufficient to reproduce the method: the paper does not state the number of remaining transformer layers, the initialization of the [User] token, the output projection, or how the predicted logits are computed for the BCE loss. Additionally, Appendix B describes a 1:1 negative-sampling strategy for training, which is reasonable, but the relationship between this train-time sampling and the claimed full-ranking evaluation is not explained; if evaluation uses a different candidate set, the train/eval mismatch should be acknowledged and analyzed.
- [§4.1.2 and code availability] The reproducibility of the experiments is currently incomplete. The anonymous code link has no commit hash or version identifier, and the paper does not report the values of several free parameters that affect the results: the alignment loss weight alpha, the temperature tau, the number of contrastive negatives K, the number of BCE negatives beyond the stated 1:1 pairing, the sizes of the three fine-tuning data mixtures beyond '10,000 data points,' and the LoRA configuration used in Appendix A. Please provide a configuration table or a tagged code release so that the reported numbers can be reproduced.
- [§3.4 and Figure 1] The paper motivates post-alignment by claiming it 'preserves collaborative signals better than early ID fusion,' but it provides no mechanism-level analysis or diagnostic experiments supporting this specific claim. The ablation in Table 5 shows that removing the alignment loss hurts performance, which is evidence that the loss helps, but it does not test whether the benefit comes specifically from post-alignment rather than from any auxiliary ID-based signal, nor does it compare directly against an early-fusion variant under matched conditions. A small experiment with early ID fusion using the same MIRM and DUEG would substantially strengthen the paper's central conceptual claim.
minor comments (6)
- [Table 2] The Molar row contains a typo: '01478' under Amazon N@20 should be '0.1478'.
- [Figure 3] Figure 3 reports comparisons of DUEG variants but provides no numeric values, error bars, or dataset labels in the caption; please add a table or axis-visible values.
- [Appendix A] The text says 'As shown in Table 5' when referring to the backbone comparison, but the table is numbered Table 6; please fix the cross-reference.
- [§4.2] The phrase 'over a 7% improvement' should state explicitly whether this is a relative or absolute improvement, and over which baseline and metric, since the values in Table 2 do not make this unambiguous.
- [§4.1.1] The phrase 'evaluated on the full amount of data' is ambiguous and should be replaced with a precise description of the evaluation candidate set, as discussed in the major comments.
- [References] Some references are cited only by arXiv preprint versions (e.g., Devlin 2018, Dosovitskiy 2020); consider citing the published versions where available.
Circularity Check
No significant circularity: Molar is an empirical architecture trained on public benchmarks with an externally pretrained MLLM; no central claim reduces to a fitted input or a self-citation chain.
full rationale
The paper's central claims are empirical: Molar outperforms baselines on Amazon, PixelRec, and MovieLens (Table 2). The method combines an externally pretrained MLLM (Qwen2-VL) with an ID-based sequential model (SASRec) via a post-alignment contrastive loss. No derivation in the paper reduces a prediction to its own input by construction. The three MIRM fine-tuning objectives (image-text alignment, structured attribute processing, temporal user behavior understanding) are training objectives, and the temporal user behavior objective does use next-item prediction, which is also the evaluation task; however, this is standard supervised learning on held-out data, not a self-definitional reduction or a fitted parameter renamed as a prediction. The post-alignment loss aligns content-based user embeddings with ID-based user embeddings from SASRec, but SASRec is itself a baseline that Molar outperforms, so the improvement is not forced by the teacher. The paper cites prior work by its own authors (e.g., Luo et al. 2024, Cheng et al. 2021), but these citations are background references and are not load-bearing justifications for Molar's architecture or results. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The underspecified evaluation protocol (e.g., unclear candidate item set and missing run counts) is a reproducibility and correctness concern, not a circularity concern, because the reported comparisons do not reduce to the method's own definitions. The ablation study (Section 4.6) independently shows that removing each component degrades performance, which is consistent with the components contributing real signal rather than being circularly encoded in the evaluation metric. Overall, the derivation chain is self-contained against external benchmarks and does not exhibit the circular patterns defined in the review criteria.
Assumptions & free parameters
free parameters (5)
- Alpha (alignment loss weight)
- Temperature (tau)
- Number of contrastive negatives (K)
- Negative sampling ratio for BCE =
1:1
- Fine-tuning data size per objective =
10,000
assumptions (4)
- domain assumption Pretrained MLLM (Qwen2-VL) representations are a useful starting point for item embeddings.
- domain assumption The three fine-tuning objectives (image-text alignment, structured attribute processing, temporal user behavior) improve downstream recommendation.
- ad hoc to paper Post-alignment of content-based and ID-based user embeddings preserves collaborative signals better than early ID fusion.
- domain assumption Leave-one-out evaluation over the full item set is a fair and standard protocol.
Cite this review
Pith. "Pith review of Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation." pith.science (2026). https://pith.science/paper/RQMZSNEV
@misc{pith2026241218176,
author = {Pith},
title = {Pith review of: Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation},
year = {2026},
howpublished = {\url{https://pith.science/paper/RQMZSNEV}},
note = {Machine review of arXiv:2412.18176}
}
read the original abstract
Sequential recommendation (SR) systems have evolved significantly over the past decade, transitioning from traditional collaborative filtering to deep learning approaches and, more recently, to large language models (LLMs). While the adoption of LLMs has driven substantial advancements, these models inherently lack collaborative filtering information, relying primarily on textual content data neglecting other modalities and thus failing to achieve optimal recommendation performance. To address this limitation, we propose Molar, a Multimodal large language sequential recommendation framework that integrates multiple content modalities with ID information to capture collaborative signals effectively. Molar employs an MLLM to generate unified item representations from both textual and non-textual data, facilitating comprehensive multimodal modeling and enriching item embeddings. Additionally, it incorporates collaborative filtering signals through a post-alignment mechanism, which aligns user representations from content-based and ID-based models, ensuring precise personalization and robust performance. By seamlessly combining multimodal content with collaborative filtering insights, Molar captures both user interests and contextual semantics, leading to superior recommendation accuracy. Extensive experiments validate that Molar significantly outperforms traditional and LLM-based baselines, highlighting its strength in utilizing multimodal data and collaborative signals for sequential recommendation tasks. The source code is available at https://anonymous.4open.science/r/Molar-8B06/.
Figures
Forward citations
Cited by 1 Pith paper
-
Large Language Model Enhanced Recommender Systems: A Survey
A survey organizing LLM-enhanced recommender systems into knowledge, interaction, and model enhancement, and tracing a shift from explicit text to implicit embeddings and fine-tuned open-source LLMs.
Reference graph
Works this paper leans on
-
[1]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1007--1014
2023
-
[4]
Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan. 2024. Hllm: Enhancing sequential recommendations via hierarchical large language models for item and user modeling. arXiv preprint arXiv:2409.12740
arXiv 2024
-
[5]
Mingyue Cheng, Zhiding Liu, Qi Liu, Shenyang Ge, and Enhong Chen. 2022. Towards automatic discovering of deep hybrid network architecture for sequential recommendation. In Proceedings of the ACM Web Conference 2022, pages 1923--1932
work page 2022
-
[6]
Mingyue Cheng, Fajie Yuan, Qi Liu, Xin Xin, and Enhong Chen. 2021. Learning transferable user representations with sequential behaviors via contrastive pre-training. In 2021 IEEE International Conference on Data Mining (ICDM), pages 51--60. IEEE
work page 2021
-
[7]
Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
arXiv 2018
-
[8]
Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
arXiv 2020
Show all 38 references
-
[9]
Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, et al. 2023. Leveraging large language models in conversational recommender systems. arXiv preprint arXiv:2305.07961
2023 arXiv
-
[10]
David Goldberg, David Nichols, Brian M Oki, and Douglas Terry. 1992. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61--70
1992
-
[11]
Aaron Grattafiori, Abhimanyu Dubey, and Abhinav Jauhri et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783
2024 arXiv
-
[12]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197--206. IEEE
2018
-
[13]
u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \
Enkelejda Kasneci, Kathrin Se ler, Stefan K \"u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \"u nnemann, Eyke H \"u llermeier, et al. 2023. Chatgpt for good? on opportunities and challenges of large language models for education....
2023
-
[14]
Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426--434
2008
-
[15]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, 42(8):30--37
2009
-
[16]
Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2023. Llara: Aligning large language models with sequential recommenders. arXiv preprint arXiv:2312.02445
2023 arXiv
-
[17]
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. 2024. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525
2024 arXiv
-
[18]
Yucong Luo, Mingyue Cheng, Hao Zhang, Junyu Lu, and Enhong Chen. 2024. Unlocking the potential of large language models for explainable recommendations. In International Conference on Database Systems for Advanced Applications, pages 286--303. Springer
2024
-
[19]
Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O'Banion, and Jun Xie. 2024. User-llm: Efficient llm contextualization with user embeddings. arXiv preprint arXiv:2402.13598
2024 arXiv
-
[20]
OpenAI, Josh Achiam, Steven Adler, and Sandhini Agarwal et al. 2024. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774
2024 arXiv
-
[21]
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824
2023 arXiv
-
[22]
Ruihong Qiu, Zi Huang, Hongzhi Yin, and Zijian Wang. 2022. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the fifteenth ACM international conference on web search and data mining, pages 813--823
2022
-
[23]
Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. 2024. Representation learning with large language models for recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3464--3475
2024
-
[24]
Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web, pages 811--820
2010
-
[25]
Paul Resnick and Hal R Varian. 1997. Recommender systems. Communications of the ACM, 40(3):56--58
1997
-
[26]
Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved recurrent neural networks for session-based recommendations. In Proceedings of the 1st workshop on deep learning for recommender systems, pages 17--22
2016
-
[27]
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805
2023 arXiv
-
[28]
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191
2024 arXiv
-
[29]
Shoujin Wang, Liang Hu, Yan Wang, Longbing Cao, Quan Z Sheng, and Mehmet Orgun. 2019. Sequential recommender systems: challenges, progress and prospects. arXiv preprint arXiv:2001.04830
2019 arXiv
-
[30]
Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. 2024. Towards open-world recommendation with knowledge augmentation from large language models. In Proceedings of the 18th ACM Conference on Recommender ...
2024
-
[31]
Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2024. A survey on multimodal large language models. National Science Review, page nwae403
2024
-
[32]
Chao Zhang, Shiwei Wu, Haoxin Zhang, Tong Xu, Yan Gao, Yao Hu, and Enhong Chen. 2024 a . Notellm: A retrievable large language model for note recommendation. In Companion Proceedings of the ACM on Web Conference 2024, pages 170--179
2024
-
[33]
Chiyu Zhang, Yifei Sun, Jun Chen, Jie Lei, Muhammad Abdul-Mageed, Sinong Wang, Rong Jin, Sem Park, Ning Yao, and Bo Long. 2024 b . Spar: Personalized content-based recommendation via long engagement attention. arXiv preprint arXiv:2402.10555
2024 arXiv
-
[34]
Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024 c . Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601
2024 arXiv
-
[35]
Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2023 a . Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001
2023 arXiv
-
[36]
Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He. 2023 b . Collm: Integrating collaborative embeddings into large language models for recommendation. arXiv preprint arXiv:2310.19488
2023 arXiv
-
[37]
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223
2023 arXiv
-
[38]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining,...
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.