Pith. sign in

REVIEW 3 major objections 2 minor 28 references

Neural Proteomics Fields for Super-resolved Spatial Proteomics Prediction

T0 review · 3 major / 2 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A general-purpose vision-language model paired with a retriever trained to follow the model's own predictions reaches competitive medical diagnostic accuracy without medical pre-training.

desk verdict Do not review this as submitted: the abstract claims a spatial proteomics model that never appears in the full text, which is an entirely different paper on medical retrieval-augmented diagnosis. read the letter →

arxiv 2508.17389 v1 pith:5JMNTPE4 submitted 2025-08-24 q-bio.QM cs.AIcs.CV

Bokai Zhao , Weiyang Shi , Hanqing Chao , Zijiang Yang , Yiyang Zhang , Ming Song , Tianzi Jiang This is my paper · ORCID
classification q-bio.QMcs.AIcs.CV
keywords retrieval-augmentedgenerationmedicaldiagnosislargevision-languagemodelsmultimodalretrievallightweightfine-tuninginconsistentpredictionsKLdistillationgeneral-purpose
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a general-purpose vision-language model, without any medical pre-training, can reach competitive clinical diagnostic accuracy when the retrieval system that feeds it evidence is optimized to the model's own judgments. The authors propose CLARE, a two-stage training scheme: first fine-tune the reader LVLM with retrieved image-text pairs, then fine-tune a dual-head multimodal retriever so that its rankings match the frozen LVLM's confidence over candidate answers. On five medical classification and three VQA benchmarks, CLARE matches or approaches systems that underwent extensive medical pre-training, using training sets as small as a few hundred images. The paper also identifies a previously uncharacterised failure class, 'inconsistent retrieval predictions', where different top-retrieved images pull the model to different answers, and shows the retriever update substantially improves these hard cases.

What carries the argument

The load-bearing mechanism is a KL-divergence objective between two distributions over the retrieved candidates: the reader's softmax over the benchmark's class tokens (a class-restricted posterior, sharpened by restricting logits to the answer set) and the retriever's softmax over dot-product similarities between query and candidate embeddings. Training is sequential—reader first with the retriever frozen, then the text retrieval head, then the image head with the reader frozen—so the retriever inherits a training signal tied to diagnostic correctness rather than to generic relevance. At inference, predictions from the top retrieved candidates are combined by a likelihood-weighted fusion.

What would settle it

On a new clinical dataset with official test splits, measure whether the KL-trained retriever ranks, among its top candidates, the ones that actually shift the reader toward the correct answer more often than the base retriever does. A concrete calculation: compare the correlation between CLARE's retrieval similarity scores and the reader's likelihood of the correct answer for each candidate, on held-out queries, against the same correlation for the untuned retriever.

Watch

Extended reading notes

Core claim

The central claim is that generation-aware optimisation of a multimodal retriever can substitute for domain-specific pre-training in medical diagnosis. Concretely, the paper shows that distilling the frozen LVLM's class-restricted posterior into the retriever's ranking, after a reader fine-tuning phase, outperforms standard fine-tuned RAG and MMed-RAG, and closes much of the gap to medically pre-trained LVLMs. The second, more conceptual discovery is the existence and character of inconsistent retrieval predictions: instances where different candidates in the top-retrieved set lead to different model predictions, which are harder for all models, and which the LVLM-aware retriever specifically mitigates. The oracle analysis then shows that in many of these hard cases a candidate that would yield the correct answer is already in the retrieved set, so the remaining gap is in fusion, not retrieval.

Load-bearing premise

The retriever is trained to trust the frozen vision-language model's confidence scores over retrieved candidates as a reliable signal of which candidates are genuinely useful, and if those confidence scores are systematically biased by the retrieved context, the retriever inherits that bias.

Editorial extensions

If this is right

  • General-purpose backbones with a reader-aware retriever can serve as a cost-effective alternative to medical pre-training in low-resource clinical settings.
  • The inconsistent-retrieval-prediction category gives a measurable target for future work, and the oracle result implies better fusion or reranking could unlock further gains.
  • The sequential recipe of reader fine-tuning followed by retriever distillation can be transferred to other domains where a general-purpose LVLM is paired with a domain corpus.
  • Retrieval augmentation remains useful even with noisy candidates, and the reader retains its standalone performance after training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to replace the simple likelihood-weighted fusion with a learned selector or a trained reranker, which the paper's oracle analysis suggests could close much of the remaining gap on inconsistent cases.
  • The same KL-distillation idea could be applied in other specialised settings, such as legal or scientific literature QA, offering a cheap alternative to expensive domain pre-training for general-purpose multimodal models.
  • Because the distillation relies on a finite answer set, extending it to fully open-ended generation would require a different supervisory signal, such as token-level likelihood differences or listwise ranking losses, a direction the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The abstract of arXiv:2508.17389 announces a new task, seq-SP (spatial super-resolution for sequencing-based spatial proteomics), a model called Neural Proteomics Fields (NPF) with Spatial Modeling and Morphology Modeling modules, a Pseudo-Visium SP benchmark, state-of-the-art results, and a public GitHub repository. The supplied full text, however, is a different paper: 'LVLM-Aware Multimodal Retrieval for RAG-Based Medical Diagnosis with General-Purpose Models' (CLARE), by different authors, about retrieval-augmented diagnosis with large vision-language models. The full text contains no mention of spatial proteomics, no NPF architecture, no Pseudo-Visium SP dataset construction, no seq-SP task definition, and no experimental results for NPF. The central claim of the submission is therefore absent from the submitted evidence.

Significance. If the abstract's claims were supported, the paper could be significant: spatial proteomics super-resolution is an active area, and a parameter-efficient tissue-specific model with a public benchmark would be a useful contribution. However, none of these components appears in the supplied manuscript. The CLARE paper that actually constitutes the full text has its own merits: it reports consistent gains over RAG baselines, includes ablations over retriever heads and candidate counts, provides an oracle analysis, and documents robustness checks. Those are strengths of a different paper. Because the artifact-level mismatch prevents evaluation of the stated contribution, the significance for the claimed NPF work cannot be assessed.

major comments (3)
  1. [Abstract vs. supplied full text] The abstract claims a Neural Proteomics Fields model with a Spatial Modeling Module, a Morphology Modeling Module, a new seq-SP task, a Pseudo-Visium SP benchmark, and state-of-the-art performance with fewer parameters. The supplied full text contains none of these elements: it describes CLARE, a retrieval-augmented medical diagnosis method, with no section, equation, dataset, or experimental table on spatial proteomics. This is a load-bearing mismatch that blocks any scientific verification of the central claim.
  2. [Manuscript metadata and authorship] The metadata and abstract identify the work as Bokai-Zhao's NPF project with a GitHub repository, while the full text is authored by Mazor and Hope and describes CLARE. As submitted, the artifact under this arXiv ID contains two unrelated papers. Even under a charitable reading, a reader cannot connect the claimed contribution to any content in the manuscript, and the mismatch is not a minor editorial issue that can be fixed with local revisions.
  3. [CLARE evaluation, if treated as the intended submission] Even if the full text were the intended manuscript, its central competitiveness claim would be weakened by evaluation choices documented in the text: BRSET and VQA-RAD use internal splits, VQA-RAD is excluded from the medical-pretrained comparison, and no error bars or repeated-seed statistics are reported. These issues are secondary to the abstract/full-text mismatch, but they reinforce that the submitted record does not support the stated claims as written.
minor comments (2)
  1. [Abstract reproducibility claim] The abstract promises publicly available code and the Pseudo-Visium SP dataset at https://github.com/Bokai-Zhao/NPF, but the full text contains no reference to this repository or dataset, leaving the reproducibility claim unverifiable.
  2. [Conclusion and limitations of full text] The full text's conclusion and limitations sections discuss retrieval-augmented medical diagnosis and never mention spatial proteomics, seq-SP, or NPF, further confirming that the supplied text is not a version of the abstract's paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the supplied full text (CLARE) contains no derivable content for the abstract's Neural Proteomics Fields claims, which is an evidence-completeness problem rather than a circularity.

full rationale

The abstract's claimed derivation chain for Neural Proteomics Fields (NPF) -- the seq-SP task, the Spatial Modeling Module, the Morphology Modeling Module, the Pseudo-Visium SP benchmark, and the state-of-the-art claim with fewer learnable parameters -- has no corresponding equations, dataset construction, or experimental tables in the supplied full text. The full text is an unrelated paper, 'LVLM-Aware Multimodal Retrieval for RAG-Based Medical Diagnosis with General-Purpose Models' (CLARE), so no NPF derivation exists in the record that could be checked for circularity. Under the circularity rubric, a step is circular only if a claimed prediction or first-principles result is equivalent to its inputs by construction, via a quoted equation, a fitted parameter renamed as a prediction, or a load-bearing self-citation chain. No such reduction can be exhibited for NPF because the derivation is absent. For the CLARE content that does appear, the retriever is trained by KL distillation to a frozen LVLM's class-restricted posterior on training data, and its evaluation is performed on held-out ground-truth labels and on GPT-5.2 relevance judgments; this is self-referential in spirit but not circular by construction, since the test-time labels and relevance ratings are external to the training objective. The abstract/full-text mismatch is a serious verifiability and completeness problem that should be flagged in correctness review, but it is not a circularity, and no load-bearing self-citation or definitional equivalence is present. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No new physical or mathematical entities are postulated. Free parameters are standard ML hyperparameters; axioms are domain assumptions about retrieval utility and the teacher signal. The 'inconsistent retrieval predictions' concept is an analytical category, not an invented entity.

free parameters (5)
  • Number of retrieved candidates N = 4
    Ablation in Tables 13-14 shows N=4 balances performance; N=2 and N=6 were also tested.
  • Retriever temperature tau = not specified
    Controls sharpness of retriever distribution in the KL loss (Section 3.2); no concrete value is reported.
  • Reader fine-tuning learning rate = ~1e-5 (text garbled)
    Section B.2 reports a learning rate that is corrupted in the provided PDF; it appears to be 1e-5.
  • Retriever fine-tuning learning rate = ~1e-5 (text garbled)
    Section B.3 similarly reports a corrupted numeric value.
  • Training epochs = 10 for reader, 100 for retriever
    Sections B.2-B.3 specify these hyperparameters.
assumptions (3)
  • domain assumption Retrieved image-text pairs from external medical corpora contain predictive signal for the query image's diagnosis.
    Central premise of the RAG setup; the index is built from MIMIC-CXR, PMC-OA, and ROCO (Sections A.2, B.1).
  • domain assumption The frozen LVLM's class-restricted token posterior is a reliable training signal for retriever ranking.
    Equations (1)-(3) in Section 3.2; if the LVLM is miscalibrated, the retriever learns that miscalibration.
  • domain assumption General-purpose LVLMs (Pixtral, Qwen2-VL) can perform medical tasks after lightweight LoRA fine-tuning.
    The whole approach avoids medical pre-training; empirically validated in Section 4 but not guaranteed outside the tested datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural Proteomics Fields for Super-resolved Spatial Proteomics Prediction." pith.science (2026). https://pith.science/paper/5JMNTPE4

@misc{pith2026250817389,
  author       = {Pith},
  title        = {Pith review of: Neural Proteomics Fields for Super-resolved Spatial Proteomics Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JMNTPE4}},
  note         = {Machine review of arXiv:2508.17389}
}
read the original abstract

Spatial proteomics maps protein distributions in tissues, providing transformative insights for life sciences. However, current sequencing-based technologies suffer from low spatial resolution, and substantial inter-tissue variability in protein expression further compromises the performance of existing molecular data prediction methods. In this work, we introduce the novel task of spatial super-resolution for sequencing-based spatial proteomics (seq-SP) and, to the best of our knowledge, propose the first deep learning model for this task--Neural Proteomics Fields (NPF). NPF formulates seq-SP as a protein reconstruction problem in continuous space by training a dedicated network for each tissue. The model comprises a Spatial Modeling Module, which learns tissue-specific protein spatial distributions, and a Morphology Modeling Module, which extracts tissue-specific morphological features. Furthermore, to facilitate rigorous evaluation, we establish an open-source benchmark dataset, Pseudo-Visium SP, for this task. Experimental results demonstrate that NPF achieves state-of-the-art performance with fewer learnable parameters, underscoring its potential for advancing spatial proteomics research. Our code and dataset are publicly available at https://github.com/Bokai-Zhao/NPF.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 20 canonical work pages

  1. [1]

    Human tonsil, https://www.10xgenomics.com/datasets/gene-protein-expression-library-of-human-tonsil-cytassist-ffpe-2-standard

    work page

  2. [2]

    Human tonsil add-on antibodies, https://www.10xgenomics.com/datasets/visium-cytassist-gene-and-protein-expression-library-of-human-tonsil-with-add-on-antibodies-h-e-6-5-mm-ffpe-2-standard

    work page

  3. [3]

    10x Genomics : Spatial gene and protein expression, www.10xgenomics.com

    work page

  4. [4]

    Nature medicine 20(4), 436--442 (2014)

    Angelo, M., Bendall, S.C., Finck, R., Hale, M.B., Hitzman, C., Borowsky, A.D., Levenson, R.M., Lowe, J.B., Liu, S.D., Zhao, S., et al.: Multiplexed ion beam imaging of human breast tumors. Nature medicine 20(4), 436--442 (2014)

    work page 2014

  5. [5]

    Insight j 2(365), 1--35 (2009)

    Avants, B.B., Tustison, N., Song, G., et al.: Advanced normalization tools (ants). Insight j 2(365), 1--35 (2009)

    work page 2009

  6. [6]

    Nature Medicine 30(3), 850--862 (2024)

    Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al.: Towards a general-purpose foundation model for computational pathology. Nature Medicine 30(3), 850--862 (2024)

    work page 2024

  7. [7]

    arXiv preprint arXiv:2205.08534 (2022)

    Chen, Z., Duan, Y., Wang, W., He, J., Lu, T., Dai, J., Qiao, Y.: Vision transformer adapter for dense predictions. arXiv preprint arXiv:2205.08534 (2022)

    arXiv 2022

  8. [8]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    arXiv 2020

Show all 28 references
  1. [9]

    Nature methods 11(4), 417--422 (2014)

    Giesen, C., Wang, H.A., Schapiro, D., Zivanovic, N., Jacobs, A., Hattendorf, B., Sch \"u ffler, P.J., Grolimund, D., Buhmann, J.M., Brandt, S., et al.: Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry. Nature methods 11(4), 417--422 (2014)

    2014

  2. [10]

    Cell 174(4), 968--981 (2018)

    Goltsev, Y., Samusik, N., Kennedy-Darling, J., Bhate, S., Hale, M., Vazquez, G., Black, S., Nolan, G.P.: Deep profiling of mouse splenic architecture with codex multiplexed imaging. Cell 174(4), 968--981 (2018)

    2018

  3. [11]

    Cell 187(10), 2485--2501 (2024)

    Greenwald, A.C., Darnell, N.G., Hoefflin, R., Simkin, D., Mount, C.W., Castro, L.N.G., Harnik, Y., Dumont, S., Hirsch, D., Nomura, M., et al.: Integrative spatial analysis reveals a multi-layered organization of glioblastoma. Cell 187(10), 2485--2501 (2024)

    2024

  4. [12]

    Nature biomedical engineering 4(8), 827--834 (2020)

    He, B., Bergenstr hle, L., Stenbeck, L., Abid, A., Andersson, A., Borg, ., Maaskola, J., Lundeberg, J., Zou, J.: Integrating spatial gene expression and breast tumour morphology via deep learning. Nature biomedical engineering 4(8), 827--834 (2020)

    2020

  5. [13]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770--778 (2016)

    2016

  6. [14]

    Cell (2025)

    Hu, B., He, R., Pang, K., Wang, G., Wang, N., Zhu, W., Sui, X., Teng, H., Liu, T., Zhu, J., et al.: High-resolution spatially resolved proteomics of complex tissues based on microfluidics and transfer learning. Cell (2025)

    2025

  7. [15]

    Nature cell biology 25(8), 1089--1100 (2023)

    Jain, S., Pei, L., Spraggins, J.M., Angelo, M., Carson, J.P., Gehlenborg, N., Ginty, F., Gon c alves, J.P., Hagood, J.S., Hickey, J.W., et al.: Advances and prospects for the human biomolecular atlas program (hubmap). Nature cell biology 25(8), 1089--1100 (2023)

    2023

  8. [16]

    Nat Methods 21, 2195--2196 (2024)

    Karimi, E., Simo, N., Milet, N., TE, W., ALSH, A., QU, N., AIL, L., ABS, R., ALIND, A., GOODMA, N.M., et al.: Method of the year 2024: spatial proteomics. Nat Methods 21, 2195--2196 (2024)

    2024

  9. [17]

    Nucleic Acids Research 52(9), 4843--4856 (2024)

    Li, S., Gai, K., Dong, K., Zhang, Y., Zhang, S.: High-density generation of spatial transcriptomics with stage. Nucleic Acids Research 52(9), 4843--4856 (2024)

    2024

  10. [18]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Lin, T.Y., Doll \'a r, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2117--2125 (2017)

    2017

  11. [19]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 10012--10022 (2021)

    2021

  12. [20]

    Communications of the ACM 65(1), 99--106 (2021)

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65(1), 99--106 (2021)

    2021

  13. [21]

    Nature Reviews Genetics 23(12), 741--759 (2022)

    Moffitt, J.R., Lundberg, E., Heyn, H.: The emerging landscape of spatial profiling technologies. Nature Reviews Genetics 23(12), 741--759 (2022)

    2022

  14. [22]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    2023 arXiv

  15. [23]

    Cell 181(2), 236--249 (2020)

    Rozenblatt-Rosen, O., Regev, A., Oberdoerffer, P., Nawy, T., Hupalowska, A., Rood, J.E., Ashenberg, O., Cerami, E., Coffey, R.J., Demir, E., et al.: The human tumor atlas network: charting tumor transitions across space and time at single-cell resolution. Cell 181(2), 236--249 (2020)

    2020

  16. [24]

    In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

    Shi, Z., Xue, S., Zhu, F., Min, W.: High-resolution spatial transcriptomics from histology images using histosge. In: 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 2402--2407. IEEE (2024)

    2024

  17. [25]

    Nature biotechnology 42(9), 1372--1377 (2024)

    Zhang, D., Schroeder, A., Yan, H., Yang, H., Hu, J., Lee, M.Y., Cho, K.S., Susztak, K., Xu, G.X., Feldman, M.D., et al.: Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology. Nature biotechnology 42(9), 1372--1377 (2024)

    2024

  18. [26]

    arXiv preprint arXiv:2010.04159 (2020)

    Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)

    2020 arXiv

  19. [27]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  20. [28]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.