Pith. sign in

REVIEW 4 major objections 2 minor 31 references

Interpretable Rheumatoid Arthritis Scoring via Anatomy-aware Multiple Instance Learning

T0 review · 4 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a two-stage pipeline—automatic joint cropping followed by attention-based multiple instance learning—can predict Sharp/van der Heijde scores from dual-hand radiographs at PCC 0.945, close to the 0.97 of experienced ra

desk verdict Plausible MIL application to RA scoring, but the 'comparable to radiologists' claim looks like mean regression and depends on unquantified joint detection. read the letter →

arxiv 2508.06218 v1 pith:U4FYWREB submitted 2025-08-08 cs.CV

Zhiyan Bo , Laura C. Coates , Bartlomiej W. Papiez This is my paper
classification cs.CV
keywords RheumatoidarthritisSharp/vanderHeijdescoremultipleinstancelearningattentionmechanismhandradiographsinterpretablemedicalimagingSvdHscoringdeep
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rheumatoid arthritis (RA) damage in hand radiographs is normally quantified with the Sharp/van der Heijde (SvdH) score, but manual scoring is slow and rarely used outside trials. The paper tries to show that a two-stage deep-learning pipeline can produce image-level SvdH scores from dual-hand X-rays that agree closely with trained readers, while also returning attention weights that point to the specific joints driving the score. The authors compare two ways of extracting disease-relevant regions: sampling tiles likely to contain abnormalities, and cropping automatically detected joints. The joint-cropping scheme performs better, reaching Pearson correlation 0.943 (RMSE 15.73) for the best single model and 0.945 (RMSE 15.57) after ensembling, against experienced-radiologist correlation 0.97 (RMSE 18.75). If it holds, this makes SvdH scoring practical in routine clinics and gives clinicians an inspectable machine second reader rather than a black-box number.

What carries the argument

Attention-based multiple instance learning (MIL): the model sees an image as a 'bag' of region patches and learns a weighted combination of patch-level features, where attention weights say which patches the final score depends on. The load-bearing variant is Scheme 2's joint-cropping: automatic joint detection turns each hand into a small set of anatomically meaningful instances, so the attention mechanism has to aggregate exactly the structures a radiologist would grade. This is what carries both the accuracy gain over tile sampling and the interpretability claim.

What would settle it

A direct test: take the model's per-joint attention weights and derive a per-joint contribution score, then compare those with a radiologist's per-joint SvdH sub-scores on the same hands. If per-joint agreement is near zero while the image-level PCC stays 0.945, the interpretability claim is unsupported. Alternatively, perturb the joint crops—shift centers by 5–10 percent of patch width or shrink crops—on a fixed test set; if image-level PCC does not drop materially, the claim that all scoring information lies inside detected joint patches is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that interpretable, image-level SvdH scoring can be achieved by decomposing each dual-hand radiograph into disease-relevant regions and learning to attend over them. Scheme 2, the more effective extraction, crops patches around automatically detected joints and treats those patches as instances in an attention-based multiple instance learning model; the attention over joint patches produces both the image-level prediction and a spatial explanation. On the authors' evaluation, the best single model reaches PCC 0.943 and RMSE 15.73; an ensemble reaches PCC 0.945 and RMSE 15.57, which the paper describes as comparable to experienced radiologists (PCC 0.97, RMSE 18.7

Load-bearing premise

The load-bearing premise is that the automatically detected joint patches contain all the visual information the SvdH score needs; if joint localization fails or damage lies just outside the crop, image-level prediction degrades.

Editorial extensions

If this is right

  • SvdH scoring could be automated at a correlation of about 0.94 with human readers, making trial-grade damage quantification feasible for routine clinic workloads.
  • Because predictions are assembled from per-joint attention weights, a clinician can check that the model's score rests on erosions and joint-space narrowing in the expected joints rather than on spurious correlations.
  • Ensembling several MIL models yields a small, consistent gain (PCC 0.943 to 0.945; RMSE 15.73 to 15.57) and can be adopted without changing the pipeline's interpretability.
  • The two extraction schemes can be compared on the same backbone, so future work can attribute gains specifically to anatomy-aware region selection rather than to model capacity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same attention weights could be renormalized into per-joint damage sub-scores, turning the image-level model into a weakly supervised auto-reader without any additional per-joint labels; the paper does not report such per-joint validation.
  • Because the architecture only needs a region detector plus an attention aggregator, the recipe should transfer to other structured radiographic scores, such as Kellgren-Lawrence knee osteoarthritis grading, wherever joint or landmark detection exists.
  • The model's lower RMSE (15.57) than the reported radiologist RMSE (18.75) is suggestive but not by itself a claim of superhuman accuracy; reader variability and regression-to-the-mean would need to be separated before drawing that conclusion.
  • Clinical deployment would need calibration and uncertainty quantification: correlation and RMSE do not reveal whether errors concentrate at the high-damage end of the SvdH scale, where treatment decisions are most consequential.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 2 minor

Summary. The paper proposes a two-stage pipeline for predicting Sharp/van der Heijde (SvdH) scores from dual-hand radiographs. Stage one extracts disease-relevant image regions: Scheme 1 samples tiles likely to contain abnormalities; Scheme 2 crops patches containing disease-relevant joints. Stage two uses attention-based multiple instance learning to aggregate patch features into an image-level prediction. The authors report PCC 0.943 and RMSE 15.73 for their best individual model and PCC 0.945 and RMSE 15.57 for an ensemble under Scheme 2, claiming state-of-the-art performance comparable to experienced radiologists (PCC 0.97, RMSE 18.75). They further claim that the attention mechanism identifies anatomical structures clinicians consider relevant to RA progression. The supplied full text is heavily corrupted, so the methods and experiments could not be audited beyond the abstract; the following assessment therefore focuses on the abstract and the region-extraction scheme that carries the headline result.

Significance. If the reported results hold and are reproducible, the pipeline would be a clinically relevant step toward automated, interpretable RA scoring. The paper makes a concrete falsifiable performance claim and includes a comparison with radiologists, which is a useful benchmark. The attention-based MIL formulation is reasonable for this problem. However, the evidence currently on display is insufficient to support the headline claims: key details about the joint detector, the radiologist comparison, and the evaluation protocol are missing from the abstract, and the unreadable full text prevents verification. The central contribution is defensible in principle but requires substantial clarification before the claims can be accepted.

major comments (4)
  1. [Region extraction schemes, Scheme 2] The headline result is achieved by Scheme 2, which feeds automatically detected joint patches to the MIL aggregator. The SvdH total score is a sum of erosion and JSN sub-scores over many joints, so any joint missed by the detector removes a component of the label that the model cannot see. The paper does not report per-joint detector recall, an error analysis stratified by detection success, or behavior on severe destruction/deformity, where detection is most likely to fail. If the detector fails preferentially in high-score hands, the reported PCC/RMSE will be optimistically biased exactly in the patients where scoring matters. Please report detector recall per joint category, compare images with and without missed joints, and include an oracle-crop or full-image ablation. If these details exist in the full text, point to the exact section/table.
  2. [Abstract (radiologist comparison)] The claim of being 'comparable to experienced radiologists' rests on radiologist PCC 0.97/RMSE 18.75 versus model PCC 0.945/RMSE 15.57. The abstract does not state whether the radiologists scored the same held-out images, under the same ground-truth definition, or how many readers were involved. Moreover, a lower RMSE with a lower PCC is consistent with mean regression: predictions compressed toward the average can lower RMSE while degrading correlation. Please provide the comparison on identical test images with confidence intervals, a calibration or Bland-Altman plot, and the test-set distribution of SvdH scores.
  3. [Experimental setup] There is a potential leakage/circularity risk in the region-extraction stage. The abstract does not state how the joint detector and the abnormal-tile selector were trained, or whether Scheme 2 was chosen after inspecting test-set performance. If the detector was trained or tuned on the same dataset, or if the choice between Scheme 1 and Scheme 2 was made post hoc on the test set, the reported metrics are optimistically biased. Please clarify the training/validation/test split for all components (detector, tile selector, MIL aggregator, ensemble) and state explicitly that no test labels were used to select the region-extraction scheme. Also report dataset size, number of patients/images, and the severity range.
  4. [Abstract] The abstract reports only point estimates (PCC 0.943/0.945, RMSE 15.73/15.57) with no confidence intervals, sample size, or test-set definition. Without this information the numbers cannot be compared with prior work or assessed for statistical significance. Please include n, bootstrap or parametric confidence intervals, and the standard deviation/range of the SvdH labels, so that the RMSE values can be interpreted relative to the label scale.
minor comments (2)
  1. [Abstract] The phrase 'achieving state-of-the-art performance' should be supported by a citation or a comparison table with prior published results on the same or comparable datasets; as it stands, it is an unsupported superlative.
  2. [Interpretability claims] The statement that the pipeline 'effectively identified and made decisions based on anatomical structures' is only supported qualitatively. A quantitative interpretability evaluation (for example, attention agreement with clinician-drawn regions or ablation of anatomical patches) would strengthen the claim. If such an evaluation appears in the full text, please cite the section explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: reported SvdH predictions come from a supervised MIL pipeline with an external radiologist benchmark; no fitted input is renamed as a prediction.

full rationale

The readable portion (the abstract) describes a two-stage pipeline: extract disease-relevant image regions (tile sampling or joint cropping) and integrate them with attention-based MIL to predict image-level Sharp/van der Heijde scores. Nothing in the abstract states that the SvdH target labels were used to fit or select the region-extraction scheme, nor that the reported PCC/RMSE values were computed on training data. The radiologist comparison (PCC = 0.97, RMSE = 18.75) is an external benchmark, not an input to the model. The supplied full text is heavily corrupted and contains only fragments, so no equation-level reduction (e.g., a fitted parameter renamed as a prediction, or a target label used to define the proposed features) can be exhibited from the paper's own equations. The joint-cropping assumption (that automatically detected joint patches contain all information needed for scoring) is a legitimate robustness concern: missed joints would remove sub-scores from the model's input and could degrade performance in high-damage cases. However, that is a correctness/validation risk, not circularity: a missed joint reduces information; it does not make the output equivalent to the input by construction. The embedded arXiv identifier from an unrelated paper further suggests the supplied text is corrupted, but even the abstract alone provides no quotable self-citation or self-definitional step. Under the rule that circularity must be demonstrated by a specific reduction rather than speculated, the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

From the abstract alone, the only identifiable axioms are domain assumptions about the scoring standard and the MIL aggregation. No free parameters or invented entities are disclosed.

assumptions (2)
  • domain assumption SvdH scores from trained readers are a valid and reliable ground truth for RA damage.
    The model is trained to regress to these scores; if the labels are noisy or biased, reported PCC and RMSE inherit that noise. Implicit in the abstract's evaluation against radiologist scores.
  • domain assumption Attention-based MIL can aggregate localized joint/pathology features into an accurate image-level severity score.
    The two-stage pipeline assumes that the selected regions contain all discriminating information; any information outside those regions is ignored. Stated in the pipeline design (Scheme 1 and 2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interpretable Rheumatoid Arthritis Scoring via Anatomy-aware Multiple Instance Learning." pith.science (2026). https://pith.science/paper/U4FYWREB

@misc{pith2026250806218,
  author       = {Pith},
  title        = {Pith review of: Interpretable Rheumatoid Arthritis Scoring via Anatomy-aware Multiple Instance Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4FYWREB}},
  note         = {Machine review of arXiv:2508.06218}
}
read the original abstract

The Sharp/van der Heijde (SvdH) score has been widely used in clinical trials to quantify radiographic damage in Rheumatoid Arthritis (RA), but its complexity has limited its adoption in routine clinical practice. To address the inefficiency of manual scoring, this work proposes a two-stage pipeline for interpretable image-level SvdH score prediction using dual-hand radiographs. Our approach extracts disease-relevant image regions and integrates them using attention-based multiple instance learning to generate image-level features for prediction. We propose two region extraction schemes: 1) sampling image tiles most likely to contain abnormalities, and 2) cropping patches containing disease-relevant joints. With Scheme 2, our best individual score prediction model achieved a Pearson's correlation coefficient (PCC) of 0.943 and a root mean squared error (RMSE) of 15.73. Ensemble learning further boosted prediction accuracy, yielding a PCC of 0.945 and RMSE of 15.57, achieving state-of-the-art performance that is comparable to that of experienced radiologists (PCC = 0.97, RMSE = 18.75). Finally, our pipeline effectively identified and made decisions based on anatomical structures which clinicians consider relevant to RA progression.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages

  1. [1]

    Seminars in Arthritis and Rheumatism 74 (2025)

    Bird, A., Oakden-Rayner, L., Chakradeo, K., Thomas, R., Gupta, D., Jain, S., Jacob, R., Ray, S., Wechalekar, M.D., Proudman, S., Palmer, L.J.: AI automated radiographic scoring in rheumatoid arthritis: Shedding light on barriers to implementation through comprehensive evaluation. Seminars in Arthritis and Rheumatism 74 (2025). doi:10.1016/j.semarthrit.2025.152761

    work page arXiv 2025

  2. [2]

    Arthritis Research and Therapy 24, 1--10 (2022)

    Bird, A., Oakden-Rayner, L., McMaster, C., Smith, L.A., Zeng, M., Wechalekar, M.D., Ray, S., Proudman, S., Palmer, L.J.: Artificial intelligence and the future of radiographic scoring in rheumatoid arthritis: a viewpoint . Arthritis Research and Therapy 24, 1--10 (2022). doi:10.1186/S13075-022-02972-X

    work page doi:10.1186/s13075-022-02972-x 2022

  3. [3]

    BMJ Best Practice : Rheumatoid arthritis - symptoms, diagnosis and treatment (4 2024), https://bestpractice.bmj.com/topics/en-gb/105

    work page 2024

  4. [4]

    z , B.W.: Deep learning models to automate the scoring of hand radiographs for rheumatoid arthritis

    Bo, Z., Coates, L.C., Papie \. z , B.W.: Deep learning models to automate the scoring of hand radiographs for rheumatoid arthritis. In: Medical Image Understanding and Analysis. pp. 398--413 (2024)

    work page 2024

  5. [5]

    Annals of the Rheumatic Diseases 60, 817 (2001)

    Boini, S., Guillemin, F.: Radiographic scoring methods as outcome measures in rheumatoid arthritis: properties and advantages. Annals of the Rheumatic Diseases 60, 817 (2001). doi:10.1016/s0003-4967(24)43379-7

    work page doi:10.1016/s0003-4967(24)43379-7 2001

  6. [6]

    Frontiers in Medicine 10 (2023)

    Gilvaz, V.J., Reginato, A.M.: Artificial intelligence in rheumatoid arthritis: potential applications and future implications. Frontiers in Medicine 10 (2023). doi:10.3389/FMED.2023.1280312

    work page arXiv 2023

  7. [7]

    IEEE Transactions on Medical Imaging 39, 2584--2594 (2020)

    Han, Z., Wei, B., Hong, Y., Li, T., Cong, J., Zhu, X., Wei, H., Zhang, W.: Accurate Screening of COVID-19 Using Attention-Based Deep 3D Multiple Instance Learning . IEEE Transactions on Medical Imaging 39, 2584--2594 (2020). doi:10.1109/TMI.2020.2996256

    work page arXiv 2020

  8. [8]

    In: Computer Vision and Pattern Recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition. pp. 770--778 (2016)

    work page 2016

Show all 31 references
  1. [9]

    Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine 233, 657--667 (2019)

    Hemalatha, R.J., Vijaybaskar, V., Thamizhvani, T.R.: Automatic localization of anatomical regions in medical ultrasound images of rheumatoid arthritis using deep learning. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine 233, 6...

    2019 doi

  2. [10]

    Rheumatology Advances in Practice 3 (2019)

    Hirano, T., Nishide, M., Nonaka, N., Seita, J., Ebina, K., Sakurada, K., Kumanogoh, A.: Development and validation of a deep-learning model for scoring of radiographic finger joint destruction in rheumatoid arthritis. Rheumatology Advances in Practice 3 (2019). doi:10.1093/RAP/RKZ047

    2019 doi

  3. [11]

    In: 2018 IEEE 4th International Symposium in Robotics and Manufacturing Automation (ROMA) (2018)

    Ho, S., Elamvazuthi, I., Lu, C.: Classification of rheumatoid arthritis using machine learning algorithms. In: 2018 IEEE 4th International Symposium in Robotics and Manufacturing Automation (ROMA) (2018)

    2018

  4. [12]

    Rheumatology 62, 2272--2283 (2023)

    Honda, S., Yano, K., Tanaka, E., Ikari, K., Harigai, M.: Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application . Rheumatology 62, 2272--2283 (2023). doi:10.1093/RHEUMATOLOGY/KEAC586

    2023 doi

  5. [13]

    In: International Conference on Machine Learning

    Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning. pp. 2127--2136. PMLR (2018)

    2018

  6. [14]

    Nature Methods 2020 18:2 18, 203--211 (12 2020)

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation . Nature Methods 2020 18:2 18, 203--211 (12 2020). doi:10.1038/s41592-020-01008-z

    2020 doi

  7. [15]

    Arthritis Research and Therapy 18 (2016)

    Landewé, R.B., Connell, C.A., Bradley, J.D., Wilkinson, B., Gruben, D., Strengholt, S., van der Heijde , D.: Is radiographic progression in modern rheumatoid arthritis trials still a robust outcome? Experience from tofacitinib clinical trials . Arthritis Research and Therapy 1...

    2016 doi

  8. [16]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1862--1874 (2015)

    Lindner, C., Bromiley, P.A., Ionita, M.C., Cootes, T.F.: Robust and accurate shape model matching using random forest regression-voting. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 1862--1874 (2015). doi:10.1109/TPAMI.2014.2382106

    2015

  9. [17]

    Maziarz, K., Krason, A., Wojna, Z.: Deep learning for rheumatoid arthritis: Joint detection and damage scoring in x-rays (2022), https://arxiv.org/abs/2104.13915

    2022 arXiv

  10. [18]

    Rheumatology and Therapy 9, 1249 (2022)

    Momtazmanesh, S., Nowroozi, A., Rezaei, N.: Artificial intelligence in rheumatoid arthritis: Current status and future perspectives: A state-of-the-art review. Rheumatology and Therapy 9, 1249 (2022)

    2022

  11. [19]

    Scientific Reports 2025 15:1 15, 1--14 (2025)

    Moradmand, H., Ren, L.: Multistage deep learning methods for automating radiographic sharp score prediction in rheumatoid arthritis. Scientific Reports 2025 15:1 15, 1--14 (2025). doi:10.1038/s41598-025-86073-0

    2025 doi

  12. [20]

    Radiologia Medica 124, 1071--1086 (2019)

    Salaffi, F., Carotti, M., Beci, G., Carlo, M.D., Giovagnoni, A.: Radiographic scoring methods in rheumatoid arthritis and psoriatic arthritis. Radiologia Medica 124, 1071--1086 (2019). doi:10.1007/S11547-019-01001-3

    2019 doi

  13. [21]

    In: CVPR

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: Inverted residuals and linear bottlenecks. In: CVPR. pp. 4510--4520 (2018)

    2018

  14. [22]

    Medical Image Analysis 79 (2022)

    Su, Z., Tavolara, T.E., Carreno-Galeano, G., Lee, S.J., Gurcan, M.N., Niazi, M.K.: Attention2majority: Weak multiple instance learning for regenerative kidney grading on whole slide images. Medical Image Analysis 79 (2022). doi:10.1016/j.media.2022.102462

    2022

  15. [23]

    Louis, J., Community, R.D.C.: A Crowdsourcing Approach to Develop Machine Learning Models to Quantify Radiographic Joint Damage in Rheumatoid Arthritis

    Sun, D., Nguyen, T.M., Allaway, R.J., Wang, J., Chung, V., Yu, T.V., Mason, M., Dimitrovsky, I., Ericson, L., Li, H., Guan, Y., Israel, A., Olar, A., Pataki, B.A., Stolovitzky, G., Guinney, J., Gulko, P.S., Frazier, M.B., Chen, J.Y., Costello, J.C., Bridges, S. Louis, J., Comm...

    2022

  16. [24]

    Tan, Y.M., Quek, R., Chong, H., Hargreaves, C.A.: Rheumatoid Arthritis: Automated Scoring of Radiographic Joint Damage (2021), https://arxiv.org/abs/2110.08812

    2021 arXiv

  17. [25]

    Rheumatology 38, 941--947 (1999)

    van der Heijde , D., Dankert, T., Nieman, F., Rau, R., Boers, M.: Reliability and sensitivity to change of a simplification of the Sharp/van der Heijde radiological assessment in rheumatoid arthritis . Rheumatology 38, 941--947 (1999). doi:10.1093/RHEUMATOLOGY/38.10.941

    1999 doi

  18. [26]

    Rheumatology 64(3), 1068--1076 (2024)

    Venäläinen, M.S., Biehl, A., Holstila, M., Kuusalo, L., Elo, L.L.: Deep learning enables automatic detection of joint damage progression in rheumatoid arthritis—model development and external validation. Rheumatology 64(3), 1068--1076 (2024). doi:10.1093/rheumatology/keae215

    2024 doi

  19. [27]

    In: MICCAI

    Viriyasaranon, T., Ma, S., Choi, J.H.: Anatomical Landmark Detection Using a Multiresolution Learning Approach with a Hybrid Transformer-CNN Model . In: MICCAI. pp. 433--443 (2023)

    2023

  20. [28]

    Complexity 2022 (2022)

    Wang, Z., Liu, J., Gu, Z., Li, C.: An Efficient CNN for Hand X-Ray Overall Scoring of Rheumatoid Arthritis . Complexity 2022 (2022). doi:10.1155/2022/5485606

    2022 doi

  21. [29]

    WHO: Rheumatoid arthritis (6 2023), https://www.who.int/news-room/fact-sheets/detail/rheumatoid-arthritis

    2023

  22. [30]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  23. [31]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.