{"id":"d2693efa-bdca-4352-b671-dc993cc2c68f","arxiv_id":"1908.07736","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An adaptive superpixel-based region of interest improves texture-based detection of knee osteoarthritis, but part of the reported gain is inflated by selecting the region on the same data used for evaluation.","lead":"This paper tests whether the location and shape of the region analyzed in knee X-rays changes how well texture features detect osteoarthritis. The authors find that placing the analysis region at the inner edge of the shinbone improves detection accuracy, with local binary patterns performing best.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The tested ROI is a fixed OAI-derived average mask, not per-subject adaptive segmentation; without a per-subject comparison, the adaptive-ROI claim is not validated.","rationale":"The reader's weakest assumption identifies the same core issue: a single fixed mask, derived from an OAI-average of the LBP-selected t26 region, is used for all subjects and all experiments, and the paper does not demonstrate that this fixed mask represents per-subject adaptive segmentation. My stress-test agrees with this and sharpens it: the method described as 'adaptive' is not what is evaluated. The per-subject SLIC step is abandoned after Figure 4, so the reported AUC gains apply only to a static mask. This is load-bearing because if individual masks vary across subjects, the headline claim that adaptive segmentation improves texture-based OA classification is untested. The external OAI-to-MOST validation is real evidence for the fixed mask and should be credited, but it does not fix the mismatch between the method narrative and the evaluated procedure. The reader's conditional verdict is therefore appropriate; my analysis does not move it to accept or reject. The proposed test directly compares per-subject adaptive masks against the fixed average mask and would settle whether the concern lands.","tokens_in":20418,"tokens_out":7909,"duration_ms":86886,"concrete_test":"On a random subset of at least 500 OAI knees, run the full per-subject pipeline: compute SLIC superpixels, identify each subject's t26 superpixel (or the per-subject best region using the same LBP grid-selection rule), extract LBP features inside the per-subject mask, and train the same logistic regression with the same subject-wise folds. Compare the AUC to the same pipeline using the fixed thresholded-average mask. If the per-subject mask AUC is not higher than the fixed-mask AUC, the central claim that adaptive segmentation is beneficial is not supported; if it is materially higher, the fixed-mask results understate the method. Also report the Dice overlap between individual t26 masks and the thresholded average mask to test the 'large overlap' assertion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the reported 'adaptive ROI' results do not actually test the adaptive procedure. In Section 2.3, after selecting t26 from the LBP-based grid search on OAI, the authors replace per-subject SLIC segmentation with a single mask obtained by Otsu-thresholding the OAI-average of the t26 masks and use that fixed mask 'in the rest of the experiments.' All result tables (2, 3, 4, S2, S3) therefore evaluate a fixed, non-adaptive mask rather than the per-subject adaptive ROIs described in the method. The paper states that there is 'a large portion of overlap between the segmentation masks' but gives no overlap statistics. If individual t26 masks vary across subjects due to anatomical variation, landmark detection errors, or acquisition differences, the averaged and thresholded mask may not be near-optimal for many subjects, and the reported gains over the standard ROI would not reflect the actual adaptive method. The authors explicitly acknowledge this in the Discussion: using the average mask 'could lower the performance.' The external OAI-to-MOST validation provides genuine evidence for this specific fixed mask, but it does not validate the adaptive segmentation claim. The internal OAI results are additionally optimistic because the same OAI labels were used both to select the mask and to report cross-validated improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes a fully automatic method for selecting a region of interest (ROI) for texture analysis of subchondral bone in knee radiographs, with the aim of improving OA-versus-non-OA classification. The method uses SLIC superpixel oversegmentation, places a dense grid over the tibia and femur, evaluates each superpixel region using LBP features, and then identifies the most informative region. In the experiments, however, the per-subject adaptive masks are replaced by a single mask obtained by Otsu-thresholding the OAI-average of the t26 masks, and this fixed mask is used for all subsequent evaluations. The authors compare five texture descriptors (LBP, fractal dimension, Haralick features, Shannon entropy, HOG) within this adaptive-mask ROI versus a standard rectangular ROI, using logistic regression in five-fold cross-validation on OAI and MOST, plus an independent OAI-to-MOST transfer experiment. They report that the adaptive ROI improves classification for LBP, HOG, and Haralick features, with the best result on the external validation being LBP features combined from medial and lateral tibia regions at AUC 0.840 [0.825, 0.852] (Table S3).","tokens_in":20647,"tokens_out":3231,"duration_ms":32157,"significance":"If the claims are substantiated, the paper would provide a useful addition to the literature on ROI placement in bone texture analysis, and the large-scale independent OAI-to-MOST validation is a genuine strength. The finding that LBP and HOG outperform fractal dimension and entropy in this setting is consistent with a broader trend and is supported by the external experiment. The paper also ships a clear, reproducible-looking pipeline description. However, the central 'adaptive segmentation' claim is weakened by the fact that all reported results use a fixed average mask rather than per-subject adaptive masks, and the OAI cross-validation results are potentially optimistic because the same OAI labels were used both to select the mask and to report the improvements. The external validation supports the fixed-mask comparison specifically, but it does not by itself validate the adaptive procedure.","major_comments":[{"comment":"The central claim of the paper is that adaptive segmentation improves ROI selection, but the experiments do not actually evaluate per-subject adaptive masks. In Section 2.3, after selecting the t26 region, the authors state: 'Instead of segmenting the tibia of each individual subject, we employed the mask which was obtained by thresholding the average mask using Otsu's method.' This fixed mask is then used 'in the rest of the experiments,' meaning Tables 2, 3, 4, S2, and S3 all compare a fixed, non-adaptive mask against the standard ROI. The paper asserts that 'there is a large portion of overlap between the segmentation masks' but provides no overlap statistics and no comparison of per-subject adaptive masks versus the fixed average mask. The Discussion acknowledges that using the average mask 'could lower the performance,' but this admitted limitation is not quantified. To support the adaptive-segmentation claim, the authors should report results for the per-subject adaptive masks and compare them with the fixed average mask and the standard ROI, or alternatively reframe the contribution as a fixed, data-driven ROI and temper the title and abstract accordingly.","section":"Section 2.3, Figure 4"},{"comment":"The OAI cross-validation results in Table 2 are optimistic because the t26 mask was selected by maximizing LBP classification performance on the OAI data, and the same OAI data are then used to report the cross-validated improvements. This is a form of selection on the evaluation set: the mask parameters are effectively fit to the OAI labels, so the reported gains of up to 9% in AUC over the standard ROI on OAI are not an unbiased estimate of the method's performance on new data. The external OAI-to-MOST experiment (Table 4) provides more trustworthy evidence for the fixed mask, and it is a genuine strength of the paper, but the internal OAI numbers should be presented as a model-selection result rather than as an independent evaluation, or the authors should use nested cross-validation so that mask selection occurs inside each training fold.","section":"Section 2.3, Table 2"},{"comment":"The claim of consistent improvement on MOST cross-validation is not fully supported by the reported confidence intervals. For LBP, the standard-ROI AUC is 0.778 [0.760, 0.792] and the adaptive-mask ROI AUC is 0.797 [0.781, 0.812]; these intervals overlap, so the observed difference is not statistically significant at the usual 95% level. A similar concern applies to several other comparisons where intervals are close. The authors state that 'scores for adaptive ROI are better except Entropy descriptor' without reporting any paired significance test or effect-size measure. Given the large sample sizes, the lack of significance testing is a notable omission. I recommend adding a paired statistical comparison (e.g., DeLong test for AUC or a bootstrap paired test) and discussing which improvements are statistically reliable.","section":"Section 3, Table 3"}],"minor_comments":[{"comment":"The Fractal AUC confidence interval is missing a closing bracket: '0.664 [0.652, 0.675' should be '0.664 [0.652, 0.675]'.","section":"Table 2"},{"comment":"In the sentence about OAI and MOST details, 'Future details' should be 'Further details'.","section":"Section 2.1"},{"comment":"In the paragraph discussing validation results, 'different distribution and amount of of training samples' contains a duplicated 'of'.","section":"Section 4"},{"comment":"The text says 'ROC AUC which is also calledc− index'; this should be 'c-index' with proper spacing and formatting.","section":"Section 2.6"},{"comment":"The HOG 'Cells per block' row lists best value as '4,4' and 'Pixels per cell' as '10,10'; these should be written as tuples (4,4) and (10,10) for consistency with the search-space notation.","section":"Supplementary, Table S1"},{"comment":"The figure caption contains garbled text fragments such as '/uni00000021/...' and 'lef'; these appear to be encoding artifacts and should be replaced with the intended caption text.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper has a genuine external validation result that is likely to be of interest to the OAI/MOST community, and the descriptor comparison is informative. However, the mismatch between the title's 'adaptive segmentation' claim and the actual fixed-mask evaluation is a substantive issue that requires either a substantial revision of the claims or an additional per-subject experiment. I would support reconsideration after major revision, but not acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this one. First, the external-validation result is real: training on OAI and testing on MOST, a mask at the medial tibial margin clearly beats the standard rectangular ROI for LBP, HOG, and Haralick texture features. That is the solid core of the paper. Second, despite the title, the deployed ROI is not adaptive: the per-subject superpixel masks were averaged over all subjects and thresholded with Otsu, producing one fixed mask used in every experiment. The adaptive search was a development tool, not the evaluated method.\n\nWhat is new: prior work used fixed or lattice ROIs, and nobody in the cited literature ran a superpixel-based search for the most informative subchondral bone region. The finding that the medial tibial margin carries the discriminative texture information is plausible and consistent with OA pathobiology. The authors also treated the standard ROI fairly — they tuned parameters on it and reused those values for the adaptive mask, which biases against them, not for them.\n\nThe soft spots are real but not fatal. The OAI cross-validation numbers (\"up to 9% AUC\") are optimistic because the t26 mask was selected on OAI labels using LBP, then evaluated on the same OAI labels; that is selection, not pure validation. The OAI-to-MOST results are the honest numbers, and they still show clear gains for three of five descriptors, so the central claim holds. The descriptor comparison is also tilted toward LBP, since LBP drove the ROI choice; HOG and Haralick improving under an LBP-derived mask is reassuring but weakens the \"LBP is best\" conclusion. Reproducibility is a genuine gap: no code, no exact mask-selection protocol, and no overlap statistics despite the claim of a \"large portion of overlap.\" To their credit, the authors disclose the average-mask simplification, parameter sensitivity, and landmark dependence in the Discussion.\n\nWho this is for: the OA imaging biomarker community. As a claim about adaptive segmentation it overstates what was tested; as a claim about data-driven ROI selection it holds up and sharpens a widely used biomarker.\n\nSend it to peer review. Ask for a re-framed contribution (fixed data-driven ROI discovery), a per-subject adaptive comparison or an explicit justification of its absence, and release of code and the exact mask-selection protocol. Those changes would make it a solid, reproducible contribution.","headline":"The external OAI-to-MOST validation is genuine and the central finding survives — but the deployed ROI is a fixed group-average mask, not an adaptive per-subject segmentation, and the OAI headline gains are selection-fitted; worth engaging, worth revising.","tokens_in":21238,"tokens_out":3009,"would_cite":true,"duration_ms":31185,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An automatically segmented medial-tibia region improves knee OA detection from radiograph texture over a fixed rectangular ROI.","keywords":["knee osteoarthritis","bone texture analysis","adaptive region of interest","knee radiograph","subchondral bone","local binary patterns","superpixel segmentation","texture descriptors"],"falsifier":"Recompute the experiments using each subject's own segmented region instead of the fixed average mask; if per-subject masks fail to beat or match the fixed t26 mask, or if a different region such as the lateral margin (t3) outperforms t26 on an independent cohort, then the claim that t26 is the optimal ROI for all settings is falsified.","tokens_in":20173,"feed_emoji":"🦴","tokens_out":10631,"duration_ms":145384,"temperature":0.7,"pith_summary":"This paper argues that the region of interest (ROI) used for bone texture analysis in knee radiographs should not be a fixed rectangle placed beneath the tibial plateau. The authors propose a fully automatic pipeline that oversegments the bone into superpixels, uses local binary patterns (LBP) to find which subregion best separates osteoarthritic from non-osteoarthritic knees, and then replaces per-subject segmentation with a single mask derived by averaging and thresholding that most informative region across training subjects. They report that this adaptive mask, located at the medial tibia margin, improves OA-versus-non-OA classification over the standard ROI for LBP, HOG, and Haralick descriptors, with cross-validated AUC gains of up to about 9 percentage points. The best result, an external-validation AUC of 0.840 [0.825, 0.852], is achieved by LBP features combined from medial and lateral tibia regions. The practical significance is that a cheap, fully automatic choice of where to measure texture could make radiograph-based OA biomarkers more sensitive without changing the imaging protocol.","feed_headline":"Adaptive ROI mask lifts knee OA detection on X-rays","feed_subtitle":"The best LBP model, fusing medial and lateral tibia margins, reaches AUC 0.840 on an independent test set.","key_machinery":"The load-bearing mechanism is SLIC superpixel oversegmentation, which partitions the tibia and femur into compact regions that follow local intensity boundaries, combined with an LBP-based search over a dense grid to find the region where a logistic-regression classifier best separates OA from non-OA. Averaging the masks of the winning region (t26) over all subjects and thresholding with Otsu's method produces one fixed 'adaptive mask' that is then used in all subsequent experiments instead of per-subject segmentation. The standard ROI against which it is compared is a square patch immediately beneath the tibial plateau, sized proportionally to knee width.","core_discovery":"The paper's central claim is that the most informative subchondral bone region for detecting radiographic knee OA is the outer medial tibia margin, the site where osteophytes typically form, and that a single adaptive mask derived from that region outperforms the standard rectangular ROI. This holds for LBP, HOG, and Haralick texture features under cross-validation on two large public cohorts and under train-on-one-test-on-the-other external validation; it does not hold for fractal dimension and Shannon entropy, whose results are mixed or worse. The authors further show that combining the same descriptor from the medial (t26) and lateral (t3) tibia margins adds complementary information, with LBP medial-plus-lateral reaching AUC 0.840 [0.825, 0.852] on the independent test set.","pith_inferences":["If the most informative region varies from patient to patient, a per-subject adaptive mask could perform even better than the fixed t26 average mask; the paper itself notes that replacing per-subject segmentation with the average mask may lower performance, so a direct comparison would quantify that loss.","Because only LBP was used to select the informative region, the same search could be repeated with HOG or a learned deep feature; a different descriptor might nominate a different region and outperform t26 for that descriptor.","The finding that tibial margins carry the discriminative signal suggests that OA texture biomarkers should intentionally include marginal and subchondral regions where osteophytes form, which may also apply to other joints or to predicting OA progression rather than prevalence.","A third-cohort test with different acquisition settings, such as beam angle, resolution, or post-processing, would show whether the mask's advantage is tied to the specific preprocessing used here or is a general property of marginal bone texture."],"forward_implications":["Switching from the standard rectangular ROI to the adaptive t26 mask raises 5-fold cross-validated AUC for LBP on the first cohort from 0.685 to 0.761 and for HOG from 0.651 to 0.742.","On the external test set, the adaptive mask improves AUC for LBP (0.818), HOG (0.800), and Haralick (0.762) relative to the standard ROI, showing that the gain is not limited to the training cohort.","Combining LBP features from the medial (t26) and lateral (t3) tibia margins yields the best overall result, with an external AUC of 0.840 [0.825, 0.852], indicating that the two sides carry complementary OA-related texture information.","Fractal dimension and Shannon entropy do not consistently benefit from the adaptive ROI and sometimes perform worse, so the ROI advantage is descriptor-dependent rather than universal."],"supporting_citations":[{"why":"Supplies the SLIC superpixel oversegmentation method used to partition bone into boundary-following regions.","marker":"38"},{"why":"Supplies the automatic landmark localization used for preprocessing and for placing the standard ROI.","marker":"37"},{"why":"Defines local binary patterns, the descriptor used both to detect the most informative region and to classify OA.","marker":"42"},{"why":"Earlier evidence that ROI placement changes OA prediction, motivating the adaptive search over regions.","marker":"27"},{"why":"Establishes the standard subchondral bone texture analysis approach and the rectangular ROI convention used as baseline.","marker":"3"}],"fun_headline_variants":["Adaptive ROI boosts knee OA detection on X-rays","Medial tibia margin ROI improves knee OA texture analysis","Adaptive segmentation finds best ROI for knee OA on X-rays","Outer medial tibia ROI key to better knee OA detection","Adaptive ROI beats standard in knee OA texture analysis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that one fixed analysis region, produced by averaging and thresholding the most informative superpixel mask across training subjects, remains optimal for every subject, every texture descriptor, and every acquisition protocol.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive ROI boosts knee OA detection on X-rays","Medial tibia margin ROI improves knee OA texture analysis","Adaptive segmentation finds best ROI for knee OA on X-rays","Outer medial tibia ROI key to better knee OA detection","Adaptive ROI beats standard in knee OA texture analysis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2966,"prompt_tokens":1065,"completion_tokens":1901,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":1820}},"tokens_in":681,"tokens_out":1901,"duration_ms":540390,"temperature":1.0,"reasoning_tokens":1820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:57:19.115681+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the experiments using each subject's own segmented region instead of the fixed average mask; if per-subject masks fail to beat or match the fixed t26 mask, or if a different region such as the lateral margin (t3) outperforms t26 on an independent cohort, then the claim that t26 is the optimal ROI for all settings is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SLIC superpixel oversegmentation method used to partition bone into boundary-following regions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the automatic landmark localization used for preprocessing and for placing the standard ROI."},{"cited_title":"& M¨aenp¨a¨a, T","cited_arxiv_id":null,"evidence_quote":"Defines local binary patterns, the descriptor used both to detect the most informative region and to classify OA."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier evidence that ROI placement changes OA prediction, motivating the adaptive search over regions."},{"cited_title":"& Buckland-Wright, J","cited_arxiv_id":null,"evidence_quote":"Establishes the standard subchondral bone texture analysis approach and the rectangular ROI convention used as baseline."}],"review_version":1}