{"id":"e871f288-2f17-4464-8c3b-854ecbb05d39","arxiv_id":"2507.19155","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"RegScore adapts beam search and k-sparse ridge regression to build interpretable regression scoring systems, and its personalized multimodal variants match black-box performance on mPAP estimation.","lead":"This paper introduces RegScore, a sparse point-based scoring system that estimates a continuous clinical measurement, mean pulmonary artery pressure, instead of only a risk category. It also builds personalized scoring versions from a multimodal transformer, with accuracy close to black-box deep learning models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Preprocessing leakage from fitting MDLP/tertile bins and f-regression feature selection outside CV could inflate Tables 1-2; nested re-run is needed.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the paper does not specify whether MDLP/tertile discretization and f-regression feature selection are fitted inside each cross-validation fold. This is the most direct threat to the central claim because the headline results hinge on small performance margins, and even modest leakage could change the ranking. I agree with the conditional verdict: the claims are plausible but not fully verifiable from the text alone, and the provided code link makes the nested-preprocessing re-run a feasible and decisive check. I did not find a stronger internal inconsistency. The limitation section acknowledges the interpretability-performance trade-off, which slightly tempers the 'comparable or better' wording but does not undermine the scoring-system comparison. The re-use of the same test set across five folds is a secondary concern that further weakens the reported significance tests, but the preprocessing leakage is the primary issue.","tokens_in":9429,"tokens_out":9353,"duration_ms":92913,"concrete_test":"Inspect the released code at https://github.com/SanoScience/RegScore and re-run the full pipeline with all preprocessing nested inside the CV: for each of the 5 folds, recompute MDLP/tertile bin edges, f-regression feature selection, and standardization on the training fold only; apply the fitted transforms to the validation fold (and to the held-out 261 test set if used). Compare the resulting Tables 1 and 2 with the submitted numbers. If any RegScore/PRS/PLR metric moves outside its reported confidence interval, or if PRS no longer has the top F1 or RegScore no longer beats FasterRisk, the central claim is overstated. Also report results from a single final model evaluated once on the 261 test set rather than five-fold reuse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that RegScore and its personalized bimodal extensions are competitive with or better than black-box models rests on the test metrics in Tables 1 and 2. Those metrics are trustworthy only if every target-dependent preprocessing step is recomputed inside each training fold. Section 3 does not say this. The text says only: 'We select demographic features and MRI-derived measurements with fewer than 500 missing values' and 'retaining only those with statistical significance based on f-regression.' The implementation-details paragraph fixes hyperparameters (B=10, tau=0.1, k=5, lambda2=1e-8, MDLP/tertiles) but never states that MDLP/tertile bin edges, f-regression feature selection, or standardization are fit on the training fold alone. If, as the wording suggests, these are fit once on the full 1790-sample training set before the 5-fold split, each fold's validation labels influence the features and bin boundaries used for training, producing optimistically biased validation/test scores. If they are fit on the full 2051-sample dataset, the 261 test samples' labels directly participate in feature selection and binning, materially overstating generalization. The margins behind the headline are small: PRS F1 (93.84) versus TabAttention (93.59); RegScoreOKR MAE (7.73) versus GBR (7.67) and RF (7.61). A modest leakage-induced bias could flip the 'comparable or better' conclusion. In addition, reporting 'test set over five folds' reuses the same 261 test samples across folds, so the paired t-test p-values are not independent; this weakens the statistical-significance support even if preprocessing is clean.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RegScore, a sparse and interpretable scoring system for regression, obtained by solving a k-sparse ridge regression on binarized tabular features using either beam search (FasterRisk-style) or OKRidge. It then extends this idea to bimodal deep learning by using the CLS token of the TIP transformer to generate Personalized Linear Regression (PLR) weights and a Personalized RegScore (PRS) with a gating mechanism. The method is evaluated on the ASPIRE dataset for estimating mean pulmonary artery pressure (mPAP) and for pulmonary hypertension classification, with comparisons against classical scoring systems, tabular machine learning models, and deep multimodal baselines. The paper claims that RegScore, PLR, and PRS achieve performance comparable to or better than state-of-the-art black-box models while providing interpretability, and it provides a public code repository.","tokens_in":9772,"tokens_out":4022,"duration_ms":46064,"significance":"If the empirical results withstand scrutiny, the contribution is meaningful: it extends interpretable scoring systems from classification to regression, introduces a way to make bimodal deep-learning predictions more transparent through personalized linear scoring, and demonstrates the approach on a clinically relevant dataset with a public code release. The formulations in Section 2 are simple and consistent, and the paper includes ablations, comparisons against multiple baselines, and example scores in Table 4. However, the central empirical claim currently depends on evaluation details that are not fully specified, and the reported margins over many baselines are small; therefore, the significance of the contribution hinges on whether the evaluation protocol is confirmable as leakage-free.","major_comments":[{"comment":"The manuscript does not state that f-regression feature selection, MDLP/tertile discretization, and standardization are fitted only on each training fold. Because f-regression uses the mPAP labels and MDLP uses target information, fitting these steps on the full 1790-sample training set before the 5-fold split would leak validation-fold information into training, and fitting them on all 2051 samples would leak test information; either would optimistically bias the metrics in Tables 1 and 2. This concern is load-bearing because the headline margins are small (e.g., PRS F1 93.84 versus TabAttention 93.59; RegScoreOKR MAE 7.73 versus GBR 7.67 and RF 7.61). The authors should specify the exact preprocessing pipeline and re-run the experiments with all target-dependent preprocessing nested inside each cross-validation fold.","section":"Section 3, Implementation details"},{"comment":"The sentence 'Mean and standard deviation are reported across the test set over five folds' is ambiguous, and the paired t-test p-values appear to compare five fold-trained models on the same 261 test samples. If the same test set is evaluated five times, the paired observations are not independent, so a paired t-test on five folds does not support the claim of a statistically significant difference. The significance claims in Section 4 (e.g., 'RegScore outperformed other scoring systems on the classification task (p-value <0.05)') therefore need either an appropriate test with independent units or a correction for the multiple comparisons across baselines, and the number of independent test observations used for testing should be reported.","section":"Section 3, Implementation details and Tables 1-2"}],"minor_comments":[{"comment":"The phrase 'Webold' appears to be a typo for 'We bold', and several numerical entries have inconsistent spacing (e.g., '66.05± 21.5').","section":"Table 1"},{"comment":"The main text refers to 'Gradient Boosting Decision Trees (GBDT)' but Table 2 labels the method 'GBR'; please unify the naming.","section":"Section 3, Comparison with state-of-the-art methods"},{"comment":"Equation (3) defines the gating scores and soft/hard gates, but the text does not fully specify how the gated binary features are combined with personalized weights to produce the final prediction; one clarifying sentence would improve reproducibility.","section":"Section 2, Personalized RegScore (PRS)"},{"comment":"The right panel begins with an apparent stray label 'Mean i' before the feature names; this should be cleaned up, and the feature abbreviations should be defined in the caption or text.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The central methodological concern is standard but decisive for this version: the preprocessing pipeline is not explicitly nested inside cross-validation, and the statistical significance claims rest on five-fold paired t-tests whose independence assumptions are not established. These issues are fixable within the manuscript's scope by re-running the pipeline with nested preprocessing and revising the significance testing, so major revision is appropriate rather than rejection. The paper would also benefit from a clearer statement of how many independent test samples are used for each comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper proposes RegScore, a sparse linear scoring system for regression, built by porting beam search and OKRidge from the classification score literature, and then adds two personalized heads (PLR, PRS) that generate per-patient regression weights from a TIP transformer's CLS token. That combination is new, and the clinical target—mPAP estimation for pulmonary hypertension—is a genuinely good fit. The paper is not a breakthrough; it is a solid, useful incremental contribution with a real empirical demonstration.\n\nWhat it does well: the method is clearly described, the ablations show each component matters (bins, SSL, images), the comparison set is broad, and code is promised. The RegScore examples in Table 4 are easy to read and clinically plausible. The PLR/PRS idea is the most interesting part: turning a CLS token into personalized linear weights is simple and elegant, and PRS gets the best F1 in Table 2.\n\nWhere I have concerns. The largest is the preprocessing pipeline. The paper says it standardizes features and retains those with statistical significance based on f-regression, but it never states that this—and the MDLP/tertile binning—is done inside each CV fold. If feature selection is fit once on the full 1790-sample training set before the 5-fold split, the validation folds leak into feature selection, biasing model selection. If it is fit on the full 2051 samples, the 261 test labels participate in the pipeline and the test numbers in Tables 1–2 are over-optimistic. The text is genuinely ambiguous; the code should settle it. Second, the reported metrics are \"test set over five folds,\" which means the same 261 test samples are scored five times with overlapping training data. That makes the paired t-test p-values non-independent and overconfident. Third, the intro states PH is defined as mPAP >20, but the classification experiments use >25; the mismatch should be explained. Minor: the claimed \"significant margin\" over FasterRisk is about one F1 point at k=5, which is a modest gap, not a margin.\n\nWho is this for? Researchers working on interpretable clinical models, particularly those who want to extend scoring systems to regression outcomes. It deserves a serious referee, but the revision needs to clarify the CV nesting, ideally re-run the key tables with fully nested preprocessing, and make the code available for inspection.","headline":"A genuinely new but incremental idea—regression scoring systems and personalized CLS-token heads—whose main claims depend on a preprocessing pipeline that must be nested inside CV to be trusted.","tokens_in":10339,"tokens_out":6197,"would_cite":true,"duration_ms":58123,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RegScore shows that sparse, interpretable scoring systems can be built for regression tasks and can match black-box accuracy when estimating mean pulmonary artery pressure.","keywords":["scoring systems","interpretable regression","sparse ridge regression","beam search","pulmonary hypertension","mean pulmonary artery pressure","multimodal deep learning","cardiac MRI"],"falsifier":"Rerun the 5-fold cross-validation with discretization and feature selection recomputed inside each training fold, using the same test folds, and compare MAE and F1; if the numbers shift by more than the reported standard deviations, the reported generalization advantage is partly an artifact of leakage.","tokens_in":9265,"feed_emoji":"🫁","tokens_out":8459,"duration_ms":74267,"temperature":0.7,"pith_summary":"This paper tries to establish that scoring systems—the simple point-based tools clinicians already use for classification—can be built directly for regression tasks without sacrificing accuracy. To do so, it relaxes the integer-only constraint of classical scoring systems and fits a k-sparse ridge regression on discretized features, producing RegScore. It then shows how to make bimodal deep learning models interpretable by using the transformer's learned representation to generate per-sample regression weights (PLR) or per-sample sparse scores (PRS). On the task of estimating mean pulmonary artery pressure from clinical records and cardiac MRI, the paper reports that RegScore matches or exceeds black-box models while remaining fully transparent. If this holds, clinicians could get continuous, trustworthy predictions with a visible decision rule instead of choosing between accuracy and interpretability.","feed_headline":"Sparse scoring system rivals black boxes on pulmonary artery pressure","feed_subtitle":"RegScore estimates pulmonary artery pressure to within about 7.7 mmHg while staying transparent.","key_machinery":"The load-bearing object is the k-sparse ridge regression on a binarized feature matrix, expressed as $\\min_\\beta \\|y - \\hat X \\beta\\|_2^2 + \\lambda_2 \\|\\beta\\|_2^2$ subject to $\\|\\beta\\|_0 \\le k$, which forces the final score to depend on at most k binary conditions. Beam search (adapted from FasterRisk) and OKRidge are the two solvers; the beam-search version assumes that a good size-k model contains a good size-(k−1) model, while OKRidge uses branch-and-bound with lower bounds. For the personalized variants, the machinery shifts to the CLS token of the TIP transformer: a linear layer turns the CLS token into N+1 per-sample weights for PLR, and a gating function with a top-k selection turns it into a sparse per-sample score for PRS.","core_discovery":"RegScore is a sparse, interpretable scoring system for regression: continuous and categorical features are first binarized by discretization (MDLP or tertiles), then a k-sparse ridge regression minimizes squared error while using at most k binary features, with the weights solved by beam search or by the OKRidge branch-and-bound algorithm. On the study's dataset, both variants reach a mean absolute error of about 7.73–7.75 mmHg for mean pulmonary artery pressure, and the same score thresholded at 25 mmHg outperforms the classification scoring systems RiskSLIM, MISS, and FasterRisk in accuracy and F1. For bimodal data, the paper uses the CLS token of a TIP transformer—the learned representation token—to generate personalized linear regression weights per patient (PLR), or to gate the binarized features so that only the top-k contribute to a per-patient score (PRS). PRS achieves the highest F1 of 93.84% among all compared methods, edging out the full black-box TIP model, while PLR stays within 0.3 mmHg of TIP's mean absolute error. The paper's claim is that these sparse, transparent models are competitive with, and sometimes better than, state-of-the-art black-box models.","pith_inferences":["The paper's ablations show that switching the discretization method (MDLP vs tertiles) degrades performance, so the method's success on a new dataset will likely depend on choosing a binning strategy matched to that data.","PLR's per-sample weight vectors could be mined for patient-level feature importance or subgroup patterns, a use the paper only gestures at with mean weights.","A direct test of the claimed generalizability would be to apply RegScore to other continuous clinical endpoints defined by thresholds—for example, ejection fraction or laboratory values—and check whether the accuracy-interpretability balance persists.","Because the paper does not state explicitly that every preprocessing step is refit inside each cross-validation fold, rerunning the pipeline with fold-local discretization and feature selection is a needed check on the size of the reported advantage."],"forward_implications":["RegScore returns a continuous mPAP estimate in mmHg, so a clinician can see exactly which binarized features (e.g. septal angle, reveal score) add or subtract points from the baseline 40.5 mmHg.","At both k=5 and k=50, RegScore outperforms the classification scoring systems RiskSLIM, MISS, and FasterRisk on PH classification, with the differences reaching statistical significance.","PRS reaches the best F1 among all models compared, including black-box multimodal transformers, while remaining interpretable through its per-patient feature gates.","RegScore can be trained in about 55 seconds, making it practical to generate and inspect many sparse models before choosing one.","Larger model sizes k improve both regression and classification performance of RegScore, giving an explicit dial for the interpretability-accuracy trade-off."],"supporting_citations":[{"why":"supplies the beam-search procedure that RegScore adapts from classification scoring to regression","marker":"[17]"},{"why":"provides the OKRidge branch-and-bound algorithm used as the second optimal k-sparse ridge regression solver","marker":"[16]"},{"why":"gives the TIP transformer whose CLS token is repurposed to generate personalized weights and gates for PLR and PRS","marker":"[4]"},{"why":"the RiskSLIM integer-optimization scoring system that serves as a key baseline in the classification comparison","marker":"[23]"},{"why":"the MISS multiclass scoring system used as another comparison baseline in Table 1","marker":"[10]"},{"why":"supplies the MDLP discretization that binarizes continuous features for building RegScore","marker":"[6]"},{"why":"provides the f-regression feature-selection procedure and several tabular machine-learning baselines","marker":"[19]"},{"why":"supplies the clinical dataset of invasive mPAP measurements matched with cardiac MRI used in all experiments","marker":"[14]"}],"fun_headline_variants":["RegScore: Sparse interpretable scoring that rivals black-box models","Interpretable regression scores match deep models on lung pressure","Sparse white-box scores give black-box accuracy for pulmonary pressure","Personalized scoring for regression performs on par with black boxes","Transparent regression scoring system matches black-box performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that discretization bins, missing-value thresholds, and f-regression feature selection are all fitted on the training folds alone and never on the held-out test set, a detail the paper does not explicitly confirm.","fun_headline_variants_meta":{"raw":{"variants":["RegScore: Sparse interpretable scoring that rivals black-box models","Interpretable regression scores match deep models on lung pressure","Sparse white-box scores give black-box accuracy for pulmonary pressure","Personalized scoring for regression performs on par with black boxes","Transparent regression scoring system matches black-box performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001379,"raw_usage":{"total_tokens":5609,"prompt_tokens":994,"completion_tokens":4615,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":4534}},"tokens_in":610,"tokens_out":4615,"duration_ms":34200,"temperature":1.0,"reasoning_tokens":4534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:59:32.747007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the 5-fold cross-validation with discretization and feature selection recomputed inside each training fold, using the same test folds, and compare MAE and F1; if the numbers shift by more than the reported standard deviations, the reported generalization advantage is partly an artifact of leakage.","supporting_citations":[{"cited_title":"In: Proceedings of Neural Information Processing Systems (2022)","cited_arxiv_id":null,"evidence_quote":"supplies the beam-search procedure that RegScore adapts from classification scoring to regression"},{"cited_title":"In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S","cited_arxiv_id":null,"evidence_quote":"provides the OKRidge branch-and-bound algorithm used as the second optimal k-sparse ridge regression solver"},{"cited_title":"In: 18th European Conference on Computer Vision (ECCV 2024) (2024)","cited_arxiv_id":null,"evidence_quote":"gives the TIP transformer whose CLS token is repurposed to generate personalized weights and gates for PLR and PRS"},{"cited_title":"Journal of Machine Learning Research 20(150), 1–75 (2019)","cited_arxiv_id":null,"evidence_quote":"the RiskSLIM integer-optimization scoring system that serves as a key baseline in the classification comparison"},{"cited_title":"In: Proceedings of the 2024 SIAM International Conference on Data Mining (SDM)","cited_arxiv_id":null,"evidence_quote":"the MISS multiclass scoring system used as another comparison baseline in Table 1"},{"cited_title":"In: Ijcai","cited_arxiv_id":null,"evidence_quote":"supplies the MDLP discretization that binarizes continuous features for building RegScore"},{"cited_title":"European Respiratory Journal 39, 945–955 (4 2012)","cited_arxiv_id":null,"evidence_quote":"supplies the clinical dataset of invasive mPAP measurements matched with cardiac MRI used in all experiments"}],"review_version":2}