{"id":"fd9d1bc6-038e-47d6-95b6-6c0e6b9b1ae1","arxiv_id":"2506.03185","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"DLiPath is presented as the first public donor-liver histopathology benchmark, with expert-annotated whole-slide images and nine MIL baselines.","lead":"The authors introduce DLiPath, a set of 636 donor-liver whole-slide images with expert labels for six features, and they benchmark nine multiple-instance learning models on it. The dataset could be a useful standard for automated donor liver assessment, but internal inconsistencies in the reported labels need scrutiny.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 9 label counts sum to ~261–263 per feature, not 636 or 304, so the paper's central claim that DLiPath contains 636 expert-annotated WSIs is internally contradicted.","rationale":"The reader's weakest assumption—that every claimed WSI has reliable expert labels for each of the six features—is exactly the load-bearing concern. The manuscript's own Table 9 shows label counts summing to 261–263 per feature, not 636 or 304, and Section 6 admits annotation subjectivity. This is an internal inconsistency, not merely a disagreement with external consensus, so it directly threatens the benchmark's central value and the validity of all baseline scores. My independent read confirms the reader's REJECT verdict: until the label inventory is corrected or the dataset is shown to contain complete annotations for the claimed 636 WSIs, the paper's central claim cannot be accepted. The concrete test is straightforward because the repository is public; running it would settle whether the discrepancy is a documentation error or a fundamental data defect. I found no need to adjust the verdict, as the reader's rejection already reflects this concern.","tokens_in":14821,"tokens_out":2415,"duration_ms":29575,"concrete_test":"Download the public repository at https://github.com/panliangrui/ACM_MM_2025, enumerate all WSI files and all label files, and count the number of WSIs that have a grade for each of the six features. Compare these counts against Table 9 and against the claimed 636 WSIs / 304 patients. Also run the provided data-loading and split code to print the actual dataset length used in each cross-validation fold for Tables 2–7. If the label counts sum to ~261–263, or if the effective N in training differs from the reported N, the central benchmark claim fails. If the repository does not contain the full label inventory, the claim is unverifiable from the released artifacts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that DLiPath is the first comprehensive donor-liver benchmark, comprising 636 WSIs from 304 patients, all expert-annotated for six pathological features, on which nine MIL baselines are evaluated. The load-bearing premise is that the label inventory is complete for these six features. The paper's own Appendix A.2 (Table 9) contradicts this premise: the grade counts per feature sum to roughly 261–263, not 636 or 304. For example, cholestasis grades sum to 131+128+2+0 = 261; portal tract fibrosis to 187+76+0+0 = 263; hepatocellular ballooning to 3+45+133+81 = 262. No explanation is given for why roughly 373 WSIs are absent from the label distribution, nor how five-fold cross-validation (Appendix A.3) is run on data whose annotated size is inconsistent with the stated corpus. If the released labels cover only about 263 WSIs, then all baseline results in Tables 2–7 are computed on an undocumented subset, and the headline claim of a 636-WSI benchmark with expert annotations is unsupported. If Table 9 is simply misreported, the documentation is still unreliable until corrected. Section 6 admits that grading is subjective and inconsistent, especially on low-quality WSIs, which further undermines label reliability; however, the internal count mismatch is the more immediate, checkable failure because it is present within the manuscript itself and directly affects every downstream result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces DLiPath, described as the first public benchmark for donor liver assessment from histopathology whole-slide images. The authors report collecting 636 WSIs from 304 donor liver patients, with expert annotations for cholestasis, portal tract fibrosis, portal inflammation, total steatosis, macrovesicular steatosis, and hepatocellular ballooning. Nine multiple-instance learning baselines are evaluated on six four-grade classification tasks using five-fold cross-validation, with TransMIL reported as the strongest model. The paper also states that data and code are available at a GitHub repository.","tokens_in":15093,"tokens_out":7401,"duration_ms":83942,"significance":"If the dataset and annotations were as described, DLiPath would fill a genuine gap: no existing public benchmark targets donor liver assessment, and the six features are clinically relevant. The use of nine established MIL baselines and the intent to publicly release data and code are strengths. However, the manuscript's own label distribution (Table 9) contradicts the stated dataset size, and the reported experimental results contain internal metric inconsistencies and are presented without uncertainty estimates. As a result, the central contribution—a reliable 636-WSI expert-annotated benchmark with trustworthy baseline results—is not supported by the evidence in the manuscript.","major_comments":[{"comment":"The label counts in Table 9 sum to approximately 261–263 per feature (cholestasis 131+128+2+0=261; portal tract fibrosis 187+76+0+0=263; portal inflammation 233+29+1+0=263; total steatosis 160+93+10+0=263; macrovesicular steatosis 184+66+12+1=263; hepatocellular ballooning 3+45+133+81=262), not the 636 WSIs or 304 patients claimed in the Abstract and Section 3.1. This directly contradicts the statement that every WSI in DLiPath is expert-annotated, and it implies that Tables 2–7 may be computed on an undocumented subset of roughly 41% of the corpus. The manuscript must state the exact number of annotated WSIs per feature, explain the discrepancy, and re-run all baselines on the actual annotated corpus.","section":"Section 3.1 and Table 9"},{"comment":"The evaluation protocol does not specify whether five-fold cross-validation splits are at WSI level or patient level. Since 636 WSIs come from 304 patients (about 2.1 WSIs per patient on average), WSI-level random splitting can place sections from the same donor in both training and test folds, inflating the reported accuracies. Please describe the split and use patient-level stratification.","section":"Appendix A.3 and Section 3.1"},{"comment":"Metrics are reported as means over five folds without standard deviations or any uncertainty estimate, despite extreme class imbalance: cholestasis has only 2 moderate and 0 severe cases; portal tract fibrosis has 0 moderate and 0 severe cases; portal inflammation has 1 moderate and 0 severe cases; total steatosis has 0 severe cases; macrovesicular steatosis has 1 severe case. Several reported AUCs are below the random baseline (e.g., ABMIL cholestasis AUC=0.4510, ABMIL portal inflammation AUC=0.3675, ABMIL macrovesicular AUC=0.4748, ILRA hepatocellular AUC=0.4761), which is not discussed. The claim that models achieve high accuracy is therefore not supported without additional per-fold results and a clear handling of empty or missing classes.","section":"Tables 2–7 and Section 4.3"},{"comment":"Some reported metrics are internally inconsistent. In Table 5, ABMIL has Precision=0.5694 and Recall=0.9837, which gives F1≈0.721, not the reported 0.6781. In Table 7, the ILRA row reports Recall=0.6845 and F1=0.8715, while the text states Recall=0.8715 and F1=0.5859; neither F1 is consistent with the stated precision and recall. In addition, the Section 5.2 sentence on DGRMIL and DSMIL accuracies (0.8955 and 0.9113) does not match Table 3, where DSMIL's accuracy is 0.8577. Such inconsistencies undermine confidence in the quantitative baseline comparisons.","section":"Tables 5 and 7"},{"comment":"The authors acknowledge that histopathological grading is subjective and inconsistent, particularly on low-quality WSIs, yet no inter-observer agreement statistic (e.g., kappa) or adjudication reliability measure is reported. Given the central role of expert labels in a benchmark, the absence of label-quality metrics is a substantive gap; please provide agreement or consensus statistics and a per-feature label reliability analysis.","section":"Section 6"}],"minor_comments":[{"comment":"There is a typo in 'histopathological asseassessment'; please correct it.","section":"Section 3.2"},{"comment":"The heading 'Post-mortem Donor Liver Biopsy' is confusing because donor liver sections are not necessarily post-mortem; please align the terminology with the rest of the text.","section":"Table 1"},{"comment":"The softmax equation is garbled in the provided text; please ensure proper typesetting of the normalization term.","section":"Section 4.1"},{"comment":"Table 9 should be referenced in the main text so that readers can locate the label distribution and understand how it relates to the claimed 636 WSIs.","section":"Appendix A.2"}],"recommendation":"reject","confidential_remarks":"The decisive issue is the Table 9 count mismatch: the paper's own label inventory sums to about 261–263 per feature instead of 636 WSIs, and no explanation is offered. Combined with the metric inconsistencies and the lack of patient-level splitting, I do not see how the central benchmark claims can stand as written. If the authors can provide the actual per-feature annotated slide counts and re-run the experiments with patient-level evaluation, a new submission would be worth considering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the clinical gap DLiPath targets is real and the dataset idea is right, but as written, the paper's own Table 9 contradicts the headline claim of 636 annotated WSIs, and that mismatch is load-bearing. I would not use any of the reported numbers until the authors explain it.\n\nWhat's new: a public donor-liver WSI benchmark with six histologic features and nine MIL baselines would be a useful addition. The related-work section correctly notes that existing liver datasets are mostly HCC. The use of CTransPath features and standard MIL models is reproducible in principle, and the limitation section is refreshingly honest about subjectivity and class imbalance.\n\nThe problem: in Appendix A.2, the label counts per feature sum to roughly 261-263, not 636 or 304. For cholestasis, 131+128+2+0=261; portal tract fibrosis 187+76=263; ballooning 3+45+133+81=262. There is no footnote explaining that some WSIs lack labels for a feature. The abstract and contributions say every WSI is expert-annotated for all six features. That is a direct internal contradiction. If the released labels actually cover only ~260 slides, then all baseline results in Tables 2-7 were computed on an undocumented subset, and the benchmark's value is much smaller than claimed. If Table 9 is simply misreported, the documentation is still unreliable. Either way, the central claim is unsupported until corrected.\n\nSecondary issues: no error bars despite five-fold CV; several AUCs are below 0.5 (ABMIL on cholestasis, portal inflammation, macrovesicular steatosis; ILRA on ballooning), which the narrative glosses over. The 20-30% graft discard reduction in the conclusion comes from nowhere. The abstract says data are publicly released, but contribution 1 says original WSIs are 'available upon request'—which one is it? These are minor-to-moderate issues once Table 9 is fixed, but they reinforce the impression of careless reporting.\n\nWho it's for: computational pathology researchers who need a donor-liver benchmark. A reader can get value from the task formulation and baseline setup, but only after the label inventory is clarified. I would want to see the actual label CSV before trusting any number.\n\nRecommendation: I would not desk reject this outright, but I would send it to review with a strong request for the authors to reconcile Table 9 with the stated corpus, release the full label metadata, and add error bars. If they cannot explain the discrepancy, it is a reject. As is, the paper is not ready for use.","headline":"The donor-liver benchmark is a good idea, but the paper's own Table 9 contradicts the 636-WSI claim, so none of the baseline numbers can be trusted until the label inventory is reconciled.","tokens_in":15634,"tokens_out":3139,"would_cite":false,"duration_ms":38920,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DLiPath is introduced as the first public benchmark that grades donor-liver whole-slide images on six pathological features and shows weak-supervision models can score most of them accurately.","keywords":["donor liver","histopathology images","whole slide images","benchmark","multiple instance learning","steatosis","liver transplantation","pathological grading"],"falsifier":"Re-grade a stratified random sample of slides with two independent pathologists using the paper's grading criteria: if weighted agreement is low (for example, kappa below 0.4), or if release inspection shows many slides missing labels for one or more features, then the reported baseline accuracies rest on labels that cannot carry them. A direct starting check is to reconcile the appendix's per-feature counts of about 260 with the claimed 636 labeled slides.","tokens_in":14620,"feed_emoji":"🔬","tokens_out":11213,"duration_ms":123954,"temperature":0.7,"pith_summary":"DLiPath is a new public benchmark for donor-liver assessment built from 636 whole-slide images of 304 donor livers, with expert four-grade labels for cholestasis, portal tract fibrosis, portal inflammation, total steatosis, macrovesicular steatosis, and hepatocellular ballooning. The paper's central claim is that this is the first benchmark tailored to donor-liver histopathology, where existing public liver datasets concentrate on cancer, and that weak-supervision models can grade these six features from slide-level labels alone. On this dataset, several attention-based multiple-instance learning models reach high accuracy, above 0.93 on cholestasis and fibrosis and above 0.98 on total steatosis, while hepatocellular ballooning proves more difficult. If the benchmark holds up, it gives transplant pathology a shared dataset for building rapid, objective intraoperative graft assessment and a concrete target for the hardest indicators.","feed_headline":"Donor livers get a public benchmark for six graft-risk indicators","feed_subtitle":"With 636 expert-graded donor-liver slides, automated models can flag steatosis, fibrosis, cholestasis, and more.","key_machinery":"The load-bearing mechanism is a weak-supervision pipeline from tissue to label. A whole-slide image is tiled into 256x256 patches, each patch is embedded into a 768-dimensional vector by a pretrained transformer-based histopathology feature extractor, and the slide becomes a bag of these patch embeddings. Attention-based multiple-instance learning aggregators then weight the instances and produce the four-grade classification for each of the six indicators. The key point is that only slide-level expert labels are required, not patch-level annotations, which is what makes large-scale donor-liver benchmarking practical.","core_discovery":"DLiPath is presented as the first public benchmark for comprehensive donor liver assessment from histopathology whole-slide images. It consists of 636 WSIs from 304 donor-liver patients, graded by multiple pathologists on six indicators using a four-point scale (none, mild, moderate, severe), and the release includes both the whole-slide images and precomputed patch features. Each indicator is treated as a multi-class classification problem under a multiple-instance learning formulation, and nine weakly supervised MIL models are compared with five-fold cross-validation. The paper's supported conclusion is that weakly supervised models already score several donor-liver indicators at high accuracy, with cholestasis, portal tract fibrosis, and total steatosis the strongest and hepatocellular ballooning and macrovesicular steatosis the remaining open problems, making automated intraoperative graft assessment a feasible next step.","pith_inferences":["The appendix's per-feature label counts sum to roughly 260 per feature rather than 636; if those counts reflect slides without labels for some features, the effective per-task dataset is much smaller than the headline number and all baseline scores should be read against that smaller denominator.","Because the paper does not report inter-observer agreement, an immediate companion study would be to release the original individual pathologist grades and agreement statistics, letting the community separate model error from label noise.","A natural next test the authors did not run is ordinal regression or continuous percentage-based regression on the thresholds in the grading table, which may outperform the four-class softmax formulation used here.","The benchmark's single feature extractor and consensus labels mean transferability to other scanners, stains, and grading conventions remains untested; multi-center slides would reveal how much of the reported accuracy is institution-specific."],"forward_implications":["DLiPath gives computational pathology a shared testbed where donor-liver methods can be compared directly on six clinically defined tasks instead of on private slide collections.","Because the pipeline needs only slide-level labels, hospitals without pixel-level annotations can reproduce the benchmark using the released patch features, lowering the entry barrier for automated graft assessment.","The strongest baselines exceed 0.98 accuracy on total steatosis and 0.93 on cholestasis and fibrosis, making those tasks credible candidates for intraoperative decision support.","Hepatocellular ballooning remains the hardest task, with the best baseline near 0.69 accuracy, so the benchmark identifies where label quality or feature representation must improve.","The authors project that automated donor-liver assessment of this kind could reduce graft discard rates by 20 to 30 percent by making borderline graft decisions faster and more consistent."],"supporting_citations":[{"why":"Provides the four-category chronic-hepatitis grading protocol that the pathologists applied to donor liver slides.","marker":"[32]"},{"why":"Provides the pretrained transformer-based histopathology feature extractor that maps every patch to the 768-dimensional embedding used by all baselines.","marker":"[38]"},{"why":"Defines the attention-based multiple-instance learning aggregation used as a baseline.","marker":"[17]"},{"why":"Defines the clustering-constrained attention MIL variants used as baselines.","marker":"[25]"},{"why":"Defines the dual-stream max-plus-self-attention MIL baseline.","marker":"[21]"},{"why":"Defines the transformer-correlated MIL baseline that yields the best accuracy in most tasks.","marker":"[33]"},{"why":"Defines the multi-branch attention MIL baseline with stochastic top-K masking.","marker":"[44]"},{"why":"Defines the global-vector cross-attention MIL baseline that ranks second on several tasks.","marker":"[46]"},{"why":"Defines the interventional-bag MIL baseline used as a comparison.","marker":"[23]"},{"why":"Defines the low-rank iterative attention MIL baseline used as a comparison.","marker":"[39]"}],"fun_headline_variants":["First public donor-liver benchmark: 636 slides, six risk indicators","Donor-liver risk grading benchmark offers MIL baselines","Automated graft-risk assessment benchmarked on 636 slides","DLiPath: public donor-liver histology benchmark for graft risk","Benchmarking weak supervision on donor-liver biopsy risks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The benchmark's value depends on the expert labels being complete and trustworthy for every slide and every feature, even though the paper acknowledges that histopathologic grading is subjective and inconsistent on low-quality slides.","fun_headline_variants_meta":{"raw":{"variants":["First public donor-liver benchmark: 636 slides, six risk indicators","Donor-liver risk grading benchmark offers MIL baselines","Automated graft-risk assessment benchmarked on 636 slides","DLiPath: public donor-liver histology benchmark for graft risk","Benchmarking weak supervision on donor-liver biopsy risks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000444,"raw_usage":{"total_tokens":2252,"prompt_tokens":959,"completion_tokens":1293,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":1206}},"tokens_in":575,"tokens_out":1293,"duration_ms":15591,"temperature":1.0,"reasoning_tokens":1206,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:19:38.813822+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-grade a stratified random sample of slides with two independent pathologists using the paper's grading criteria: if weighted agreement is low (for example, kappa below 0.4), or if release inspection shows many slides missing labels for one or more features, then the reported baseline accuracies rest on labels that cannot carry them. A direct starting check is to reconcile the appendix's per-feature counts of about 260 with the claimed 636 labeled slides.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the four-category chronic-hepatitis grading protocol that the pathologists applied to donor liver slides."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pretrained transformer-based histopathology feature extractor that maps every patch to the 768-dimensional embedding used by all baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the clustering-constrained attention MIL variants used as baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the dual-stream max-plus-self-attention MIL baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the transformer-correlated MIL baseline that yields the best accuracy in most tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the multi-branch attention MIL baseline with stochastic top-K masking."},{"cited_title":"backdoor adjustment","cited_arxiv_id":null,"evidence_quote":"Defines the global-vector cross-attention MIL baseline that ranks second on several tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the interventional-bag MIL baseline used as a comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the low-rank iterative attention MIL baseline used as a comparison."}],"review_version":1}