{"id":"8d99f61d-c764-4043-9174-49787fc300a6","arxiv_id":"2505.09372","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A multi-aspect contrastive pretraining scheme with LLM-generated disease and concept sub-captions improves zero-shot dermatology classification and retrieval relative to existing vision-language models.","lead":"A new vision-language pretraining method for skin images splits long clinical text into knowledge aspects generated by a large language model, then aligns each aspect with the image. The authors report gains over existing vision-language models on zero-shot skin disease classification, concept annotation, and retrieval across eight datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-shot gains presume Derm1M pretraining is disjoint from the DermNet and Fitzpatrick17k test images, but no deduplication against evaluation sets is reported, so the headline improvements may reflect memorization rather than generalization.","rationale":"The reader identified the same load-bearing assumption: the Derm1M pretraining corpus may overlap with the evaluation datasets, especially DermNet and Fitzpatrick17k. My review of the manuscript found no deduplication statement, no overlap analysis, and no explicit acknowledgement of this risk in the experimental section. Because DermNet is a public atlas and is a known source component of Fitzpatrick17k, the possibility of overlap is not merely hypothetical. If overlap is confirmed, the central claim of significant zero-shot improvement collapses into a memorization effect. I considered alternative concerns—lack of error bars, unreleased code/data, and post hoc hyperparameter tuning—but those affect reproducibility and confidence rather than directly invalidating the comparison. The contamination concern is the most load-bearing because it targets the interpretation of the headline numbers themselves. The reader's CONDITIONAL verdict remains appropriate: the paper should not be rejected without evidence of overlap, but it must provide an overlap audit or dataset split guarantee before the zero-shot generalization claim can be accepted.","tokens_in":9250,"tokens_out":5702,"duration_ms":60695,"concrete_test":"Compute image-level overlap between Derm1M and the exact test splits of DermNet, Fitzpatrick17k, SkinCon, Derm7pt, and SkinCAP using perceptual hashing (e.g., pHash) plus URL matching where source URLs are retained; additionally run a CLIP-based near-duplicate search with a similarity threshold calibrated on known duplicate pairs. Report the overlap rate per dataset and re-run Tables 1 and 2 after removing all training images whose hash or nearest-neighbor similarity exceeds the threshold. If the overlap rate is negligible (e.g., <0.1%) and MAKE's margins persist on the deduplicated set, the concern is resolved; if DermNet or Fitzpatrick17k overlap is substantial, the 7.58% and 3.95% gains should be recomputed on the clean subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that MAKE's zero-shot gains (7.58% on DermNet, 3.95% on F17K) reflect generalization—rests on the unstated assumption that the 403,563 Derm1M training pairs are disjoint from the evaluation images. Section 3 specifies that Derm1M contains 100,487 pairs from PubMed and medical textbooks plus remaining data from YouTube and Twitter, but it never reports exact-duplicate or near-duplicate removal against the eight evaluation sets. This is a concrete external-validity risk: Fitzpatrick17k is known to draw images from DermNet and Atlas Dermatologico, and DermNet is a public dermatology atlas that could easily appear among crawled education resources. If overlap exists, Tables 1 and 2 measure retrieval of memorized training images rather than zero-shot generalization, and the reported advantage of MAKE over baselines could reflect which method best memorizes the test distribution rather than any benefit of multi-aspect knowledge alignment. The manuscript contains no limitation statement acknowledging this possibility, and the Derm1M construction details are only cited, not summarized, so the overlap status cannot be checked from the paper alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MAKE, a vision-language pretraining framework for dermatology that decomposes clinical captions into multiple knowledge aspects via LLMs, aligns them with image and patch embeddings through several contrastive losses, and weights them by similarity to a disease-aspect embedding. The model is pretrained on Derm1M (403,563 image-text pairs) and evaluated zero-shot on eight datasets covering disease classification, concept annotation, and cross-modal retrieval. MAKE is reported to outperform CLIP, SigLIP, CoCa, BiomedCLIP, PMC-CLIP, and MONET on most benchmarks, with ablations attributing gains to the three proposed components.","tokens_in":9503,"tokens_out":7699,"duration_ms":74730,"significance":"If the results are robust and free of data overlap, MAKE would be a useful contribution: it addresses text length limits by structured knowledge decomposition, and the fine-grained patch-text alignment and diagnosis-guided weighting are reasonable, clearly described additions to standard CLIP-style training. The paper also compares several baselines under matched pretraining data and includes ablations. However, the absence of a dataset-overlap analysis and the lack of statistical validation mean that the headline zero-shot gains are not yet established.","major_comments":[{"comment":"The central zero-shot claim rests on the assumption that the 403,563 Derm1M pretraining pairs are disjoint from the evaluation images, but the paper provides no evidence for this. Derm1M is assembled from PubMed, textbooks, YouTube, and Twitter following the pipelines of [13] and [9], while DermNet and Fitzpatrick17k are public atlases whose images can easily appear in such sources. Exact- or near-duplicate removal against the eight evaluation sets is never reported, and Derm1M construction is only cited to the companion paper [22], so the overlap status cannot be checked from this manuscript. If overlap exists, the reported gains (e.g., 7.58% on DermNet and 3.95% on F17K) measure memorization rather than zero-shot generalization. Please report a deduplication analysis and either restrict Derm1M by removing overlapping sources or validate on a held-out set known to be disjoint.","section":"§3 Experimental details, Tables 1–2"},{"comment":"All results are single point estimates with no error bars, confidence intervals, or significance tests, despite the abstract and text using the phrase \"significantly outperforms.\" Given that several reported differences are small (e.g., 0.68% and 0.85% in the Table 3 ablation), the improvements may not be stable across training runs. Please report means and standard deviations over at least three pretraining seeds and apply an appropriate statistical test to support the significance claim.","section":"§4 Results, Tables 1–3"},{"comment":"The statement \"we use the final checkpoint and conduct extensive hyperparameter tuning to find the optimal model\" does not specify the selection criterion or whether any held-out validation set was used. If hyperparameters were selected on the test sets of Tables 1–2, the comparisons are overoptimistic; if not, the protocol should be described. Please clarify the tuning and validation procedure and report the chosen hyperparameters for every model.","section":"§3 Implementation Details"}],"minor_comments":[{"comment":"The loss weighting notation is ambiguous: Eq. (7) produces a vector of K subtext weights, while the knowledge-set embeddings have default weights of 1, but Eq. (8) writes w_mkcl and w_slra as scalars multiplying scalar losses. Please specify the elementwise application of these weight vectors.","section":"§2.4, Eq. (8)"},{"comment":"It is unclear whether Derm1M† supplies the three text types (raw, disease, concept) separately, as used by MAKE, or as a single prepended string, as stated for baselines; please clarify the input format for each model.","section":"§3 Experimental details"},{"comment":"The LLM used for knowledge extraction is not specified; please provide the model name, prompt template, and sampling settings, since the generated sub-texts are a core component and affect reproducibility.","section":"§2.1 Encoding Stage"},{"comment":"The zero-shot classification and concept-annotation protocols are underspecified: no class-name templates are given for classification and no description of how per-concept AUROC is computed; please add this information.","section":"§4 Results"},{"comment":"The contribution statement claiming \"the first vision-language pretraining framework for dermatology\" overstates novelty given MONET [13] and other dermatology-oriented VLP efforts; please rephrase to claim the first multi-aspect knowledge-enhanced framework or similar.","section":"§1 Introduction"},{"comment":"The caption contains a typo: \"without spitted text\" should read \"without split text.\"","section":"Table 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The manuscript relies heavily on the companion Derm1M dataset (ref 22) from the same group, but the dataset is not publicly released and the overlap analysis against evaluation sets is absent. The editor may also wish to consider whether the 'first VLP framework for dermatology' novelty claim is credible given MONET [13]. The statistical-reporting gaps and unclear hyperparameter selection procedure further prevent the paper from being accepted in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering contribution whose central claim is currently hostage to unverified data hygiene. The method is a sensible assembly of three known ideas—LLM-decomposed multi-caption contrastive learning, patch-text fine-grained alignment, and similarity-based caption weighting—applied to a dermatology pretraining corpus. The ablations show each piece contributes, and the evaluation design is better than most: baselines are retrained on the same Derm1M data, not just lifted from other papers.\n\nThe load-bearing problem is contamination. Derm1M is built from PubMed, textbooks, YouTube, and Twitter; DermNet and Fitzpatrick17k are public atlases. The paper never reports exact- or near-duplicate removal between pretraining and test sets. If DermNet images appear in Derm1M, the headline +7.58% on DermNet and part of the F17k gain are retrieval of memorized images, not zero-shot generalization. The paper has no limitation statement acknowledging this, and Derm1M construction is only cited to a companion paper, so a reader cannot verify. This has to be fixed before the core claim is acceptable.\n\nOther issues are real but smaller. No error bars or significance tests anywhere; the paper reports single runs and says \"extensive hyperparameter tuning\" was used to pick final checkpoints. That is not disqualifying—the baselines get the same treatment—but it makes the 5% average gains feel less solid than the text suggests. Code and data are promised but not released, which is particularly annoying here because the overlap question is checkable only with the actual corpus. One thing I would push back on: the reader's \"label leakage\" concern about the diagnosis-guided weighting is not really accurate. The weighting uses disease terms present in the pretraining captions; that is the same supervision CLIP uses, not a test-set leak. The contamination issue is the real one.\n\nMinor overclaim: \"first vision-language pretraining framework for dermatology\" ignores MONET, but that is a one-line fix.\n\nWho is this for: researchers working on medical VLP and dermatology triage. It is a reasonable contribution to that applied literature, but I would not cite it in my own work until the data hygiene question is resolved. It deserves a serious referee: the method is coherent, the evaluation is broad, and the fixes are concrete and doable.\n\nRecommendation: send to peer review, with the expectation of a major revision that adds deduplication analysis, error bars, and released data/code.","headline":"Useful combination of known VLP techniques for dermatology, but the zero-shot claim depends on unverified dataset hygiene—dedup analysis is the missing piece.","tokens_in":10028,"tokens_out":3835,"would_cite":false,"duration_ms":38786,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAKE pretrains a vision-language model on 403,563 dermatology image-text pairs decomposed into multiple knowledge aspects, and claims state-of-the-art zero-shot skin disease classification, concept annotation, and cross-modal retrieval…","keywords":["vision-language pretraining","dermatology","zero-shot classification","contrastive learning","clinical text decomposition","concept annotation","cross-modal retrieval","knowledge enhancement"],"falsifier":"Search DermNet image URLs or near-duplicate images within Derm1M; if even a modest fraction of DermNet test images appear in pretraining, retrain on a deduplicated corpus and check whether the 7.58% DermNet accuracy gain over the strongest baseline shrinks or disappears.","tokens_in":9064,"feed_emoji":"🩺","tokens_out":6771,"duration_ms":52737,"temperature":0.7,"pith_summary":"Dermatology images come with long, unstructured clinical narratives, and standard vision-language models truncate those texts at a fixed token length, losing diagnostic detail. MAKE claims that decomposing each narrative into multiple knowledge aspects—the raw text, a disease-aspect text, a concept-aspect text, and individual sentences—and aligning each aspect with image patches during pretraining yields a model that transfers better to unseen skin conditions. Pretrained on 403,563 dermatological image-text pairs from public educational sources, MAKE reports state-of-the-art zero-shot results across eight datasets, including a 7.58% accuracy gain over the best baseline on DermNet and 3.95% on Fitzpatrick17K. The framework matters because it offers a way to use long clinical language in contrastive vision-language models without extending token limits, and it suggests that how text is structured during pretraining, not just its content, drives zero-shot generalization.","feed_headline":"Splitting clinical text into knowledge aspects lifts zero-shot skin AI","feed_subtitle":"MAKE gains 7.58% on DermNet and 3.95% on Fitzpatrick17K over the best baseline, with no fine-tuning.","key_machinery":"The central object is the multi-aspect text decomposition: each image-text pair is expanded into a knowledge set of three texts (raw, disease-aspect, concept-aspect) and a subtext set of K sentences, producing K+3 embeddings that all serve as positive pairs in a multi-positive contrastive loss. Two additional mechanisms carry the argument: a fine-grained alignment that weights image patches by dot-product similarity to the raw text and pools them into a knowledge-enhanced visual embedding, and a diagnosis-guided weighting that scales each sub-caption's contribution by its cosine similarity to the disease-aspect embedding. Together the losses are $\\mathcal{L}_{\\text{total}} = \\hat{w}\\,\\mathcal{L}_{\\text{mkcl}} + \\lambda \\hat{w}\\,\\mathcal{L}_{\\text{slra}}$, where the weights $\\hat{w}$ come from the diagnosis similarity. This lets the model learn which aspects of clinical language matter for which visual features, without needing annotations of those aspects.","core_discovery":"The central claim is that a vision-language model tailored to dermatology can beat general-purpose and domain-agnostic medical models on zero-shot skin tasks if it is trained to associate not one caption but a set of knowledge-decomposed sub-captions with each image. MAKE encodes the raw text plus two LLM-derived aspects (disease terminology and clinical concept descriptors) plus sentence-level sub-captions, and trains with three losses: a multi-positive contrastive loss aligning all K+3 texts with the image, a fine-grained alignment loss matching each sub-caption to a knowledge-enhanced pooled visual embedding, and a weighting term that up-weights sub-captions most similar to the disease-aspect text. The paper reports that this setup outperforms strong baselines on zero-shot disease classification, concept annotation, and cross-modal retrieval, with the largest gains on datasets that require broad clinical vocabulary.","pith_inferences":["A concrete risk not resolved by the paper: the pretraining corpus Derm1M draws on PubMed, textbooks, YouTube, and Twitter, while evaluation datasets like DermNet are public atlases; if any DermNet images or captions leaked into pretraining, the 7.58% DermNet gain would reflect memorization. A clean held-out evaluation on a corpus curated after removing all evaluation-set images would settle this.","The method's success suggests that sentence-level decomposition acts as a soft form of data augmentation for the text side; one testable extension is whether random sentence sampling with a similar multi-positive loss reproduces part of the gain without LLM-generated knowledge aspects.","Because the weighting uses cosine similarity to the disease-aspect text, the framework implicitly assumes the disease is named in or inferable from the raw text; in fully unlabeled clinical narratives without diagnosis mentions, this weighting signal would need an alternative source."],"forward_implications":["The framework's success in dermatology suggests the same decomposition approach will transfer to other medical specialties with long unstructured reports, such as pathology and ophthalmology.","The finding that knowledge-augmented captions hurt conventional VLMs but help MAKE implies that the benefit comes from the multi-aspect alignment structure, not from the augmented text alone, so future work can focus on structuring, not just enriching, medical captions.","Zero-shot concept annotation and retrieval improve alongside classification, so the model's internal alignments are finer-grained than label-level supervision, which may support interpretability tools that ground predictions in clinical concepts.","The diagnosis-guided weighting offers a principled way to inject clinical priors into contrastive pretraining, and it could be extended to patient history or demographic context as additional weighting signals."],"supporting_citations":[{"why":"Supplies the base contrastive vision-language architecture and the zero-shot evaluation protocol, and defines the token-length truncation problem MAKE addresses.","marker":"[17]"},{"why":"Provides the multi-positive contrastive learning loss that MAKE extends to multiple knowledge-aspect texts.","marker":"[5]"},{"why":"The Derm1M dataset of 403,563 dermatological image-text pairs on which MAKE and all same-data baselines are pretrained.","marker":"[22]"},{"why":"A state-of-the-art dermatology VLP baseline and the source of the PubMed/textbook crawling process used to build part of Derm1M.","marker":"[13]"},{"why":"Provides the SkinCon dataset used for zero-shot concept annotation evaluation.","marker":"[1]"},{"why":"Provides the SkinCAP dataset with rich captions used for cross-modal retrieval evaluation.","marker":"[29]"},{"why":"SigLIP serves as a strong baseline; the best prior retrieval results come from SigLIP on Derm1M-dagger.","marker":"[26]"},{"why":"CoCa serves as a baseline vision-language model pretrained on Derm1M and Derm1M-dagger for comparison.","marker":"[25]"}],"fun_headline_variants":["Knowledge-split captions boost zero-shot skin diagnosis","MAKE: multi-aspect captions lift zero-shot dermatology AI","Decomposing clinical text enhances dermatology vision-language model","Multi-aspect pretraining improves zero-shot skin disease detection","Skin AI benefits from knowledge-decomposed captions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that none of the test images or captions, especially from DermNet, appear in the 403,563-image pretraining corpus, since Derm1M was crawled from public sources that could overlap with the test atlases.","fun_headline_variants_meta":{"raw":{"variants":["Knowledge-split captions boost zero-shot skin diagnosis","MAKE: multi-aspect captions lift zero-shot dermatology AI","Decomposing clinical text enhances dermatology vision-language model","Multi-aspect pretraining improves zero-shot skin disease detection","Skin AI benefits from knowledge-decomposed captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1366,"prompt_tokens":951,"completion_tokens":415,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":333}},"tokens_in":567,"tokens_out":415,"duration_ms":3738,"temperature":1.0,"reasoning_tokens":333,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:32:36.850200+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Search DermNet image URLs or near-duplicate images within Derm1M; if even a modest fraction of DermNet test images appear in pretraining, retrain on a deduplicated corpus and check whether the 7.58% DermNet accuracy gain over the strongest baseline shrinks or disappears.","supporting_citations":[{"cited_title":"Advances in Neural Information Processing Systems35, 18157–18167 (2022)","cited_arxiv_id":null,"evidence_quote":"Provides the SkinCon dataset used for zero-shot concept annotation evaluation."}],"review_version":1}