{"id":"5752ed8c-4f01-41e6-8be6-6fe0a8ccab71","arxiv_id":"2501.13341","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A multi-aspect knowledge distillation method that appends binary question-answer logits from an MLLM to a classifier's output improves fine-grained image classification accuracy by up to about 6 points.","lead":"This paper augments image classification models with extra output heads that learn yes/no answers to LLM-generated questions about each image, distilled from a multimodal LLM's logits. The resulting models beat standard baselines on fine-grained and coarse-grained classification, and the authors show the gains grow when training data is scarce.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never tests whether gains depend on image-specific MLLM aspect content; the Rand ablation only changes target distribution, so the knowledge-transfer mechanism is unverified.","rationale":"The paper is a well-executed empirical study: MaKD is simple, the ablations across LLMs/MLLMs, Q, and alpha show robustness, and the 32 positive comparisons in Table 1 are internally consistent. I therefore do not think the paper should be rejected. The load-bearing gap is that the claim of transferring 'aspect knowledge' rests entirely on the assumption that the teacher's aspect probabilities carry useful image-specific information. The Rand baseline (Table 2) is an inadequate control because it alters both content and target entropy. A permutation test would cleanly separate the two. If the permuted-target performance matches the reported MaKD results, the method's value is as a form of auxiliary-task regularization, not knowledge distillation, and the paper's mechanistic interpretation would need revision. The reader's CONDITIONAL verdict is appropriate; our concern adds a specific, feasible test to the conditions.","tokens_in":13786,"tokens_out":7089,"duration_ms":61508,"concrete_test":"Run a permutation control: take the extracted MLLM soft targets q_i (Eq. 1) for each image and question, and randomly permute them across images within the training set, preserving each question's marginal distribution. Retrain the same student architectures (e.g., ResNet18 on StanfordCars and CUB200) with the same hyperparameters and the same L_total (Eq. 6). If permuted-target accuracy is within ~0.5 points of the original MaKD accuracy, the gains are driven by the auxiliary task structure, not by the image-specific content of the MLLM aspects; if it drops substantially, the content matters. A secondary check would be to have humans label a subset of images for the selected aspects and compare with InternVL2-8B's probabilities.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that distilling 'multi-aspect knowledge' from an MLLM improves classification. The only experiment isolating the information content of the teacher targets is the Rand control in Table 2 (StanfordCars), which replaces the MLLM soft targets with Gaussian random logits. This control simultaneously changes both the content and the marginal statistics of the targets: random sigmoid probabilities are concentrated near 0.5, whereas the MLLM targets are often near 0 or 1. The improvement over Rand could therefore be due to the sharpness/entropy of the auxiliary target distribution, not to image-specific correctness. The paper never evaluates the accuracy of InternVL2-8B's yes/no answers against any ground truth; Sec 4.1 only manually reviews the generated questions, not the answers. Given that the same MLLM has zero-shot class accuracy of only 10-14% on fine-grained datasets (Table 1), its attribute judgments are plausibly noisy or biased. Figures 5 and 6 show the student matching the teacher, but any distillation method will produce that, so it does not validate the teacher. Consequently, the asserted mechanism 'aspect knowledge enhances performance' is not established; the observed gains may be a regularization artifact of adding a binary auxiliary head.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Multi-aspect Knowledge Distillation (MaKD), a training procedure for image classifiers that appends Q auxiliary output units to the class logits and trains them with binary cross-entropy against yes/no aspect targets. The aspect questions are generated by GPT-4o from the dataset class list, and the targets are the softmax yes-token probabilities extracted from InternVL2-8B for each image. The total loss combines the usual cross-entropy class loss with an alpha-weighted BCE aspect loss (Eq. 6). Experiments on six fine-grained and two coarse-grained datasets with four architectures report consistent accuracy gains over the cross-entropy baseline (Table 1), along with ablations (Tables 2-3, Figures 2-3), extensions to traditional knowledge distillation and reduced training data (Tables 4-5), an object detection extension (Table 6), and qualitative analyses (Figures 4-6).","tokens_in":14008,"tokens_out":4571,"duration_ms":44996,"significance":"If the gains indeed come from the content of the multi-aspect supervision, the method is a simple and broadly applicable contribution: it requires no change to the student architecture beyond widening the final layer, and the evaluation across eight datasets, four architectures, reduced data, and detection is a genuine strength. The paper also includes an explicit limitation discussion in Sec. 6. However, the current experiments do not yet isolate the proposed mechanism. The Rand ablation in Table 2 changes both the semantic content and the marginal distribution of the targets, and the accuracy of the MLLM's yes/no answers is never measured, so the central claim that aspect knowledge, rather than an auxiliary-training regularizer, drives the improvements is not fully established.","major_comments":[{"comment":"The Rand control replaces the MLLM targets with Gaussian random logits, which simultaneously changes the semantic content and the marginal distribution of the targets. The observed gap between Rand and Ours could therefore be due to differences in target sharpness or entropy rather than to image-specific aspect knowledge. Please add a control that preserves the marginal statistics of the MLLM targets, for example by permuting the MLLM targets across images within each question or by using a fixed set of sharp targets, and report whether the gain persists. Without such a control, the statement that the multi-aspect logits \"represent knowledge from various aspects\" is not supported by the evidence presented.","section":"Sec. 4.3, Table 2"},{"comment":"The final values of the hyperparameters Q (number of aspects) and alpha (loss weight) are not reported for any dataset or model, even though the text states that Q is tuned per dataset and neural network and Figure 3 shows clear sensitivity to alpha. This is a reproducibility problem and it also leaves open the possibility that part of the reported gains comes from per-dataset hyperparameter selection rather than from the method itself. Please report the chosen Q and alpha values, the search ranges, and the corresponding standard deviations across the three runs.","section":"Sec. 4.1, Figures 2-3"},{"comment":"The paper only manually reviews the generated questions and does not validate the MLLM's yes/no answers against any ground truth. Since InternVL2-8B has low zero-shot classification accuracy on the fine-grained datasets (Table 1), its aspect judgments may be noisy or systematically biased. The t-SNE visualizations and probability comparisons in Figures 5-6 show that the student matches the teacher, but any distillation method would produce such a match, so these figures do not validate the teacher's correctness. Please measure the teacher's aspect-answer accuracy on a labeled sample (or against an oracle), or show that the gains survive when the targets are replaced by an equally sharp but semantically meaningless distribution. This is necessary to support the claimed knowledge-transfer mechanism.","section":"Sec. 4.1 and Sec. 5"},{"comment":"All tables report averages over three runs but no standard deviations or significance tests are reported. Several improvements are small, for example +0.72 accuracy points on FGVC-Aircraft with EfficientNet in Table 1 and +0.23 AP on MS-COCO with MobileNet-V2 in Table 6. Without variance estimates or significance tests, the abstract's claim that \"in all experimental results, our method improves the performance of the baselines\" cannot be assessed for the smaller entries. Please include standard deviations or confidence intervals, and ideally a paired test for the smallest gains.","section":"Tables 1-7"}],"minor_comments":[{"comment":"The caption of Table 5 reads \"Extension to class logit distillation with MLLM on Caltech101,\" but the table shows reduced-data experiments on StanfordCars, OxfordPets, and Caltech101; this appears to be a copy-paste error and should be corrected.","section":"Table 5 caption"},{"comment":"The notation around Eq. (5) is confusing: q is defined as the softmax yes probability from the MLLM in Sec. 3.2, but the text says \"where y-hat represents the predicted probability from MLLM,\" which seems to refer to the student's prediction rather than the teacher's. Please clarify which quantities are teacher targets and which are student outputs.","section":"Sec. 3.4, Eq. (5)"},{"comment":"The sentence \"we add features to the backbone network of Faster R-CNN-FPN\" is vague; please specify where the auxiliary aspect head is attached, how its loss is combined with the detector losses, and how the Q=50 aspect questions were generated for MS-COCO.","section":"Sec. 4.4, object detection"},{"comment":"The x-axis labels of Figure 3 are difficult to read (the tick labels appear garbled), and the alpha values used in the experiments should be listed explicitly in the caption or text rather than only appearing in the plot.","section":"Figure 3"},{"comment":"The limitation paragraph acknowledges that performance \"may be affected by the multi-aspect questions in MLLMs, making generalization across datasets challenging.\" This concern is directly relevant to the open control issue in Table 2 and should be addressed with the proposed additional experiments rather than deferred entirely to future work.","section":"Sec. 6, Limitation"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the proposed method is simple enough to be of interest, but the central mechanistic claim is currently under-supported by the ablation design. I do not see a circularity problem, since the aspect targets come from an external MLLM and frozen question generator. The main risk is that the reported gains are a regularization artifact of the auxiliary binary head; this can be resolved with the target-permutation control and teacher-accuracy evaluation requested in the major comments. I would be willing to reconsider after those experiments are added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a simple, broadly applicable recipe — append Q binary heads to a classifier and train them with BCE on the MLLM's softmax 'yes' probabilities for LLM-generated aspect questions — and it consistently improves accuracy: gains on all eight datasets, all four backbones, biggest on fine-grained and low-data. That's a useful result, and the paper deserves a real referee. I agree with the reader's conditional verdict, and I think the stress-test note is on target but not fatal.\n\nWhat's new: prior LLM-attribute methods use hard labels or bottleneck layers; here the student gets the MLLM's soft yes/no probabilities on abstract questions, with the output head expanded to carry them. That's a different mechanism, assembled from known parts. The paper does the obvious ablations: KL vs BCE, Rand vs MLLM targets, GPT-3.5 vs GPT-4o, InternVL vs LLaVA, and Q and alpha sweeps. The Rand comparison in Table 2 is the one that matters, and it's flawed. It replaces the MLLM targets with Gaussian random logits, which changes both the content and the sharpness of the targets. MLLM targets are often near 0 or 1; random sigmoid targets are concentrated near 0.5. So the gain over Rand could come from target sharpness, not from image-specific content. The paper never verifies the MLLM's answers against any ground truth, and the same MLLM gets 10–14% zero-shot accuracy on the fine-grained sets. So the knowledge-transfer mechanism is unverified. That's a genuine soft spot, but it doesn't break the accuracy claim. The method works; we just don't know exactly why.\n\nCredit where due: the experiments are broad, the method is cheap at inference (no MLLM needed), it integrates with standard KD, and the reduced-data gains are real. The paper also states its limitation clearly. The missing pieces are standard: no error bars despite 'three runs,' no final Q and alpha values, no release of the generated questions, and the detection results are marginal. A simple control — permuting MLLM targets across images, or using synthetic targets matched for sharpness — would isolate content and settle the mechanism question.\n\nBottom line: this is a modest, honest contribution that advances a useful recipe. It deserves peer review. I'd recommend conditional acceptance with a request for the target-content control and full hyperparameter disclosure. The reader's score of 6/6/6 sounds about right.","headline":"Simple, broadly effective distillation recipe; the accuracy gains are consistent, but the knowledge-transfer mechanism is under-supported and needs a proper control.","tokens_in":14571,"tokens_out":2701,"would_cite":true,"duration_ms":25737,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding yes/no aspect answers distilled from a multimodal LLM's logits improves image classification on every dataset and architecture tested.","keywords":["multi-aspect knowledge distillation","multimodal large language models","image classification","fine-grained classification","logit distillation","binary cross-entropy loss","output dimension expansion","object detection"],"falsifier":"A decisive test is to replace the MLLM's aspect targets with the same targets randomly permuted across classes (or with answers deliberately falsified for a subset of questions) and retrain: if accuracy stays at the same level, the gain is not from the content of the aspect knowledge. A complementary check is to evaluate the method on a fine-grained dataset with human-annotated attributes, such as CUB attributes, and measure how well the MLLM's yes/no probabilities match the attribute ground truth; if the correlation is near zero, the method cannot be transferring accurate aspect knowledge, and any improvement would have to be attributed to the auxiliary task structure alone.","tokens_in":13574,"feed_emoji":"🎯","tokens_out":14599,"duration_ms":105054,"temperature":0.7,"pith_summary":"The paper proposes a training-time addition for image classifiers: before training, it asks a large language model to generate yes/no questions about visual, categorical, and environmental aspects of each dataset's classes (e.g., 'Does the car have a convertible roof?'). It then feeds each training image and those questions to a frozen multimodal LLM (InternVL2-8B), extracts the softmax probability of the 'yes' token for each question, and expands the classifier's output head to predict those probabilities alongside its class logits. The total loss is the usual cross-entropy on class logits plus a weighted binary cross-entropy on the aspect probabilities. The paper's central claim is that this multi-aspect knowledge distillation improves classification accuracy over the cross-entropy baseline for every dataset and architecture tested, with gains from +0.72 to +6.24 points, and that the gains grow when training data is scarce. The authors argue this matters because it transfers abstract and complex knowledge — not just labels — into the model at almost no training cost, since the multimodal teacher is frozen and queried offline.","feed_headline":"Boost classification accuracy with multimodal LLM yes/no questions","feed_subtitle":"Distilling an MLLM's yes/no answers lifts accuracy on every tested dataset, including low-data regimes.","key_machinery":"The engine of the method is the multi-aspect knowledge distillation loss $L_{\\mathrm{MaKD}}$ (Eq. 5): a binary cross-entropy loss between the student's aspect-logit predictions and the MLLM's softmaxed yes-probabilities. This loss is attached to an expanded output head of dimension $D = C + Q$ (Eq. 2), so the same backbone that classifies also predicts per-image answers to $Q$ yes/no aspect questions. The targets come from a two-step pipeline: GPT-4o generates $N$ candidate yes/no questions from the class list, an LLM ranks and filters them down to $Q$, and InternVL2-8B, prompted with the image and each question, supplies the yes/no token logits whose softmax $q_i$ (Eq. 1) becomes the target. The paper also tunes a weighting factor $\\alpha$ on the MaKD loss and finds performance degrades only at very large values (e.g., 50), showing the auxiliary loss is fairly robust to its weight.","core_discovery":"The central discovery the paper argues for is that a frozen multimodal LLM's yes/no probabilities on aspect questions are a transferable supervision signal for image classification, even though the same MLLM has poor zero-shot accuracy on fine-grained class labels (e.g., 10.27% on CUB200). The method works by distilling the MLLM's softmaxed yes-logits through an expanded output head: the model outputs $D = C + Q$ values, where the first $C$ are class logits trained with cross-entropy and the remaining $Q$ are aspect logits trained with binary cross-entropy against targets $q_i = e^{z_y}/(e^{z_y}+e^{z_n})$ extracted from the MLLM. The paper reports consistent improvements across four architectures and eight datasets, including fine-grained sets like StanfordCars and CUB200 and coarse-grained sets like Caltech101 and Mini-ImageNet, and shows the aspect head also helps when combined with standard knowledge distillation and when transferred to a Faster R-CNN object detector. It further demonstrates through t-SNE and per-sample probability comparisons that the student's aspect logits track the MLLM's aspect logits, which the authors take as evidence that the distilled aspect knowledge is learned rather than ignored.","pith_inferences":["If the gain is genuine, the softmaxed yes/no probability of an MLLM can serve as a dense 'semantic attribute' target, suggesting the same trick could be extended to other dense prediction tasks such as semantic segmentation or depth estimation by attaching an aspect head.","Because the method only needs an offline, frozen MLLM, it acts like a cheap way to inject common-sense priors into any vision backbone; a testable prediction is that the gain should shrink as the student's capacity or the amount of training data grows, since the aspect loss then becomes redundant with the label signal.","The paper never verifies the MLLM's answers against ground truth, so an important open question is whether the benefit comes from the correctness of the aspect knowledge or merely from the multi-task auxiliary loss; comparing against human-attribute labels or against targets from a weak captioning model would separate these two explanations."],"forward_implications":["Adding the MaKD loss to cross-entropy improves image classification accuracy over baseline for every tested dataset and architecture, with gains between +0.72 and +6.24 points (Table 1).","The improvement is larger when training data is scarce: with 40% of StanfordCars, ResNet18 gains +24.01 points over baseline (Table 5).","The aspect head composes with existing logit-distillation methods: MaKD adds gains on top of both KD and DKD (Table 4).","The same output-expansion recipe carries over to object detection, giving small but consistent AP improvements on MS-COCO with Faster R-CNN-FPN (Table 6).","Distilling aspect probabilities rather than class logits sidesteps the MLLM's poor fine-grained zero-shot classification, since aspect questions are easier to answer than fine-grained class labels."],"supporting_citations":[{"why":"Supplies the frozen MLLM (InternVL2-8B) whose yes/no token logits become the distillation targets.","marker":"[4]"},{"why":"Provides the LLM (GPT-4o) that generates and ranks the multi-aspect yes/no questions.","marker":"[1]"},{"why":"Fine-grained bird dataset (CUB200) where the method adds up to +6.24 accuracy points.","marker":"[32]"},{"why":"Fine-grained car dataset (StanfordCars), the main testbed for ablations and the largest gains.","marker":"[10]"},{"why":"Fine-grained pet dataset (OxfordPets) used to demonstrate consistent gains.","marker":"[22]"},{"why":"Texture dataset (DTD) showing the method's gains on non-organic classes.","marker":"[5]"},{"why":"Flower dataset (102Flowers) used as a fine-grained testbed.","marker":"[19]"},{"why":"Aircraft dataset (FGVC-Aircraft) confirming gains on fine-grained classes.","marker":"[17]"},{"why":"Coarse-grained object dataset (Caltech101) where MaKD also improves baselines.","marker":"[6]"},{"why":"Coarse-grained Mini-ImageNet confirming gains on a generic-object setting.","marker":"[24]"}],"fun_headline_variants":["Distill MLLM yes/no answers to boost image classification","Use an MLLM's yes/no opinions as training targets","Aspect questions from LLMs improve classification without fine-tuning","Boost vision models by distilling aspect yes/no from an MLLM","Multi-aspect distillation from LLM yes/no questions lifts accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the MLLM's yes/no probability answers to the generated aspect questions are correct and informative enough to supervise the student; the paper manually inspects the questions for hallucination but never checks the MLLM's answers against ground truth, so the measured gains could partly come from regularization rather than from accurate aspect knowledge.","fun_headline_variants_meta":{"raw":{"variants":["Distill MLLM yes/no answers to boost image classification","Use an MLLM's yes/no opinions as training targets","Aspect questions from LLMs improve classification without fine-tuning","Boost vision models by distilling aspect yes/no from an MLLM","Multi-aspect distillation from LLM yes/no questions lifts accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000738,"raw_usage":{"total_tokens":3359,"prompt_tokens":1072,"completion_tokens":2287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":688,"completion_tokens_details":{"reasoning_tokens":2200}},"tokens_in":688,"tokens_out":2287,"duration_ms":27700,"temperature":1.0,"reasoning_tokens":2200,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:14:12.471534+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test is to replace the MLLM's aspect targets with the same targets randomly permuted across classes (or with answers deliberately falsified for a subset of questions) and retrain: if accuracy stays at the same level, the gain is not from the content of the aspect knowledge. A complementary check is to evaluate the method on a fine-grained dataset with human-annotated attributes, such as CUB attributes, and measure how well the MLLM's yes/no probabilities match the attribute ground truth; if the correlation is near zero, the method cannot be transferring accurate aspect knowledge, and any improvement would have to be attributed to the auxiliary task structure alone.","supporting_citations":[{"cited_title":"Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen MLLM (InternVL2-8B) whose yes/no token logits become the distillation targets."},{"cited_title":"The caltech-ucsd birds-200-2011 dataset","cited_arxiv_id":null,"evidence_quote":"Fine-grained bird dataset (CUB200) where the method adds up to +6.24 accuracy points."},{"cited_title":"3d object representations for fine-grained categorization","cited_arxiv_id":null,"evidence_quote":"Fine-grained car dataset (StanfordCars), the main testbed for ablations and the largest gains."},{"cited_title":"Cats and dogs","cited_arxiv_id":null,"evidence_quote":"Fine-grained pet dataset (OxfordPets) used to demonstrate consistent gains."},{"cited_title":"Describing textures in the wild","cited_arxiv_id":null,"evidence_quote":"Texture dataset (DTD) showing the method's gains on non-organic classes."},{"cited_title":"Automated flower classification over a large number of classes","cited_arxiv_id":null,"evidence_quote":"Flower dataset (102Flowers) used as a fine-grained testbed."},{"cited_title":"Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories","cited_arxiv_id":null,"evidence_quote":"Coarse-grained object dataset (Caltech101) where MaKD also improves baselines."},{"cited_title":"Optimization as a model for few-shot learning","cited_arxiv_id":null,"evidence_quote":"Coarse-grained Mini-ImageNet confirming gains on a generic-object setting."}],"review_version":1}