{"id":"cf0d9275-32c0-4166-81da-7b267eab195a","arxiv_id":"2509.14860","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MARIC decomposes image classification into outliner, aspect, and reasoning prompts for a vision-language model, reporting accuracy gains that are inconsistent across datasets and statistically unsupported.","lead":"A four-agent chain of vision-language prompts, where an outliner sets the topic, three aspect agents describe colors and context, and a reasoning agent combines them, is tested on four image classification datasets. The paper claims large gains over single-pass VLM baselines, but its own table shows inconsistent results and no error bars.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 itself contradicts the central claim of 'consistent and significant improvements': MARIC loses to CoT on Weather (7B, 65.6 vs 70.1) and to SAVR on Skin Cancer (13B, 56.3 vs 62.6), with no significance tests reported.","rationale":"The Reader's REJECT verdict is well supported, and my analysis agrees that the empirical evidence does not establish MARIC's central claim. However, the reader's stated weakest assumption focuses on the orthogonality of the Aspect Agents and the effectiveness of the reflection step, which is an architectural-mechanism concern. The most load-bearing issue I see is more direct: even accepting the paper's own numbers, Table 1 contradicts the claim of 'consistent' improvements in two cells, so the central empirical claim fails internally. The missing significance tests and undocumented subsets compound this by preventing the reader from determining whether the reported gains are reliable. The ablation and human study are too weak to substitute for a proper evaluation. My recommendation is unchanged relative to the reader's REJECT: the central claim is not established by the presented evidence. I mark agreement as partial because my primary concern is the direct Table 1 contradiction rather than the untested aspect-orthogonality assumption, though both point to the same overall verdict.","tokens_in":6271,"tokens_out":2611,"duration_ms":25618,"concrete_test":"Reproduce Table 1 from the released code using full, documented test splits and per-sample predictions, then run pairwise significance tests (e.g., McNemar's test) between MARIC and each baseline for each model and dataset. Specifically verify the Weather/7B cell (MARIC 65.6 vs CoT 70.1) and the Skin Cancer/13B cell (MARIC 56.3 vs SAVR 62.6). If these cells reproduce and the baseline advantages are significant, the claim of consistent significant improvement fails. If the cells flip under a corrected evaluation protocol, the claim could be salvaged.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, restated in the Abstract and Conclusion, is that MARIC achieves 'consistent and significant improvements in accuracy' over baselines. This claim is load-bearing because the method's value is entirely empirical; there is no theoretical guarantee that agent decomposition helps classification. The paper's own Table 1 refutes the 'consistent' part: in the LLaVA-1.5-7B row, MARIC scores 65.6 on Weather versus 70.1 for Chain-of-Thought; in the LLaVA-1.5-13B row, MARIC scores 56.3 on Skin Cancer versus 62.6 for SAVR. Thus in 2 of 8 model-dataset cells, MARIC does not outperform a baseline. In addition, no significance tests are reported, the evaluation uses undocumented subsets (CIFAR-10 and OOD-CV are sampled at 100 images per class; Skin Cancer has only 174 images total), and the ablation in Table 2 removes all three Aspect Agents together, so it cannot isolate the reflection step or the claimed orthogonal-aspect mechanism. The human study covers only 30 images and rates subjective criteria. Because the only evidence for the central empirical claim is this table, and the table undermines 'consistent', the claim is not established. This is a correctness risk, not a disagreement about novelty.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MARIC, a multi-agent framework for zero-shot VLM image classification that decomposes inference into an Outliner Agent (generating global prompts), three Aspect Agents (extracting fine-grained descriptions along distinct visual dimensions), and a Reasoning Agent (reflecting on and synthesizing the descriptions into a final label with a reasoning trace). The authors evaluate MARIC on CIFAR-10, OOD-CV, Weather, and Skin Cancer using LLaVA-1.5-7B and 13B, comparing against Direct Generation, Chain-of-Thought, and a Single-Agent Visual Reasoning baseline. They report accuracy improvements, an ablation without Aspect Agents, a small human study of aspect quality, and a t-SNE visualization of reasoning embeddings. The main claim is that MARIC yields 'consistent and significant improvements' in accuracy across all benchmarks.","tokens_in":6653,"tokens_out":4002,"duration_ms":42061,"significance":"If the claimed gains were supported, MARIC would be a simple and useful contribution to zero-shot VLM classification, showing that a fixed multi-agent decomposition with reflective synthesis can improve both accuracy and interpretability. The architecture is clearly described, the code is promised, and the motivation is reasonable. However, the current evidence is not sufficient: the paper's own Table 1 contains two clear failures, no statistical significance is reported, the ablation cannot isolate the proposed mechanism, and the human study is too small and subjective to establish complementarity. The central empirical claim is therefore not established, and the current significance is limited.","major_comments":[{"comment":"The abstract and conclusion claim 'consistent and significant improvements' in accuracy, but the paper's own Table 1 contradicts this claim. MARIC-7B scores 65.6 on Weather versus 70.1 for Chain-of-Thought, and MARIC-13B scores 56.3 on Skin Cancer versus 62.6 for SAVR. These are not negligible margins. No significance tests, confidence intervals, or repeated runs are reported, so even the positive entries cannot be assessed. Because the contribution is entirely empirical, this contradiction undermines the central claim.","section":"Table 1 and Section 4.2.1"},{"comment":"The only ablation removes all three Aspect Agents together. This design cannot isolate the contribution of the Reasoning Agent's reflection step (Section 3.4) or test the claimed orthogonality/complementarity of the aspect prompts (Sections 3.2-3.3). The reported differences are small (e.g., 93.5 vs 93.4 on CIFAR-10; 56.3 vs 52.9 on Skin Cancer) and no variance is given, so the result is compatible with the aspect agents providing no significant additional information.","section":"Section 4.3, Table 2"},{"comment":"The evaluation uses undocumented subsets: CIFAR-10 and OOD-CV are sampled at 100 images per class, and Skin Cancer has only 174 images total. With these sample sizes, point accuracy differences are noisy, and the selection procedure is not described. The word 'significant' in the abstract and conclusion requires at least McNemar's test or bootstrap confidence intervals; none are reported.","section":"Section 4.1.1"},{"comment":"The human study is based on only 30 images from CIFAR-10 and rates subjective quality criteria (relevance, diversity, accuracy) without comparing against a non-agent prompt or a single-prompt baseline. As a result, it does not provide evidence that the three Aspect Agents extract complementary information rather than redundant paraphrases. The caption also omits the meaning of the M ± SD columns.","section":"Section 5, Table 3"}],"minor_comments":[{"comment":"Inconsistent terminology: 'Outliner Agent' in the text and Section 3.2, but 'Outlier Agent' in Figure 1 and Section 3.1. Please unify.","section":"Throughout"},{"comment":"Broken formatting: 'M ulti-Agent' and 'LLaV A' appear with spurious spaces throughout; also 'integrated reflection step' should be 'an integrated reflection step'.","section":"Abstract"},{"comment":"The notation 'C_θ(I) ≃ G_rea(I, S_rea|D)' is vague; clarify how the classification head is 'approximated' and how the label is extracted from the ⟨answer⟩ tag.","section":"Section 3.4, Eq. (3)"},{"comment":"The table header 'M ± SD' is not defined in the caption; specify that M is the mean Likert score and SD the standard deviation.","section":"Table 3"},{"comment":"Several references are incomplete for a journal submission (e.g., [16] and [17] are Kaggle entries without version/date; [8] and [13] are arXiv preprints).","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper's own Table 1 is enough to invalidate the headline claim of 'consistent and significant improvements'. The method is a prompting ensemble with no theoretical guarantee, and the empirical support is too thin without significance testing and a proper ablation. Even after revising the central claim, the contribution would likely be a workshop-level empirical note rather than a journal paper, unless substantially strengthened with more datasets, statistical rigor, and an ablation that isolates the reflection mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is a legitimate extension of the multi-agent prompting pattern to zero-shot VLM image classification: an outliner agent sets prompts, three aspect agents describe distinct visual dimensions, and a reasoning agent reflects and synthesizes. That decomposition is new in this specific form, the writing is clear, the framework is easy to understand, and the code is released. The authors also cite their own prior agent work [8,13] appropriately as design inspiration, not as evidence for the numbers. The limitation section is honest about overlap and token overhead. There is a real, if modest, contribution here: a training-free way to coax more accuracy out of VLMs with interpretable reasoning traces.\n\nThe problem is that the central empirical claim collapses on inspection. The abstract and conclusion say MARIC produces \"consistent and significant improvements,\" but Table 1 shows MARIC losing to CoT on Weather (7B: 65.6 vs 70.1) and to SAVR on Skin Cancer (13B: 56.3 vs 62.6). Calling that \"consistent\" is wrong. There are no error bars, no significance tests, no repeated runs. CIFAR-10 and OOD-CV are sampled at 100 images per class, which is undocumented in the main text and makes the gap of a few points even less trustworthy. The ablation in Table 2 removes all three aspect agents together, so it cannot isolate the reflection step or the claimed orthogonality of the aspects. The human study on 30 images with 11 participants is suggestive, but it only rates subjective quality and says nothing about whether the aspects carry complementary information versus redundant paraphrases of the outliner prompt.\n\nSo the architecture is plausible, but the evidence is not. The paper needs: full benchmark results (no arbitrary subsets), multiple seeds or at least variance estimates, a proper significance test, and an ablation that removes the reasoning agent alone to show reflection actually helps. Without those, the load-bearing claim is unverified.\n\nWho is this for? Researchers working on prompt-based VLM inference might find the decomposition worth trying, but they should not cite it as evidence of gains. I would not cite it yet; I would bring it to a reading group as a case study in overclaiming from small experiments.\n\nFor peer review: yes, send it out. The idea is coherent and testable, the code is available, and a serious referee could push the authors to run the missing controls. That is exactly what peer review is for.","headline":"MARIC is a clean reconfiguration of multi-agent prompting for zero-shot VLM classification, but its own Table 1 undermines the 'consistent significant gains' claim and the evaluation lacks the rigor to support it.","tokens_in":7097,"tokens_out":1143,"would_cite":false,"duration_ms":14864,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MARIC claims that decomposing image classification into a multi-agent pipeline—global outline, three aspect-specific descriptions, and reflective synthesis—improves zero-shot VLM accuracy across four benchmarks.","keywords":["multi-agent reasoning","image classification","vision-language models","zero-shot classification","prompt decomposition","reflective synthesis","interpretability","LLaVA"],"falsifier":"Run MARIC while measuring the pairwise textual similarity of the three aspect descriptions for a sample of images; if the average similarity is high (e.g., >0.8 with standard embedding models), the claimed orthogonality is absent. Additionally, an ablation that keeps only one Aspect Agent (but retains the Reasoning Agent) that achieves accuracy close to the full framework would indicate the multi-agent decomposition contributes little beyond the reasoning/reflection prompt.","tokens_in":6203,"feed_emoji":"🧩","tokens_out":1703,"duration_ms":19154,"temperature":0.7,"pith_summary":"MARIC is a training-free framework that reworks image classification as a collaborative reasoning process: an Outliner Agent proposes three targeted prompts, three Aspect Agents produce fine-grained descriptions along distinct visual dimensions, and a Reasoning Agent reflects on these descriptions before outputting a label with an explicit reasoning trace. The paper claims this decomposition outperforms direct VLM generation, chain-of-thought prompting, and a single-agent reasoning baseline on CIFAR-10, OOD-CV, Weather, and Skin Cancer datasets, using LLaVA 1.5 models. MARIC aims to overcome the limits of single-pass VLM representations and parameter-heavy fine-tuning while adding interpretability through structured reasoning. A sympathetic reader would see the contribution as a prompt-driven, agent-based alternative to monolithic VLM inference for classification.","feed_headline":"Multi-agent prompts beat single-pass VLM classification","feed_subtitle":"A three-agent pipeline—outline, aspect descriptions, reflective reasoning—lifts zero-shot accuracy on four benchmarks.","key_machinery":"The central mechanism is the three-agent decomposition: the Outliner Agent (G_out) generates targeted prompts P = {p1, p2, p3} using a prefix–postfix structure to ensure focused, orthogonal attention; the Aspect Agents (G_asp) each produce a fine-grained description d_i from the image conditioned on prompt p_i; and the Reasoning Agent (G_rea) takes the set D = {d1, d2, d3}, performs an integrated reflection step, and outputs a reasoning trace and label <reasoning>r</reasoning><answer>ŷ</answer>. The paper replaces the conventional classification head with this reflective synthesis, approximating C_θ(I) ≈ G_rea(I, S_rea|D).","core_discovery":"The paper introduces a multi-agent framework that reformulates image classification as a three-stage collaborative reasoning process. First, an Outliner Agent analyzes the image's global theme and generates a set of targeted prompts. Three Aspect Agents then each produce fine-grained descriptions of the image from complementary visual perspectives, guided by those prompts. Finally, a Reasoning Agent synthesizes all descriptions into a unified reasoning trace and final prediction, incorporating an explicit reflection step that critiques and filters inconsistencies before deciding. The authors report consistent accuracy gains over direct generation, chain-of-thought prompting, and a single-age","pith_inferences":["A testable extension is to vary the number of Aspect Agents (n = 2, 4, 5) and measure accuracy; the paper's fixed n=3 may not be optimal for all datasets, and the paper itself flags this as a limitation.","The claimed orthogonality of aspects could be directly measured by computing textual similarity (e.g., cosine similarity of descriptions) across aspects; high overlap would suggest the gains come mostly from the reasoning agent's reflection, not the decomposition.","If the reflection step is the true source of gains, a simpler single-agent prompt that instructs 'before answering, describe three different visual aspects and then critique them' might match MARIC's accuracy, which would challenge the need for separate agents.","The framework could likely be transferred to other vision-language tasks such as visual question answering or fine-grained recognition, where multi-perspective description and reflective synthesis could help disambiguate subtle visual differences."],"forward_implications":["If correct, MARIC demonstrates that zero-shot VLM image classification can be improved without retraining or fine-tuning, purely by structuring prompts and reasoning across multiple perspectives.","The explicit reasoning trace provides a transparent, inspectable justification for each prediction, which could make VLM classifiers more trustworthy in application settings.","The framework's gains on OOD-CV suggest that multi-perspective description and reflection may improve robustness to out-of-distribution shifts compared to single-pass inference.","Because MARIC uses off-the-shelf VLMs with temperature set to zero, the method can be applied to any instruction-following VLM without architectural changes."],"fun_headline_variants":["Three agents beat single-pass VLM classification","Collaborative reasoning improves image classification","Multi-agent pipeline lifts zero-shot classification on 4 benchmarks","Outliner, aspects, reflection: multi-agent wins image classification"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"MARIC's gains depend on the premise that the three Aspect Agents produce genuinely orthogonal and complementary visual descriptions, rather than redundant paraphrases of the same global prompt, and that the Reasoning Agent's reflection step actually corrects errors rather than just rephrasing.","fun_headline_variants_meta":{"raw":{"variants":["Three agents beat single-pass VLM classification","Collaborative reasoning improves image classification","Multi-agent pipeline lifts zero-shot classification on 4 benchmarks","Outliner, aspects, reflection: multi-agent wins image classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000537,"raw_usage":{"total_tokens":2387,"prompt_tokens":686,"completion_tokens":1701,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":430,"completion_tokens_details":{"reasoning_tokens":1641}},"tokens_in":430,"tokens_out":1701,"duration_ms":12333,"temperature":1.0,"reasoning_tokens":1641,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:12:46.581583+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MARIC while measuring the pairwise textual similarity of the three aspect descriptions for a sample of images; if the average similarity is high (e.g., >0.8 with standard embedding models), the claimed orthogonality is absent. Additionally, an ablation that keeps only one Aspect Agent (but retains the Reasoning Agent) that achieves accuracy close to the full framework would indicate the multi-agent decomposition contributes little beyond the reasoning/reflection prompt.","supporting_citations":[],"review_version":1}