{"id":"f61c1d26-e3a9-4422-a53c-edca890af0eb","arxiv_id":"2508.08066","paper_version":2,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic visual-grounding design study on LLaVA-1.5 identifies the best paradigm and data design, improving RefCOCO/+/g by +5.6/+6.9/+7.0 over the baseline.","lead":"Using LLaVA-1.5, this paper tests which fine-tuning designs make multimodal AI point at the right objects, and reports gains of 5.6 to 7.0 points on the RefCOCO/+/g benchmarks. It matters as a possible default recipe for grounding fine-tuning, replacing ad hoc choices with tested ones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Selection bias in design search could inflate headline RefCOCO gains; the abstract gives no evidence that ablations were separated from the final evaluation split.","rationale":"The reader's verdict was CONDITIONAL with low confidence, listing two load-bearing premises: cross-architecture transfer and selection bias. I agree that both are open, but the selection-bias premise is the more fundamental one: if the design was selected using the same benchmark family, the headline improvements are not a trustworthy estimate of the design's effect even on LLaVA-1.5. The reader identified this as premise 2, so we partially agree. My concrete test directly checks whether the protocol separates design selection from final evaluation, and includes a held-out benchmark test to quantify selection inflation. I do not propose changing the verdict because the concern is unresolved, not confirmed; CONDITIONAL remains the appropriate verdict until the protocol is inspectable and the check is run.","tokens_in":9700,"tokens_out":2887,"duration_ms":37241,"concrete_test":"Obtain the full paper and inspect the experimental protocol. Specifically: (1) Identify the split used for the final reported numbers (e.g., RefCOCO val, refcoco test) and the split used for the paradigm/data design ablations. (2) If they are the same split, the selection-bias concern lands; if different, count the number of design comparisons and check whether any inference was made from the final split during development. (3) As a stronger test, apply the final recipe to a held-out grounding benchmark never used in the paper (e.g., Flickr30K Entities or ReferItGame) and compare the gain over the baseline to the RefCOCO gains. If the gain disappears or reverses, selection bias is likely.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a specific grounding paradigm and data design yield +5.6/+6.9/+7.0 over LLaVA-1.5 on RefCOCO/+/g. This claim is only valid if the design was selected without using the final benchmark split. If the paradigm and data ablations were tuned on the same split (or even on test after many comparisons), the reported gains reflect selection bias, not a transferable improvement. The abstract provides no protocol information—no indication of which split was used for ablations, how many configurations were compared, or whether a separate validation set was held out. Given the typical size of such design searches (several paradigms × multiple data designs), even a few percent improvement can be an artifact of multiple comparisons. This concern is more load-bearing than the cross-architecture transfer claim because it questions whether the headline numbers are real for LLaVA-1.5 itself, not just whether they generalize. Without access to the full experimental protocol, the risk remains unresolved.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript, as available in the supplied material, consists of a readable abstract and a full text that is almost entirely corrupted mojibake, including a header from an unrelated astro-ph paper (arXiv:2508.08072v1). The abstract describes a systematic empirical study of visual grounding (VG) design choices in multimodal large language models (MLLMs). Using LLaVA-1.5 as the testbed, the authors claim to identify the most effective VG paradigm and an optimized grounding-data design, reporting improvements of +5.6% / +6.9% / +7.0% on RefCOCO/+/g over the LLaVA-1.5 baseline. The abstract further claims these findings are 'broadly applicable and extendable to other architectures' on the basis of LLaVA-1.5's popularity rather than cross-architecture experiments. Because the full text is unreadable, none of the experimental protocol, ablation tables, hyperparameters, evaluation splits, or error bars can be verified.","tokens_in":9808,"tokens_out":3056,"duration_ms":39090,"significance":"If the empirical claims are correct, this paper would be a useful contribution to the somewhat fragmented literature on VG fine-tuning for MLLMs: a systematic comparison of paradigms and data-design choices on a widely used baseline, with concrete performance gains. The paper could provide practical guidance for practitioners. However, the significance cannot currently be assessed: the central evidence is a set of numbers in the abstract, with no legible methods or results. The paper does not appear to ship machine-checked proofs, reproducible code, or parameter-free derivations; it is an empirical design study whose value depends entirely on the transparency and soundness of the experimental protocol, neither of which is available in the supplied manuscript.","major_comments":[{"comment":"The entire manuscript body is corrupted mojibake; it is not a readable research paper. The full text contains a header from arXiv:2508.08072v1 (an astro-ph paper) and no legible sections, equations, tables, or figures. The central claims of the paper are empirical and are presented only in the abstract. Without a readable description of the VG paradigms, data designs, fine-tuning details, evaluation metrics, and ablations, the reported +5.6/+6.9/+7.0 improvements cannot be checked. This is a load-bearing issue: the paper's contribution is an empirical design guide, and the absence of legible evidence blocks acceptance.","section":"Full Text"},{"comment":"The abstract does not report the evaluation protocol. It gives no information about which split of RefCOCO/+/g was used for ablations versus final evaluation, how many configurations were compared, whether a separate validation set was held out, or whether the reported numbers are averages over multiple runs with variance. Because the paper selects a 'winning' paradigm and data design, the possibility of selection bias on the same benchmark family is real and not addressed. If the design choices were tuned on RefCOCO/+/g validation and the final numbers are on the corresponding test sets, the gains may partly reflect overfitting to the benchmark. The authors must explicitly describe the split discipline, the number of configurations searched, and any significance/error-bar analysis. This is essential to establish that the headline numbers are transferable improvements rather than artifac","section":"Abstract"},{"comment":"The claim that the findings are 'broadly applicable and extendable to other architectures' is not supported by any experiment reported in the abstract or legible in the full text. The justification given is that LLaVA-1.5 is 'widely adopted,' which is a popularity argument, not an empirical transferability result. Design-space conclusions from a single architecture may not hold for other MLLMs with different visual encoders, projection layers, or instruction-tuning schemes. The authors should either add cross-architecture experiments (e.g., on a second MLLM family) or substantially soften the generalizability claim to what can be legitimately inferred from one model.","section":"Abstract"}],"minor_comments":[{"comment":"The reported gains are given as '+5.6% / +6.9% / +7.0%' without stating whether these are absolute percentage-point improvements in accuracy (e.g., Acc@0.5) or relative improvements. Please specify the metric, the baseline values, and the exact evaluation setting (e.g., RefCOCO testA/testB splits) for each number.","section":"Abstract"},{"comment":"The manuscript text is corrupted and includes an unrelated astro-ph header. This is a critical presentation issue that makes the paper impossible to review. The authors should resubmit a clean, readable version.","section":"Full Text"},{"comment":"No code, configuration files, or data-processing scripts appear to be provided. For an empirical design-space study, releasing the exact training/evaluation code is important for reproducibility and for the claimed 'extendable' findings to be adopted by the community.","section":"Full Text"}],"recommendation":"uncertain","confidential_remarks":"The supplied full text is so corrupted that I cannot evaluate the methods or results. The abstract alone raises two significant concerns: potential selection bias in the design search and unsupported cross-architecture generalizability. I would recommend that the editor ask the authors to provide a complete, readable manuscript, and then have it reviewed with specific attention to the evaluation protocol and the split discipline. If the full text is genuinely as provided, this is not ready for peer review in its current form. I chose 'uncertain' rather than 'reject' because the corruption appears to be a submission/pipeline issue rather than a demonstrated scientific error; however, the authors must also address the protocol concerns before any positive recommendation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I couldn't read the paper. The full text I got is a corrupted extraction, full of mojibake and with a header from an unrelated astro-ph paper. So this is an abstract-only review, and my confidence is accordingly low.\n\nThat caveat aside, the paper has a real virtue on its face. A lot of visual grounding fine-tuning work makes arbitrary choices about output format, data mixing, and training objectives, and nobody has done a clean, systematic comparison across paradigms and data designs. The authors take a widely used base model (LLaVA-1.5) and report consistent gains of +5.6/+6.9/+7.0 on RefCOCO/+/g. For practitioners, a tested default recipe is genuinely useful, and the claims are falsifiable.\n\nThe soft spots are the usual ones for this kind of design-space paper, and the abstract doesn't rule them out. The biggest is selection bias: if the winning paradigm and data design were chosen after looking at validation performance on the same benchmark family that produces the headline numbers, then some fraction of those gains is just multiple comparisons. The abstract gives no protocol detail on what split was used for ablations, how many configurations were compared, or whether there was a held-out validation set. Second, no error bars or seed variance, so we don't know if +5.6 is meaningful. Third, the cross-architecture generalizability claim rests on the popularity of LLaVA-1.5, not on an actual experiment. That's a weaker claim, but it should be softened or tested.\n\nThe corrupted full text is a review problem, not a paper problem, but it means I can't verify any of the ablation tables or method details. The authors should ship code and data; that would resolve most of my concerns.\n\nWho is this for? People who fine-tune MLLMs for grounding and want a starting point, and anyone doing ablation-heavy empirical work who wants to see how a design search should be reported. I would send it to peer review, because the question is timely and the claims are testable. But I would tell the authors that the protocol details—split hygiene, number of configurations, seeds—are essential, and that one cross-architecture experiment would substantially strengthen the generalization claim.","headline":"A plausible design-space study for visual grounding that I couldn't actually read; the abstract's claims are useful but unverifiable without protocol details.","tokens_in":10391,"tokens_out":2972,"would_cite":false,"duration_ms":31516,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a specific, reproducible combination of visual-grounding output format and fine-tuning data design lifts LLaVA-1.5 by +5.6/+6.9/+7.0 points on RefCOCO/+/g, and that the design findings should transfer to other multimo","keywords":["visual grounding","multimodal large language models","fine-tuning","referring expression comprehension","RefCOCO","design space exploration","LLaVA-1.5","ablation study"],"falsifier":"Run the winning paradigm and data recipe on a second MLLM with a different architecture and compare RefCOCO/+/g against that model's own baseline; if the gains shrink to zero or reverse, the broad-applicability claim fails.","tokens_in":9492,"feed_emoji":"🎯","tokens_out":5052,"duration_ms":55929,"temperature":0.7,"pith_summary":"Visual grounding—getting an MLLM to point at the image region named by a phrase—is central to fine-grained multimodal ability, yet existing systems fine-tune for it with ad hoc, unverified choices. This paper tries to settle which choices matter by systematically varying two things on LLaVA-1.5: the grounding paradigm, or how the model is asked to express a location, and the design of the grounding data used in fine-tuning. A sympathetic reading is that the paper identifies one specific combination as best and shows that it lifts RefCOCO/+/g scores by +5.6/+6.9/+7.0 over the baseline. If that recipe transfers, future MLLM grounding work can stop guessing at these design choices and adopt a tested configuration.","feed_headline":"A fine-tuning recipe lifts visual grounding by 5.6 to 7.0 points","feed_subtitle":"Systematic tests on LLaVA-1.5 isolate the output format and data design that deliver grounding gains.","key_machinery":"The central machinery is the pair of controlled comparisons: (1) visual-grounding paradigms—the different ways an MLLM can be asked to output a located object, such as textual coordinate formats versus alternative grounding-token schemes—and (2) grounding-data design ablations, which vary the composition and format of the fine-tuning examples. Combining the winning paradigm with the optimized data design on LLaVA-1.5 is what produces the reported gains.","core_discovery":"On the paper's own terms, design choices in visual-grounding fine-tuning are not interchangeable. Among the paradigms for turning an image region into model output, and among the ways of constructing grounding data for fine-tuning, one combination wins consistently. The evidence is a set of ablations on LLaVA-1.5; the final configuration improves the RefCOCO/+/g benchmarks by +5.6/+6.9/+7.0 percentage points over the standard LLaVA-1.5 baseline. The paper further argues that, because it used a widely adopted model as its testbed, the identified design is broadly applicable and extendable to other architectures.","pith_inferences":["Not tested in the paper: whether the winning configuration transfers across architectures; the abstract asserts transfer from LLaVA-1.5's popularity, so transfer is an assumption rather than a demonstrated result.","Not tested in the paper: whether the design decisions were selected using the same RefCOCO family that reports the headline numbers; if so, part of the gain could be selection rather than generalizable improvement.","Because the benchmarks are referring-expression comprehension splits, the design advice may not extend to open-vocabulary grounding, video grounding, or 3D grounding without further experiments.","A natural extension would be to take the same recipe to a held-out grounding benchmark never used in the ablations, to measure whether the gains reflect genuine alignment rather than benchmark fitting."],"forward_implications":["Using the identified paradigm and data recipe on LLaVA-1.5 yields +5.6% on RefCOCO, +6.9% on RefCOCO+, and +7.0% on RefCOCOg over the untuned baseline.","Because the winning setup comes from systematic ablations rather than an ad hoc choice, other MLLMs can adopt the same output format and data recipe without redesigning their architecture.","The study isolates which design choices matter, letting future work skip ineffective variations and spend compute on the components that actually move grounding accuracy.","The data-design ablations show that the composition and format of grounding training data, not only the model architecture, determine how well an MLLM localizes described objects."],"supporting_citations":[],"fun_headline_variants":["Visual grounding fine-tuning: design choices shift RefCOCO by up to 7","Systematic ablations on LLaVA-1.5 identify top visual grounding setup","VG fine-tuning: output format and data design boost RefCOCO by ~7","Design choices in grounding fine-tuning yield up to +7.0 on RefCOCO","LLaVA-1.5 grounding: output format and data design made a 7-point difference"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that design choices found on LLaVA-1.5 transfer to other MLLMs; the paper reports no experiment on a second architecture to test that.","fun_headline_variants_meta":{"raw":{"variants":["Visual grounding fine-tuning: design choices shift RefCOCO by up to 7","Systematic ablations on LLaVA-1.5 identify top visual grounding setup","VG fine-tuning: output format and data design boost RefCOCO by ~7","Design choices in grounding fine-tuning yield up to +7.0 on RefCOCO","LLaVA-1.5 grounding: output format and data design made a 7-point difference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000984,"raw_usage":{"total_tokens":4020,"prompt_tokens":758,"completion_tokens":3262,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":3164}},"tokens_in":502,"tokens_out":3262,"duration_ms":24233,"temperature":1.0,"reasoning_tokens":3164,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:39:59.875522+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the winning paradigm and data recipe on a second MLLM with a different architecture and compare RefCOCO/+/g against that model's own baseline; if the gains shrink to zero or reverse, the broad-applicability claim fails.","supporting_citations":[],"review_version":1}