{"id":"705ead02-36cd-404a-8c34-bf869c8b9d53","arxiv_id":"2507.18311","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"FieldLVLM compresses flow-field images and fine-tunes a vision-language model on text generated by specialist physics models, reporting high accuracy on flow analysis benchmarks that copy those same specialists.","lead":"This paper builds a vision-language model pipeline that turns fluid flow simulation data into text descriptions, then fine-tunes Qwen2.5-VL to answer questions about flow type, Reynolds number, and vortices. The core idea is to use specialist physics models to generate the training text, making the reported accuracy largely a measure of how well the model imitates those specialists.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation labels are not independent of the teacher-generated training captions, so the reported accuracies may reflect memorization of teacher outputs rather than physical understanding.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the evaluation labels are generated by the same kind of specialist models that produced the training captions, so high accuracy could simply be imitation. This is the single most important threat because the paper's contribution is claimed to be improved understanding of field data, not improved memorization of teacher annotations. The comparison to general-purpose LVLMs is vacuous since all baselines score 0/NA, and Table III's +Key data result of 100% is a red flag for answer leakage. A held-out evaluation with disjoint flow configurations and independently computed labels would settle whether the model generalizes or merely reproduces teacher text. Since the paper provides no code, data, or such an experiment, the rejection stands.","tokens_in":11546,"tokens_out":2039,"duration_ms":23360,"concrete_test":"Construct a disjoint test set from FlowBench/CFDBench configurations never used in training captions (e.g., hold out all samples from specific geometries, Reynolds regimes, or meshes). Train FieldLVLM only on teacher captions from the remaining configurations, then evaluate on the held-out set with independently computed ground-truth labels. If held-out accuracy drops substantially below the reported 99.79/97.23/85.41 while in-distribution accuracy stays high, the headline numbers are explained by label leakage or memorization rather than understanding. Additionally report teacher-model accuracy on the held-out set as an upper bound; if FieldLVLM does not exceed it, the method is imitation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that FieldLVLM understands scientific field data. That requires the benchmark labels to be independent of the text used to train the model. Section III.A generates training captions from special-purpose models that output flow category, Reynolds number, vortex parameters, and maximum velocity; Section IV.B evaluates exactly these quantities. The benchmarks are 'extended versions' of FlowBench and CFDBench, the same sources used for training, and no separation is described. A model trained on teacher text can score high by copying teacher outputs, not by reasoning from the field images. The vacuous baselines (0/NA for all three comparison models) and the +Key data ablation jumping to 100% reinforce this concern. The paper does not show held-out generalization, code, data, or error bars. The weakest assumption is therefore false as stated; the evaluation does not establish physical understanding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FieldLVLM, a framework for fine-tuning the Qwen2.5-VL-7B large vision-language model on scientific flow field data (velocity and pressure fields). The method combines a 'field-aware language generation strategy,' in which special-purpose machine-learning models produce textual descriptions of flow type, Reynolds number, vortex parameters, and maximum velocity, with a 'data-compressed multimodal model tuning' stage that maps fields to RGB images, compresses them with VQGAN into discrete tokens, selects 'key values,' and fine-tunes the LVLM with LoRA. The authors report very high accuracies on four tasks (flow categorization 100%, Reynolds number estimation 99.79%, vortex identification 97.23%, field data analysis 85.41%) and claim that FieldLVLM significantly outperforms DeepSeek-VL-7B, LLaVA-v1.6, and Llama-3.2-Vision, all of which score 0/NA. Ablation studies on vortex identification and field data analysis show gains from compression and key-data selection, with the latter jumping to 100%.","tokens_in":11675,"tokens_out":3570,"duration_ms":35516,"significance":"If the claims were sound, the paper would present a useful recipe for adapting LVLMs to scientific field data, including a scalable caption-generation pipeline and a token-compression scheme that addresses input-length limits. The authors also introduce a benchmark from extended FlowBench and CFDBench datasets, which could be a community resource. However, the central empirical claims are not supported: the baseline comparison is vacuous because all comparison models score 0/NA, the evaluation is circular with respect to the teacher-generated training captions, and the reported results show suspicious patterns (e.g., 100% accuracy with key-data selection) without error bars or held-out generalization checks. As presented, the paper does not establish that FieldLVLM understands physical content beyond reproducing teacher outputs.","major_comments":[{"comment":"The central comparative claim is unsupported because all three baselines (DeepSeek-VL-7B, LLaVA-v1.6-7B, Llama-3.2-11B-Vision) score 0/NA on every task. The paper attributes this to 'architectural incompatibilities with scientific data structures,' yet Tables IV and V show these same models producing text responses on the same types of input. If the baselines cannot process the inputs, the evaluation protocol itself is at fault and the scores are meaningless; if they can process the inputs, the 0/NA scores are unexplained. Either way, no valid conclusion about 'significantly outperforming existing methods' can be drawn from this comparison.","section":"§IV.C, Table I"},{"comment":"The evaluation is circular. The training captions are generated by special-purpose models that output exactly the quantities used as evaluation targets: flow classification, Reynolds number, vortex parameters, and maximum velocity (Section III.A). The benchmarks are 'extended versions' of FlowBench and CFDBench, the same sources used for training (Section IV.A), and no train/test split is described. The paper never establishes that the ground-truth labels are independent of the teacher-generated captions, so the reported high accuracies may reflect memorization of teacher outputs rather than physical understanding of the field data. This is the weakest assumption of the paper, and it is not addressed.","section":"§III.A and §IV.B"},{"comment":"The ablation study reports that adding key-data selection raises Field Data Analysis accuracy from 53.94% to 100% (Table III). This is a red flag: a single design choice producing perfect accuracy on a benchmark of over 70,000 samples, without error bars, statistical significance, or any discussion of variance, strongly suggests that the task is trivially solvable by the selected heuristics or that there is leakage between training and evaluation. The paper should provide a detailed analysis of why 100% is achieved and on what subset.","section":"Table III"},{"comment":"The evaluation protocol is under-specified. The paper defines accuracy for each task but does not state how many samples per task, how the 70,000+ samples are partitioned into training, validation, and test sets, or whether any test sample overlaps with training data. It also does not define what '0/NA' means operationally (e.g., failure to produce parseable output, out-of-token-limit errors, or incorrect answers). Without this information, the reported numbers cannot be interpreted or reproduced.","section":"§IV.B and Tables I–III"}],"minor_comments":[{"comment":"The sentence 'Therefore, based on? The principle is to select a key piece of data for focused analysis to enhance the effectiveness of the overall data analysis.' is garbled and contains a dangling question mark; it should be rewritten.","section":"§III.B.1"},{"comment":"Reference [29] is cited as DeepSeek-VL-7B, but the reference list entry is for DeepSeek-Coder-V2, which is a different model; please correct the citation or the reference. Similarly, [30] and [31] are not clearly tied to the classification and detection models used in Section III.A.","section":"References [29]–[31]"},{"comment":"Both tables include the placeholder phrase 'Visual input example, Extreme Ironing:' followed by a source path. 'Extreme Ironing' appears to be an artifact and should be removed or replaced with a proper description of the input image.","section":"Tables IV and V"},{"comment":"Figure 1 is referenced in the text but the caption and the figure content do not appear in the provided manuscript; all figures should be checked for inclusion and correct placement.","section":"Fig. 1"},{"comment":"The metric 'Accuracy (%)' is not fully defined: it is unclear whether it is a per-sample exact match, a tolerance-based match, or an aggregate over a test set, and what the denominator in '0/NA' represents. Please define the metrics precisely.","section":"§IV.B"},{"comment":"The ablation tables report different 'Base Fine-tuning' numbers for the same baseline model on different tasks (82.28% vs 53.94%), which is expected for different tasks, but the caption should clarify that these numbers are task-specific; also, the 0/NA baseline appears in both tables and warrants explanation.","section":"Tables II and III"}],"recommendation":"reject","confidential_remarks":"The paper has a number of presentation issues (placeholder text 'Extreme Ironing', inconsistent references, missing figures) that suggest the manuscript is not yet in polished form. More importantly, the two load-bearing problems—degenerate baseline comparison and circularity between training captions and evaluation labels—cannot be fixed by local edits; they require a fundamentally different experimental design with independent ground truth and meaningful baselines. If the authors can provide that, the underlying idea of combining teacher-model captions with token compression for scientific LVLMs might be worth revisiting in a future submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take: the paper builds a genuine pipeline—VQGAN-compressed flow fields, teacher-generated captions, LoRA fine-tuning of Qwen2.5-VL—and that specific combination is absent from the cited literature. The qualitative examples show the model can produce structured, domain-aware responses, which suggests the engineering is not fake. However, the evaluation is not usable. All three comparison models score 0/NA, so the claimed superiority is vacuous. The ablation table with +Key data at 100% accuracy smells like answer leakage. The deeper problem is circularity: the teacher models that generate training captions also produce the same physical quantities used for evaluation labels (flow type, Reynolds number, vortex parameters, Umax), and the benchmark extends FlowBench/CFDBench, the same sources used for training, with no described separation. The high accuracies are plausibly imitation of the teacher, not physical understanding. There are also smaller issues: no code, data, or error bars, and the text has typos and an incomplete sentence. The central claim that FieldLVLM significantly outperforms existing models is not established. That said, the idea is worth a careful redo; if the authors rerun with real baselines and a held-out test set whose labels are independent of the teachers, the paper could become a legitimate applied contribution. My recommendation: desk reject would be acceptable; do not send to peer review in this form. The method has merit, but the evidence does not support publication.","headline":"A plausible engineering pipeline for adapting LVLMs to flow fields, but the evaluation is circular and the baselines are vacuous—the quantitative claims do not hold up.","tokens_in":644,"tokens_out":1799,"would_cite":false,"duration_ms":38447,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FieldLVLM shows that a general-purpose vision-language model, fine-tuned on compressed flow fields and structured captions generated by specialist models, can classify flow type, compute Reynolds numbers, and identify vortices with high…","keywords":["large vision-language models","field data","VQGAN compression","Reynolds number estimation","vortex identification","multimodal scientific data","flow field understanding","LoRA fine-tuning"],"falsifier":"Take a held-out set of flow fields, generate captions with the specialist models but deliberately corrupt or reverse the stated Reynolds numbers and vortex parameters in the caption text, retrain under the same scheme, and check whether the model still reports the original correct values; if it reports the corrupted values instead, the claimed physical understanding collapses.","tokens_in":11339,"feed_emoji":"🌊","tokens_out":4980,"duration_ms":49257,"temperature":0.7,"pith_summary":"The paper tries to show that a large vision-language model can be made to understand scientific field data, specifically fluid velocity and pressure fields, rather than only natural images and text. The proposed framework, FieldLVLM, first converts raw fields into images, compresses them into a small token budget, and fine-tunes the model on captions produced by a pipeline of specialist machine-learning models. The reported results are high: 100% flow-type classification, 99.79% Reynolds-number estimation, 97.23% vortex identification, and 85.41% general field-data analysis on benchmark datasets, while comparison vision-language models score zero on the same tasks. If these numbers hold, the work would offer a recipe for pointing general-purpose multimodal models at scientific data without building task-specific architectures from scratch.","feed_headline":"Vision-language model hits 99.8% on Reynolds number from flow images","feed_subtitle":"A 256-token compressed field plus specialist-generated captions lets a general AI classify vortices and flow regimes.","key_machinery":"The load-bearing machinery is the two-stage input pipeline. Raw 256-by-256 fields of horizontal velocity, vertical velocity, and pressure are normalized and stacked as RGB channels, then encoded by a pretrained VQGAN into 256 tokens, a 99.6% reduction from the 65,536-token raw representation. In parallel, a handful of representative physical values are selected from the field and fed alongside the tokens, and the specialist-generated textual description is itself converted into an image input. These compressed tokens, key values, and the semantic image are what the fine-tuned language decoder attends to, so the model never sees the raw field as text and stays within the context window.","core_discovery":"The central discovery claimed is that a standard open vision-language model, Qwen2.5-VL-7B, can be adapted to fluid-flow field data by two coordinated interventions: a field-aware language generation strategy that uses specialized models to label flow type, Reynolds number, and vortex structure and then rewrites those labels into consistent text, and a data-compression strategy that maps velocity and pressure fields to an RGB image, compresses it with VQGAN into 256 discrete tokens, and augments the input with a small number of representative key values. With LoRA fine-tuning, the resulting model answers structured questions about flow fields with the accuracies listed above. The paper interprets this as evidence that the bottleneck for vision-language models in science is not the model itself but the format and availability of training data: once field data is compressed and paired with structured physical language, the model can reason about it.","pith_inferences":["Because the training captions were produced by the same specialist models that define the evaluation labels, the high scores may partly reflect the student learning to echo the teacher rather than deriving physics from the field; a version trained on captions whose labels are withheld or corrupted would separate these possibilities.","The same pipeline could generalize to other scientific field data, such as weather maps, medical images, or material microstructures, wherever a specialist model can generate structured captions; that extension is not tested in the paper.","A testable extension is to scramble or drop the key-value channel while keeping the VQGAN tokens; if accuracy collapses, the model is reading the provided numbers rather than the field itself."],"forward_implications":["If the reported accuracy holds, a 256-token compressed representation preserves enough physical information for a vision-language model to compute Reynolds numbers and locate vortices, so token compression is not the limiting factor.","Field-aware language generation provides a scalable way to build multimodal scientific datasets: specialist models supply precise labels and a language model turns them into consistent captions, reducing the need for manual annotation.","The large gap between base fine-tuning and the version with key-value guidance (53.94% to 100% on field-data analysis) implies that explicitly feeding selected informative values is the dominant driver of semantic understanding.","The zero scores of comparison models suggest that, in their default form, general vision-language models cannot even process such long-sequence scientific inputs, making the compression component a prerequisite rather than an enhancement."],"supporting_citations":[{"why":"Supplies the base vision-language model whose context window motivates the compression pipeline.","marker":"[7]"},{"why":"Provides the VQGAN token compressor that reduces the field data to 256 discrete tokens.","marker":"[8]"},{"why":"Serves as the language model that generates question lists and structured field descriptions in the data pipeline.","marker":"[29]"},{"why":"Provides the LoRA parameter-efficient fine-tuning method used to adapt the model.","marker":"[32]"},{"why":"Supplies the FlowBench dataset of flow simulations used for training and evaluation.","marker":"[34]"},{"why":"Supplies the CFDBench dataset, including cavity flow cases used in the vortex identification experiments.","marker":"[35]"},{"why":"One of the main comparison baselines that the paper reports as scoring 0/NA on all tasks.","marker":"[17]"}],"fun_headline_variants":["AI vision model reads flow fields: 99.8% Reynolds accuracy","Compressed flow data lets general AI classify vortices","Field-aware text + 256 tokens: LVLM tackles fluid dynamics","VQGAN compression unlocks scientific reasoning in VLMs","General AI model fine-tuned on flow data hits 99.8% Reynolds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's numbers stand on the assumption that the evaluation labels are independent of the teacher models that generated the training captions, so the reported accuracy reflects genuine physical understanding rather than the model repeating labels it was trained on.","fun_headline_variants_meta":{"raw":{"variants":["AI vision model reads flow fields: 99.8% Reynolds accuracy","Compressed flow data lets general AI classify vortices","Field-aware text + 256 tokens: LVLM tackles fluid dynamics","VQGAN compression unlocks scientific reasoning in VLMs","General AI model fine-tuned on flow data hits 99.8% Reynolds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000339,"raw_usage":{"total_tokens":1895,"prompt_tokens":994,"completion_tokens":901,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":812}},"tokens_in":610,"tokens_out":901,"duration_ms":7583,"temperature":1.0,"reasoning_tokens":812,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:14:42.318145+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of flow fields, generate captions with the specialist models but deliberately corrupt or reverse the stated Reynolds numbers and vortex parameters in the caption text, retrain under the same scheme, and check whether the model still reports the original correct values; if it reports the corrupted values instead, the claimed physical understanding collapses.","supporting_citations":[],"review_version":2}