{"id":"bb5a7883-4037-4dd5-abe2-b2a6e7f8c9a3","arxiv_id":"2507.09139","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PoseLLM swaps LocLLM's linear vision-language projector for a two-layer MLP with GELU, reporting +0.4 AP on COCO (77.8) with comparable zero-shot transfer.","lead":"A language-guided pose estimation model replaces the linear projector used in LocLLM with a two-layer MLP, reporting 77.8 AP on COCO, up 0.4 points from LocLLM. The gain is small and the comparison lacks a same-setting ablation, so the result is conditional on further verification.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +0.4 AP gain is credited to the MLP connector, but PoseLLM does not retrain LocLLM's linear projector under its own recipe or report seed variance, so the causal attribution is not yet established.","rationale":"The paper's forward-looking claim is that a nonlinear MLP connector causes a 0.4 AP improvement over LocLLM and that this gain does not come at the cost of zero-shot generalization. For that claim to hold, PoseLLM and LocLLM would need to differ only in the connector, or at least the training differences would need to be controlled. That condition is not demonstrated. The note in Table 2 that all results use official model weights implies the LocLLM baseline was not retrained under PoseLLM's hyperparameters, so the comparison conflates connector choice with training differences. There is also no multi-seed error bar, so 77.4 versus 77.8 on COCO val is not established as a statistically meaningful effect. The reader's weakest assumption identifies exactly this issue, and my read agrees. The manuscript is otherwise internally consistent and provides a code link, which is a positive but unverified step. The missing evidence is empirical rather than conceptual, so the appropriate disposition remains conditional pending a controlled linear-versus-MLP ablation under identical conditions with multiple seeds.","tokens_in":8232,"tokens_out":4744,"duration_ms":57293,"concrete_test":"Using the released code, rerun the exact PoseLLM recipe but replace only the two-layer GELU MLP (Eqs. 3-4) with a single linear projection to 4096 dimensions, matching initialization and random seed. Train at least three seeds per connector type, evaluate on COCO val under the same protocol, and report per-seed AP with mean and standard deviation. Also run LocLLM's official pipeline in the same evaluation harness for one seed. If the linear connector's mean AP is within one standard deviation of 77.8 or exceeds it, the +0.4 AP causal claim fails; if not, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that replacing LocLLM's linear vision-language projector with a two-layer MLP improves COCO AP by +0.4 while preserving zero-shot generalization. This claim rests on the comparison in Table 2 and Section 4.2. The comparison is not a controlled ablation: Table 2 states that all results are obtained using official model weights, meaning the LocLLM row is a pretrained checkpoint produced by a different training pipeline. PoseLLM's own training setup (12 epochs, batch size 32, lr 5e-4, LoRA on both the DINOv2 vision encoder and Vicuna-7B, 224x224 inputs, AdamW with weight decay 0.05) is reported only for PoseLLM. There is no evidence that LocLLM was trained under the same settings, and the paper does not run a linear-projector baseline under its own configuration. Furthermore, only a single run is reported, so the 0.4 AP gap is within plausible seed-to-seed and protocol variation for COCO val. The same issue applies to Table 3, where the Human-Art and MPII differences between PoseLLM and LocLLM are at most +0.1, well within the noise of a single run. Without a matched linear baseline and variance estimates, the improvement cannot be attributed to nonlinearity rather than to differences in LoRA rank, learning schedule, resolution, or evaluation details.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PoseLLM, a language-guided human pose estimation model that replaces LocLLM's linear vision-language projector with a two-layer MLP with GELU activation. The model is trained on COCO only and reports 77.8 AP on the COCO validation set, out-performing LocLLM by 0.4 AP, while claiming comparable zero-shot generalization on Human-Art and MPII. The method section describes the architecture (DINOv2 ViT-L/14 encoder, Vicuna-7B LLM, LoRA fine-tuning, two-layer MLP connector), and the experiments compare against heatmap-based, regression-based, and language-based baselines on COCO, Human-Art, and MPII. The central claim is that the nonlinear connector improves localization accuracy without sacrificing cross-dataset generalization.","tokens_in":8501,"tokens_out":2813,"duration_ms":32266,"significance":"If the claimed causal effect were established, the paper would provide a simple, reproducible architectural change—replacing a linear projector with a two-layer MLP—that improves an existing LLM-based keypoint localization baseline. The idea is plausible and consistent with practices in general vision-language models such as LLaVA-1.5. A strength is that the code is promised to be released, which would facilitate verification. However, the empirical evidence as presented is insufficient to support the central attribution: the LocLLM comparison is not a controlled ablation, no variance is reported, and the reported gains (0.4 AP on COCO, at most 0.2 on cross-dataset metrics) are small relative to typical run-to-run and protocol variation. The paper's contribution would be more convincing if the authors retrained a linear-projector baseline under their exact training recipe and reported multiple seeds.","major_comments":[{"comment":"The +0.4 AP improvement over LocLLM is not a controlled comparison. Table 2's caption states that all results were obtained using official model weights, so the LocLLM row comes from a model trained with a different pipeline, not under PoseLLM's training configuration (12 epochs, batch size 32, learning rate 5e-4, LoRA on DINOv2 and Vicuna-7B, 224×224 inputs). Any of these differences—LoRA rank and alpha, learning schedule, resolution, or evaluation details—could account for the gap. To support the claim that the nonlinear connector causes the improvement, the authors must retrain a linear-projector baseline under the identical PoseLLM recipe and compare it against the MLP connector under identical conditions.","section":"§4.2, Table 2"},{"comment":"Only a single run is reported, with no error bars or seed-to-seed variance. The COCO improvement is 0.4 AP and the cross-dataset differences in Table 3 are at most about 0.1–0.2, which is within plausible variation for a single training run on COCO val. Please report results across at least three seeds (mean and standard deviation) and state whether the differences are statistically distinguishable. Without this, the central causal claim is not yet established.","section":"§4.2, Tables 2 and 3"},{"comment":"The text states that PoseLLM shows 'slight improvements' on Human-Art and MPII, but several columns in Table 3 show decreases: Human-Art AP50 drops from 88.6 to 87.7, and MPII Elbow drops from 98.2 to 98.1. Given the absence of variance estimates, the data support only parity with LocLLM on cross-dataset generalization, not improvement. Please adjust the wording to match the numbers and provide matched comparisons before claiming that generalization is 'maintained' or 'slightly improved.'","section":"§4.2, Table 3"}],"minor_comments":[{"comment":"The symbol for V* appears as 'V ∗ [20]' with a formatting artifact; it should be rendered as 'V*' consistently.","section":"§2.1"},{"comment":"The section heading contains a typo: 'T ext Encoding' should be 'Text Encoding'.","section":"§3.2"},{"comment":"The figure text contains 'Vison-Language Connector' twice; this should be 'Vision-Language Connector'.","section":"Figure 1"},{"comment":"The MPII column labeled 'Mean0.1' should be labeled 'PCKh@0.1' for clarity and consistency with the evaluation metrics described in Section 4.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a vision-language or pose-estimation venue, and the proposed modification is simple enough to be thoroughly validated with a controlled experiment. The central issue is that the current evidence does not support the causal attribution to the MLP connector; however, this is fixable by retraining a linear-projector baseline under the same recipe and reporting variance. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a modest but legitimate incremental paper. The novel bit is narrow—replacing LocLLM's linear projector with a two-layer MLP+GELU for LLM-based keypoint localization—and it is honestly positioned as such. The paper clearly cites LLaVA-1.5 and EarthDial as the source of the MLP-connector idea, and it correctly notes that those works target semantic reasoning, not coordinate regression. The writing is clean, the architecture description is precise, and the authors state their training recipe (12 epochs, batch 32, lr 5e-4, LoRA on both encoder and LLM) and provide code. That is real evidence and deserves credit.\n\nWhere it goes soft is exactly where the reader's stress-test lands: the central claim—that the MLP causes the +0.4 AP gain—is not actually tested. Table 2 compares PoseLLM against LocLLM's published checkpoint, not against a linear projector retrained under PoseLLM's own recipe. The gap is 0.4 AP on COCO val, and the cross-dataset differences in Table 3 are at most 0.1 AP. With a single run and no seed variance, that gap is within the noise floor for COCO val. The paper would be much stronger with a matched ablation: same data, same LoRA config, same schedule, only the projector changed, plus error bars over at least three seeds. As written, the causal attribution to nonlinearity is plausible but not established.\n\nOther concerns are minor. The \"first LLM-based pose estimation framework with MLP connector\" novelty claim is accurate within the cited literature, though it is a small delta. The self-citations (refs 25, 26) are related prior work and not load-bearing. No circularity issues—this is an empirical comparison, not a fitted derivation. The qualitative figure is fine but adds little.\n\nWho is this for? Practitioners working on language-guided pose estimation or VLM spatial reasoning tasks. They will find a clean baseline and a clear recipe. A serious referee should see this because the core idea is testable and the paper is honest—it just needs the matched ablation and variance reporting to back its headline claim. I'd accept it for peer review with a request for that experiment.\n\nRecommendation: send it out, but push for the controlled linear-vs-MLP ablation before acceptance.\n\nBest,\n[You]","headline":"A clean, honest incremental paper: the MLP connector swap is real and reproducible-ish, but the +0.4 AP claim is not yet causally established because the baseline is a published number, not a matched retrain.","tokens_in":9048,"tokens_out":632,"would_cite":false,"duration_ms":8708,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing a linear projector with a two-layer MLP raises language-guided pose estimation from 77.4 to 77.8 AP on COCO while keeping zero-shot transfer.","keywords":["human pose estimation","keypoint localization","vision-language models","MLP connector","GELU activation","zero-shot generalization","LLM-based pose estimation"],"falsifier":"Train the exact PoseLLM setup (12 epochs, batch size 32, learning rate 5e-4, LoRA on DINOv2 and Vicuna, 224x224 input) with the connector replaced by a single linear layer and run several seeds; if the linear version matches or exceeds 77.8 AP, the MLP is not the causal factor.","tokens_in":8027,"feed_emoji":"🧍","tokens_out":5054,"duration_ms":53910,"temperature":0.7,"pith_summary":"PoseLLM claims that a nonlinear vision-language connector is enough to make an LLM-based pose estimator more precise. The paper replaces LocLLM's single linear projection between the vision encoder and Vicuna-7B with a lightweight two-layer MLP with GELU activation. Trained only on COCO keypoint data, it reports 77.8 AP on the COCO validation set, 0.4 AP above LocLLM, while retaining comparable zero-shot results on Human-Art and MPII. If correct, the result points to the connector rather than model scale as the bottleneck in language-guided keypoint localization.","feed_headline":"A two-layer MLP lifts LLM pose estimation to 77.8 AP","feed_subtitle":"Replacing LocLLM's linear projector with a small GELU MLP adds +0.4 AP on COCO and keeps zero-shot transfer.","key_machinery":"The central mechanism is the nonlinear vision-language connector, a two-layer MLP with GELU nonlinearity: $Z = \\mathrm{GELU}(I W_1 + b_1)$, $V = Z W_2 + b_2$, where $I$ is the sequence of 1024-dimensional patch features from DINOv2 and $V$ is the 4096-dimensional token sequence fed to the LLM. It replaces LocLLM's single linear projector and is the only architectural change evaluated. The MLP is fully trained while the vision encoder and LLM are adapted with LoRA, so the experiment isolates the effect of connector nonlinearity.","core_discovery":"The paper's central claim is that the connector between the visual encoder and the language model is a bottleneck for LLM-based keypoint localization, and that a nonlinear connector removes that bottleneck. Concretely, PoseLLM replaces LocLLM's single linear projection with a two-layer MLP using GELU activation, mapping DINOv2 patch features into Vicuna-7B's input space. On the COCO validation set, the paper reports 77.8 AP versus LocLLM's 77.4 AP, with similar or slightly better cross-dataset numbers on Human-Art and MPII under zero-shot evaluation. The authors present this as establishing a stronger baseline for language-guided pose estimation and as evidence that lightweight nonlinear alignment helps fine-grained spatial tasks.","pith_inferences":["An immediate test is per-keypoint AP: if nonlinearity matters, the gain should concentrate on joints with strong spatial-textual ambiguity such as elbow, wrist, and ankle rather than nose and eyes, and the paper does not report that breakdown.","The same MLP connector could be dropped into other VLM spatial tasks such as object keypoint detection or dense correspondence, where the linear-projector bottleneck is likely similar; the paper does not test these tasks.","Under the assumption that the +0.4 AP is causal, wider or deeper connectors might buy further precision, but saturation is plausible because the MLP is trained from scratch while the backbones are LoRA-constrained."],"forward_implications":["PoseLLM becomes the new numerical baseline for LLM-based pose estimation on COCO, ahead of the language-based methods in Table 2.","The parity of cross-dataset scores on Human-Art and MPII indicates that the added nonlinearity does not come at the cost of zero-shot flexibility.","The improvement is achieved without scaling the LLM or the training data, pointing to connector design as a low-cost axis for spatial accuracy.","Because the connector adds only a few parameters, the same replacement can be applied to other LocLLM-style models to test whether the gain transfers."],"supporting_citations":[{"why":"Supplies the LocLLM baseline, the linear-projector paradigm, the instruction format, and the comparison numbers that PoseLLM claims to beat.","marker":"[19]"},{"why":"Provides the MLP connector design used in LLaVA-1.5 that PoseLLM adapts for spatial regression.","marker":"[13]"},{"why":"DINOv2 ViT-L/14 is the vision encoder whose 1024-dimensional patch features enter the MLP.","marker":"[15]"},{"why":"Vicuna-7B is the language backbone and tokenizer that consumes the concatenated visual and text tokens.","marker":"[5]"},{"why":"Defines the GELU activation placed between the two MLP layers.","marker":"[6]"},{"why":"COCO Keypoint is the training set and primary evaluation benchmark for the main AP numbers.","marker":"[12]"},{"why":"Human-Art is the zero-shot cross-dataset evaluation target.","marker":"[8]"},{"why":"MPII is the second zero-shot evaluation target with PCKh metrics.","marker":"[2]"},{"why":"LoRA is the parameter-efficient fine-tuning method applied to both the vision encoder and the LLM.","marker":"[7]"}],"fun_headline_variants":["MLP connector lifts LLM pose estimation to 77.8 AP","Replacing linear projector with GELU MLP adds 0.4 AP","Nonlinear alignment boosts language-guided pose estimation","PoseLLM: Two-layer MLP beats linear projector for keypoints"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on comparing PoseLLM's 77.8 AP with LocLLM's published 77.4 AP, assuming both used the same training schedule, resolution, LoRA setup, and evaluation protocol, because the paper does not retrain the linear-projection baseline under its own configuration.","fun_headline_variants_meta":{"raw":{"variants":["MLP connector lifts LLM pose estimation to 77.8 AP","Replacing linear projector with GELU MLP adds 0.4 AP","Nonlinear alignment boosts language-guided pose estimation","PoseLLM: Two-layer MLP beats linear projector for keypoints"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00021,"raw_usage":{"total_tokens":1398,"prompt_tokens":921,"completion_tokens":477,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":401}},"tokens_in":537,"tokens_out":477,"duration_ms":5447,"temperature":1.0,"reasoning_tokens":401,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:02:21.672960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the exact PoseLLM setup (12 epochs, batch size 32, learning rate 5e-4, LoRA on DINOv2 and Vicuna, 224x224 input) with the connector replaced by a single linear layer and run several seeds; if the linear version matches or exceeds 77.8 AP, the MLP is not the causal factor.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the LocLLM baseline, the linear-projector paradigm, the instruction format, and the comparison numbers that PoseLLM claims to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Vicuna-7B is the language backbone and tokenizer that consumes the concatenated visual and text tokens."},{"cited_title":"In: Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13","cited_arxiv_id":null,"evidence_quote":"COCO Keypoint is the training set and primary evaluation benchmark for the main AP numbers."},{"cited_title":"In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","cited_arxiv_id":null,"evidence_quote":"Human-Art is the zero-shot cross-dataset evaluation target."},{"cited_title":"In: Proceedings of the IEEE Conference on computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"MPII is the second zero-shot evaluation target with PCKh metrics."},{"cited_title":"ICLR1(2), 3 (2022)","cited_arxiv_id":null,"evidence_quote":"LoRA is the parameter-efficient fine-tuning method applied to both the vision encoder and the LLM."}],"review_version":1}