Artificial Intelligence · arXiv:2610.11231 v1 · submitted 8 Oct 2026 · 76 references
Harness Compilation: Which Decisions Should a Small Vision-Language Model Keep?
The paper says
“Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness.”
more of the abstract
A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. Deployment requires neither weight updates nor teacher calls. Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. Recompilation for a new student model helps when the transferred interface no longer fits the student. With 100 practice items, HC exceeds answer-only LoRA on three tasks. Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. These findings support allocating work from measured student behavior rather than uniformly removing decisions.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 11 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.
Teacher-dependent procedure with honest experiments; core claim holds only when a very strong teacher is available. This line is wrong
1 major objection · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 76 references say.
The paper says, at §5.1, §B.1, Table 21
The review did not quote this passage.
Pith says: Objection 1 of 1 · major
The central claim that HC improves over bare students is supported only under a single very strong teacher (GPT-5.5). The paper's own ablations (Table 21 and Appendix B.1) show that on InfoSeek, using gpt-5.4-mini or a local 30B teacher reduces the gain to near-bare levels (22–23 vs. 37.7).
more of objection 1
The paper acknowledges this limitation but does not demonstrate that HC can be reliably applied with any other capable teacher, nor does it sufficiently bound the generality of the claim. Since the method is presented as a general offline adaptation procedure, the authors should either (a) provide evidence that HC works with at least one additional strong teacher (e.g., a capable open-source model) on knowledge-intensive tasks, or (b) explicitly revise the scope and conclusions to state that HC's effectiveness depends on having a very strong teacher and may not transfer to settings where such a teacher is unavailable. Without this, the generality of the central claim remains uncertain.
5 minor objections
The paper says, at §4
Pith says: minor objection 1 of 5
The description of the validation floor ('no candidate below bare validation performance is retained') is clear, but the relationship between the probe set, validation set, and the outer selection budget (used on InfoSeek and DocVQA) could be stated more concisely. Consider a summarizing sentence in §4 or a footnote.
The paper says, at Table 2
Pith says: minor objection 2 of 5
The column 'Human recipe' is labeled as an alternative policy, not an upper bound. This is good, but the term 'Human recipe' could be misinterpreted as a human oracle. A brief note in the caption reiterating that these are manually designed policies (as done in §5.1) would help.
show 3 more
The paper says, at §5.2
Pith says: minor objection 3 of 5
The sentence 'The initial-to-selected gain measures search and selection together' could be misinterpreted as meaning that search and selection are the only factors. Consider rewording to: 'the initial-to-selected gain reflects the combined effect of feedback-guided revision and validation selection.'
The paper says, at Table 5
Pith says: minor objection 4 of 5
The column headers 'A (3)' and 'B (3)' are not defined until later in Appendix A.5. Consider adding a footnote or parenthetical explanation in the table caption.
The paper says, at §A.1
Pith says: minor objection 5 of 5
The description of the teacher reference scores (Table 12) notes that different populations prevent treating this column as a common upper bound. This is appropriate, but consider moving this caveat closer to the main results table (Table 2) where readers might compare.
What would settle it
Evaluate HC on InfoSeek and DocVQA using a teacher of comparable capability from a different model family (e.g., Claude 4 or Gemini 3) under the identical protocol (three builds, 150 probe/300 validation, GPT-5.5-level reasoning effort). If the macro-average improvement over bare drops by more than 5 points relative to the reported GPT-5.5 results, the teacher dependency is a substantial bound on the claim's generality. If the improvement is within 2 points, the concern is alleviated.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 11 Oct 2026
- Reader holds up
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 1 major objection, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The paper introduces Harness Compilation, an offline procedure that fits the decision interface between a frozen small VLM and its external harness using a large teacher. The experiments are well-controlled: three independent builds per setting, paired statistical tests, honest reporting of failures (SlideVQA regression, harmful card compression), and a low circularity burden (disjoint validation, teacher card generation without seeing questions). The decision-type analysis (Table 4) is a genuinely new contribution, showing that requesting evidence and generating open queries can be costly, while bounded choices and reading supplied text remain useful. The method improves over bare students on all 21 builds.
The load-bearing soft spot is teacher dependence. The paper tests almost entirely with GPT-5.5, and its own ablations (Table 21, Appendix B.1) show that on knowledge-intensive tasks like InfoSeek, weaker teachers (gpt-5.4-mini, local 30B) fail to regenerate the fact store and remain near the bare score. The method's central claim that 'HC improves over bare students' is therefore guaranteed only when a very strong teacher is available and performs well on the task. The paper acknowledges this limitation but does not demonstrate robustness across multiple strong teachers. This is not a minor concern—it is the condition under which the whole procedure works.
The other soft spots are minor: BLINK test pools are small; the formal evaluation reuses benchmark splits; three builds per setting only partly characterize variation. The paper states these limitations.
Who is this for? Researchers with access to a strong proprietary teacher (like GPT-5.5) who want a principled, reusable method for adapting the decision interface around a frozen small VLM for low-cost deployment. The paper deserves a serious referee—it is important enough and empirically grounded enough—but the review should push the authors to address the teacher dependence more substantially, perhaps by testing with at least one other strong teacher or by analyzing the conditions under which a weaker teacher suffices.
The strongest objection
The central claim that HC improves over bare students relies on the teacher (GPT-5.5) being able to propose effective harnesses from scratch and revise them from traces. The paper's own ablations (Table 21 and Appendix B.1) show that on InfoSeek, using gpt-5.4-mini or a local 30B teacher drops the score from 37.7 to about 22–23, barely above the bare 18.0. On DocVQA, smaller teachers perform near GPT-5.5, confirming that teacher dependence is task-specific but acute where content creation (e.g., fact cards) is needed. The paper acknowledges this limitation but does not demonstrate that HC can be reliably applied with any teacher other than GPT-5.5 on the hardest tasks. Since the teacher is a proprietary model subject to change, and the method's success on knowledge-intensive tasks rests almost entirely on its quality, the claim that 'HC improves over bare students' is only guaranteed in the regime where such a teacher is available and performs well. This is a load-bearing concern because the procedure is offered as a general offline adaptation method, yet its empirical support is almost entirely from a single powerful teacher. Without evidence of robustness across multiple strong teachers, the generality of the central claim is uncertain.
The reader identified the weakest assumption as the teacher's capability to infer useful modifications. I agree that this is the most load-bearing point. The paper partially examines it with smaller teachers and shows that on knowledge-intensive tasks (InfoSeek) the method collapses with a weaker teacher, while on other tasks it remains robust. This asymmetry is acknowledged. However, the paper does not test with a different strong teacher from another provider, leaving open the question of whether the results are specific to GPT-5.5's particular strengths. Despite this, the empirical evidence for the central claim (HC improves over bare) is clear in the tested regime, and the limitations are honestly stated. Therefore, the reader's ACCEPT verdict should remain unchanged, but the confidence in the method's generality beyond GPT-5.5 should be modest.
The referee's report
If the results hold, HC provides a practical offline method for adapting frozen small VLMs to specific tasks without weight updates, relying only on a capable teacher during compilation. The decision-type analysis offers useful guidance for designing interfaces around small models. The experiments are well-controlled, with multiple builds, paired tests, and honest reporting of regressions. The results are credible within the tested conditions.
Pith's reading of the paper (a machine's summary, not the authors')
Harness Compilation (HC) is an offline procedure that adapts the division of work between a frozen small vision-language model and its external harness. A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. The central claim is that not all decisions should be removed from the student: obtaining evidence and generating open queries can be costly, whereas bounded choices, reading supplied text, and following output contracts remain useful student work. Across seven visual question-answering settings with students at most 9B parameters, HC raises the macro-average task score from 48.3 to 62.9, improving over bare students in all 21 independent builds. A sympathetic reader would care because HC offers a practical way to improve small VLMs without weight updates or large-model calls at deployment, using only offline adaptation and local tools.
What carries the argument. Harness Compilation (HC) is the central mechanism: an offline procedure that uses a large teacher to iteratively revise a harness composed of prompts, tools, routing, memory, and executable control. The teacher receives student execution traces, edits the harness, and validation selects the deployed version. The harness itself is structured around five decision types (decide, select, generate, integrate, persist) that describe which operations the student retains. The procedure relies on a developer-specified runtime that limits router observations to inputs (question text, CLIP candidates, OCR) and restricts parsers to extract only spans of student output. This architecture allows one frozen model to serve different tasks by loading different harnesses without weight updates or teacher calls at deployment.
Core claim. The paper establishes that a harness's benefit depends not only on what it provides but also on which decisions it leaves to the model. Harness Compilation (HC) fits this division of work to the target student by having a teacher propose reusable content and control, execute candidates through the student's runtime, revise from execution reports, and validate with disjoint data. The key empirical finding is that across seven VQA settings, HC improves over bare students in every build (macro-average 48.3 to 62.9). Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration, and abstention) reveal that requesting evidence and generating open lookup arguments can be costly, whereas bounded choices and reading supplied text can remain useful student work. HC also works with 100 practice items, outperforming answer-only LoRA on three tasks, and can complement weight adaptation to improve beyond either approach alone.
The premise it rests on. The iterative improvement depends on the capability of the large teacher to infer useful harness modifications from student execution traces, and the teacher's rationale does not establish causality when multiple components change together.
If the paper is right (Pith's inferences, not the authors')
Local document assistants and image-query services can adapt reusable harnesses offline to reduce online generation, cutting student token counts by 69–189× on some tasks compared to chain-of-thought sampling with voting.
When task labels are scarce (100 items), HC offers a viable alternative to answer-only LoRA, exceeding it on all three tested tasks.
HC and weight adaptation (LoRA) are complementary: applying the same harness to a fine-tuned student improves SlideVQA and DocVQA beyond either component alone.
The decision-type taxonomy (decide, select, generate, integrate, persist) provides a structured way to diagnose and allocate work, suggesting that uniform removal of student decisions is suboptimal.
Recompilation for a new student model helps when the transferred interface no longer fits the student, as shown by mismatched pick-and-read programs on DocVQA.
The five decision types could generalise to other agent settings (e.g., tool-use agents, robotics policies) where the cost of student-generated arguments vs. bounded choices must be balanced.
HC's reliance on a large teacher for trace interpretation may be the weakest link; future work could explore smaller teachers or automated rule extraction from traces to reduce offline cost.
The finding that some decisions are costly while others are beneficial suggests that future systems should dynamically adjust the division of work per student or per task, perhaps learned from history.
Because HC selects the best harness by validation on a disjoint set, the approach could be extended to multi-objective optimisation (e.g., latency, privacy, token budget) beyond accuracy.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Small vision-language models may be able to read external evidence yet struggle to obtain it. We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. Deployment requires neither weight updates nor teacher calls. Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. Recompilation for a new student model helps when the transferred interface no longer fits the student. With 100 practice items, HC exceeds answer-only LoRA on three tasks. Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. These findings support allocating work from measured student behavior rather than uniformly removing decisions.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-09
The authors' main claim, as the classifier picked it out 1.00
“These findings support allocating work from measured student behavior rather than uniformly removing decisions.”
How the main claim is stated: asserted 0.73
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.84 · human participants 0.05 · controlled comparison 0.95 · randomized assignment 0.04 · states quantitative result 0.99 · compares with prior results 0.96 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Small vision-language models may be able to read external evidence yet struggle to obtain it. | background 0.90 | 0.09 | 0.04 | hedged possibility 1.00 |
| We introduce Harness Compilation (HC), an offline procedure that adapts the division of work between a frozen small VLM and its external harness. | contribution 0.91 | 0.90 | 0.07 | asserted 1.00 |
| A large teacher uses student execution traces to revise reusable content and control, while a separate validation set selects the deployed harness. | method 0.99 | 0.86 | 0.09 | asserted 0.99 |
| Deployment requires neither weight updates nor teacher calls. | method 0.77 | 0.67 | 0.06 | asserted 1.00 |
| Across seven visual question-answering settings with students of at most 9B parameters, HC improves scores over bare students by 9.9-23.9 points, averaged over three independent builds per setting. | result 1.00 | 0.97 | 0.17 | asserted 1.00 |
| Interventions on five runtime decision types (invocation, selection, argument generation, evidence integration and abstention) show why this allocation matters: requesting evidence and generating open queries can be costly, whereas bounded choices and reading supplied text can remain useful student work. | interpretation 0.46 | 0.94 | 0.29 | asserted 0.98 |
| Fact cards benefit all ten evaluated students, but decision policies transfer unevenly. | result 1.00 | 0.93 | 0.07 | asserted 1.00 |
| Recompilation for a new student model helps when the transferred interface no longer fits the student. | result 0.94 | 0.89 | 0.06 | asserted 0.56 |
| With 100 practice items, HC exceeds answer-only LoRA on three tasks. | result 1.00 | 0.95 | 0.06 | asserted 1.00 |
| Larger training budgets can match or surpass a fixed harness, while combining the two improves SlideVQA beyond either alone. | result 0.99 | 0.93 | 0.08 | asserted 0.91 |
| These findings support allocating work from measured student behavior rather than uniformly removing decisions. | interpretation 1.00 | 0.83 | 0.84 | asserted 0.73 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.99
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-09: 1 request, 8,069 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (11)
Figures are the authors', reproduced from the paper.
References (76)
- [1] Lee, Yoonho and Nair, Roshen and Zhang, Qizheng and Lee, Kangwook and Khattab, Omar and Finn, Chelsea , journal= on Pith
- [2] Qian, Cheng and Zhao, Wenting and Yang, Liangwei and Wang, Heng and Qiu, Jielin and Ji, Heng and Savarese, Silvio and Wang, Huan and Heinecke, Shelby , journal= on Pith
- [4] Park, Sungho and Kim, Wonjoong and Tan, Rongyuan and Zhang, Jue and Han, Wook-Shin and Gao, Pengfei and Park, Chanyoung and Yao, Yongqiang and Fu, Rao and Nallipogu, Elsie and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei , journal= on Pith
- [5] Sun, Yutao and Miao, Yanting and Ma, Hao-Xuan and Zhou, Mengyu and Chen, Mingshuai and Zhao, Tiancheng and Wang, Dexin and Lv, Lei and Xu, Li and Jiang, Xiaoxi and Jiang, Guanjun , journal= on Pith
- [6] Nguyen, Michael and Tan, Wei Chen and Hassan, Nurul Aisyah and Raman, Arvind and Lim, Li Hua and Razak, Ahmad Faiz , journal=. Where Does Harness-Optimization Value Live? on Pith
- [7] Xu, Guoyang and Chen, Hao , journal= on Pith
- [8] Lin, Minhua and Wu, Juncheng and Wang, Zijun and Shi, Zhan and Sang, Yisi and He, Bing and Liu, Zewen and Wei, Tianxin and Wu, Zongyu and Zhang, Zhiwei and Wang, Dakuo and Zhang, Xiang and Dumoulin, Benoit and Xie, Cihang and Zhou, Yuyin and Wang, Suhang and Lu, Hanqing , jour... on Pith
- [9] Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning , author=. arXiv preprint arXiv:2605.08741 , year= on PithPith's reviewarXiv
- [10] Zhao, Jinqing and Wu, Chengcan , journal=. Making Prospective Memory on Pith
- [11] Agrawal, Lakshya A. and others , booktitle=. 2026 , note= on Pith
- [12] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models , author=. International Conference on Learning Representations (ICLR) , year= on Pith
- [13] Khattab, Omar and others , journal= on Pith
- [15] Yuksekgonul, Mert and others , journal= on Pith
- [16] Large Language Models as Tool Makers , author=. International Conference on Learning Representations (ICLR) , year= on Pith
- [17] Offline Training of Language Model Agents with Functions as Learnable Weights , author=. International Conference on Machine Learning (ICML) , year= on Pith
- [18] Qiu, Jiahao and Juan, Xinzhe and Wang, Yimin and Yang, Ling and Qi, Xuan and Zhang, Tongcheng and Guo, Jiacheng and Lu, Yifu and Yao, Zixin and Wang, Hongru and Liu, Shilong and Jiang, Xun and Leqi, Liu and Wang, Mengdi , journal= on Pith
- [19] Small Models Struggle to Learn from Strong Reasoners , author=. arXiv preprint arXiv:2502.12143 , year= on PitharXiv
- [20] Shen, Weizhou and others , journal=. Small on Pith
- [23] Distilling the Knowledge in a Neural Network , author=. arXiv preprint arXiv:1503.02531 , year= on PithPith's reviewarXiv
- [24] Symbolic Knowledge Distillation: from General Language Models to Commonsense Models , author=. arXiv preprint arXiv:2110.07178 , year= on PithPith's reviewarXiv
show 56 more
- [25] Learning by Distilling Context , author=. arXiv preprint arXiv:2209.15189 , year= on PithPith's reviewarXiv
- [26] Hu, Edward J. and others , booktitle= on Pith
- [28] Yan, Yibin and Xie, Weidi , journal= on Pith
- [29] Mathew, Minesh and Karatzas, Dimosthenis and Jawahar, C. V. , booktitle= on Pith
- [30] Fu, Xingyu and others , booktitle=. 2024 , note= on Pith
- [31] Intrinsic Images in the Wild , author=. ACM Transactions on Graphics (SIGGRAPH) , year= on Pith
- [32] Min, Juhong and Lee, Jongmin and Ponce, Jean and Cho, Minsu , journal= on Pith
- [33] The Functional Correspondence Problem , author=. ICCV , year= on Pith
- [34] Oquab, Maxime and others , journal= on Pith
- [35] Lindenberger, Philipp and Sarlin, Paul-Edouard and Pollefeys, Marc , booktitle= on Pith
- [36] Yang, Lihe and others , journal= on Pith
- [38] Maharana, Adyasha and Lee, Dong-Ho and Tulyakov, Sergey and Bansal, Mohit and Barbieri, Francesco and Fang, Yuwei , booktitle=. Evaluating Very Long-Term Conversational Memory of. 2024 , note= on Pith
- [39] Jeddi, Ahmadreza and Le, Minh Ngoc and Kazerouni, Amirhossein and Karaimer, Hakki Can and Nguyen, Hue and Mohomed, Iqbal and Brudno, Michael and Levinshtein, Alex and Derpanis, Konstantinos G. and Taati, Babak and Grzeszczuk, Radek , journal= on Pith
- [40] Efficient Test-Time Scaling for Small Vision-Language Models , author=. International Conference on Learning Representations (ICLR) , year= on Pith
- [41] Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. International Conference on Learning Representations (ICLR) , year= on Pith
- [42] Qin, Yujia and Liang, Shihao and Ye, Yining and Zhu, Kunlun and Yan, Lan and Lu, Yaxi and Lin, Yankai and Cong, Xin and Tang, Xiangru and Qian, Bill and Zhao, Sihan and Hong, Lauren and Tian, Runchu and Xie, Ruobing and Zhou, Jie and Gerstein, Mark and Li, Dahai and Liu, Zhiyu... on Pith
- [43] Fei, Senyu and Wang, Siyin and Shi, Junhao and Dai, Zihao and Cai, Jikun and Qian, Pengfang and Ji, Li and He, Xinzhe and Zhang, Shiduo and Fei, Zhaoye and Fu, Jinlan and Gong, Jingjing and Qiu, Xipeng , journal=. 2025 , url= on Pith
- [44] Seeking and Updating with Live Visual Knowledge , author=. Advances in Neural Information Processing Systems , year= on Pith
- [45] Tanaka, Ryota and Nishida, Kyosuke and Nishida, Kosuke and Hasegawa, Taku and Saito, Itsumi and Saito, Kuniko , booktitle=. 2023 , doi= on Pith
- [46] Lakshya A. Agrawal et al. GEPA : Reflective prompt evolution can outperform reinforcement learning. In International Conference on Learning Representations (ICLR), 2026. arXiv:2507.19457 on PithPith's reviewarXiv
- [47] Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild. ACM Transactions on Graphics (SIGGRAPH), 2014 on Pith
- [48] Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. In International Conference on Learning Representations (ICLR), 2024. arXiv:2305.17126 on PithPith's reviewarXiv
- [49] Yang Chen et al. Can pre-trained vision and language models answer visual information-seeking questions? arXiv preprint arXiv:2302.11713, 2023 on PithPith's reviewarXiv
- [50] Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, Jinlan Fu, Jingjing Gong, and Xipeng Qiu. LIBERO-Plus : In-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626, 2025. URL... on PithPith's reviewarXiv
- [51] Mingyang Fu, Yuyang Peng, Dongping Chen, Zetong Zhou, Benlin Liu, Yao Wan, Zhou Zhao, Philip S. Yu, and Ranjay Krishna. Seeking and updating with live visual knowledge. In Advances in Neural Information Processing Systems, 2025. URL https://arxiv.org/abs/2504.05288. Datasets a... on PithPith's reviewarXiv
- [52] Xingyu Fu et al. BLINK : Multimodal large language models can see but not perceive. In European Conference on Computer Vision (ECCV), 2024. arXiv:2404.12390 on PithPith's reviewarXiv
- [53] Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. arXiv preprint arXiv:2211.11559, 2022 on PithPith's reviewarXiv
- [54] Edward J. Hu et al. LoRA : Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022 on Pith
- [55] Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni, Hakki Can Karaimer, Hue Nguyen, Iqbal Mohomed, Michael Brudno, Alex Levinshtein, Konstantinos G. Derpanis, Babak Taati, and Radek Grzeszczuk. AVIS : Adaptive test-time scaling for vision-language models. arXiv preprint arXi... on PithPith's reviewarXiv
- [56] Mehmet Onurcan Kaya, Desmond Elliott, and Dim P. Papadopoulos. Efficient test-time scaling for small vision-language models. In International Conference on Learning Representations (ICLR), 2026. arXiv:2510.03574 on PitharXiv
- [57] Omar Khattab et al. DSPy : Compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714, 2023 on PithPith's reviewarXiv
- [58] Zihang Lai, Senthil Purushwalkam, and Abhinav Gupta. The functional correspondence problem. In ICCV, 2021 on Pith
- [59] Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-Harness : End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052, 2026 on PithPith's reviewarXiv
- [60] Minhua Lin, Juncheng Wu, Zijun Wang, Zhan Shi, Yisi Sang, Bing He, Zewen Liu, Tianxin Wei, Zongyu Wu, Zhiwei Zhang, Dakuo Wang, Xiang Zhang, Benoit Dumoulin, Cihang Xie, Yuyin Zhou, Suhang Wang, and Hanqing Lu. Harness updating is not harness benefit: Disentangling evolution c... on PithPith's reviewarXiv
- [61] Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Pollefeys. LightGlue : Local feature matching at light speed. In ICCV, 2023 on Pith
- [62] Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. DocVQA : A dataset for VQA on document images. In WACV, 2021 on Pith
- [63] Juhong Min, Jongmin Lee, Jean Ponce, and Minsu Cho. SPair-71k : A large-scale benchmark for semantic correspondence. arXiv preprint arXiv:1908.10543, 2019 on PithPith's reviewarXiv
- [64] Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. arXiv preprint arXiv:2306.09683, 2023 on PithPith's reviewarXiv
- [65] Michael Nguyen, Wei Chen Tan, Nurul Aisyah Hassan, Arvind Raman, Li Hua Lim, and Ahmad Faiz Razak. Where does harness-optimization value live? L ocalized gains and the budget-splitting trap in self-evolving LLM agents. arXiv preprint arXiv:2609.02889, 2026 on PithPith's reviewarXiv
- [66] Krista Opsahl-Ong et al. Optimizing instructions and demonstrations for multi-stage language model programs. arXiv preprint arXiv:2406.11695, 2024 on PithPith's reviewarXiv
- [67] Maxime Oquab et al. DINOv2 : Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023 on PithPith's reviewarXiv
- [68] Sungho Park, Wonjoong Kim, Rongyuan Tan, Jue Zhang, Wook-Shin Han, Pengfei Gao, Chanyoung Park, Yongqiang Yao, Rao Fu, Elsie Nallipogu, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. AutoSaddler : Automatic harness optimization with durable updates from agent execution trac... on PithPith's reviewarXiv
- [69] Cheng Qian, Wenting Zhao, Liangwei Yang, Heng Wang, Jielin Qiu, Heng Ji, Silvio Savarese, Huan Wang, and Shelby Heinecke. AI4AI at test-time: Strong-to-weak capability transfer via harnesses. arXiv preprint arXiv:2608.12307, 2026 on PithPith's reviewarXiv
- [70] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. ToolLLM : Facilitating large language models to ... on Pith
- [71] Jiahao Qiu, Xinzhe Juan, Yimin Wang, Ling Yang, Xuan Qi, Tongcheng Zhang, Jiacheng Guo, Yifu Lu, Zixin Yao, Hongru Wang, Shilong Liu, Xun Jiang, Liu Leqi, and Mengdi Wang. AgentDistill : Training-free agent distillation with generalizable MCP boxes. arXiv preprint arXiv:2506.1... on PithPith's reviewarXiv
- [72] Yutao Sun, Yanting Miao, Hao-Xuan Ma, Mengyu Zhou, Mingshuai Chen, Tiancheng Zhao, Dexin Wang, Lei Lv, Li Xu, Xiaoxi Jiang, and Guanjun Jiang. Dynamo : Dynamic skill-tool evolution for vision-language agents. arXiv preprint arXiv:2606.30185, 2026 on PithPith's reviewarXiv
- [73] D \' dac Sur \'i s, Sachit Menon, and Carl Vondrick. ViperGPT : Visual inference via python execution for reasoning. arXiv preprint arXiv:2303.08128, 2023 on PithPith's reviewarXiv
- [74] Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida, Taku Hasegawa, Itsumi Saito, and Kuniko Saito. SlideVQA : A dataset for document visual question answering on multiple images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 13636--13645, 2023. d... on Pithdoi:10.1609/aaai.v37i11.26598
- [75] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023 on Pith
- [76] Guoyang Xu and Hao Chen. VideoHarness-RSI : Recursive harness self-improvement for long-video understanding with frozen vision-language models. arXiv preprint arXiv:2608.24302, 2026 on PithPith's reviewarXiv
- [77] Yibin Yan and Weidi Xie. EchoSight : Advancing visual-language models with wiki knowledge. arXiv preprint arXiv:2407.12735, 2024 on PithPith's reviewarXiv
- [78] Chenyang Yang, Xinran Zhao, Tongshuang Wu, and Christian K \"a stner. Better harnesses, smaller models: Building 90\ automated harness adaptation. arXiv preprint arXiv:2607.08938, 2026 on PithPith's reviewarXiv
- [79] Lihe Yang et al. Depth Anything V2 . arXiv preprint arXiv:2406.09414, 2024 on PithPith's reviewarXiv
- [80] Mert Yuksekgonul et al. TextGrad : Automatic ``differentiation'' via text. arXiv preprint arXiv:2406.07496, 2024 on PithPith's reviewarXiv
- [81] Qizheng Zhang et al. Agentic context engineering: Evolving contexts for self-improving language models. In International Conference on Learning Representations (ICLR), 2026. arXiv:2510.04618 on PithPith's reviewarXiv
- [82] Shaokun Zhang, Jieyu Zhang, Jiale Liu, Linxin Song, Chi Wang, Ranjay Krishna, and Qingyun Wu. Offline training of language model agents with functions as learnable weights. In International Conference on Machine Learning (ICML), 2024. arXiv:2402.11359 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment