Pith.

Artificial Intelligence · arXiv:2611.23890 v1 · submitted 18 Nov 2026 · 75 references

EVA-RSI: Recursive Self-Improvement of Video Evaluators for Judging Temporal Quality

Yu Yuan, Ruizhou Ding, Shaoheng Fang, Stanley H. Chan, Zhuo Chen

The paper says

“Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos.”

more of the abstract

Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 19 Nov 2026 · read the full text (96,782 characters) · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. EVA-RSI shows that recursive self-improvement can work for video evaluation by using a constructed-target exam and multi-agent Nash bargaining, producing a stronger evaluator that transfers to real generation tasks. This line is wrong

no objections raised

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

The paper, first character to last

The reviewer was given the whole text, 96,782 characters. Not checked: figures, code, what the 75 references say.

What would settle it

If a human study showed that the EVA-RSI evaluator's scores on the constructed-target exam do not correlate with human judgments of motion quality on real generated videos (especially for complex, interacting defects), the central claim would be undermined. The paper acknowledges this limitation and reports that human-labeled real videos are included in the exam to mitigate it, but the exam's coverage of real failure modes remains the load-bearing premise.

from Pith's summary of the paper

How the review went

5 passes by deepseek-v4-flash · 19 Nov 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic concern withdrawn: Pith's reviewer misread the paper 11 s · 34,005 tokens read, 602 written
  3. Referee no objections raised 6 s · 34,326 tokens read, 230 written
  4. Check against the paper kept 0 of 1 objection 36 s · 104,510 tokens read, 2,872 written
  5. Editor wrote the verdict above 10 s · 34,710 tokens read, 588 written

Not run on this paper: the simulated authors' defence.

Pith's passes raised 1 objection. A check against the paper's text kept 0 and withdrew 1 (1 Pith's reviewer misread the paper).

The review in full

The review in brief, machine-written

This paper asks whether recursive self-improvement (RSI) can improve a video evaluator when there is no native verifier and when multiple competing objectives must be balanced. The answer is a qualified yes, and the evidence is substantial.

The core idea is to build a fixed proxy exam with construction-defined targets—controlled edits like reversal, frame dropping, and object disappearance—that supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to perceptual judgment. LLM agents propose improvements for each dimension, a leader coordinates them, and a Nash bargaining rule selects updates that balance progress across dimensions. The loop runs for 170 candidates over 88 rounds in two stages, starting from Qwen3-VL-8B.

The resulting evaluator, EVA-RSI, improves the main mean on a hidden test bank from 0.706 to 0.906, with the largest gain on smoothness (0.461 to 0.888). Gains transfer to unseen defect types and to public benchmarks, where EVA-RSI achieves the highest observed scores among 21 tested models on dynamics subsets. As an RL reward, it increases generated motion and earns human preference at update 50. A data-selection pipeline incorporating it also increases motion. The paper includes extensive controls: a non-recursive search, a Union SFT baseline, and a recipe replay that isolate the contribution of recursion.

The main soft spot is the load-bearing assumption that the constructed-target exam is a valid proxy for real motion quality. The authors acknowledge this limitation explicitly: controlled edits simplify motion failures and differ from the distribution of real generated videos. The exam's coverage and fidelity to real failures is what the entire RSI loop depends on. The reliance on Claude Opus 5.5 for agent proposals means the exact search is not reproducible with open models. The loss of the stage-1 state file limits full reproducibility. These are real but acknowledged limitations, and the paper is transparent about them.

This paper is for researchers working on video evaluation, recursive self-improvement, and multi-objective optimization. It deserves a serious referee. The experimental design is thorough, the controls are informative, and the results are clearly presented. The limitations are honestly stated and point to clear directions for future work.

Recommendation: send to peer review. The paper would benefit from addressing the reproducibility concerns (e.g., testing with open-source agents, releasing all available artifacts) and further validating the exam's generalization to complex, interacting failures, but these are directions for revision, not reasons for rejection.

The stress test's concern

Withdrawn: Pith's reviewer misread the paper.

The paper says

“The main limitation of EV A-RSI is its verifier exam. Controlled edits simplify motion failures and differ from the distribution of real generated videos, where defects interact with scene content and intent. Construction-defined targets describe individual edits. Human-labeled real videos broaden exam coverage; complex, interacting failures remain a target for expansion. Future work should build richer exams from generator failures, validate targets with human judgments, and test transfer across generators.”

The referee's report

If the results hold, this work demonstrates a practical approach to recursive self-improvement for video evaluation, a domain lacking native verifiers. The multi-agent coordination via Nash bargaining and the constructed-target exam are novel contributions. The extensive evaluation with controls, hidden test bank, and public benchmarks strengthens the claims. The paper is significant for advancing video evaluation and generative model training.

Pith's reading of the paper (a machine's summary, not the authors')

The paper is trying to establish that recursive self-improvement (RSI) can work for video evaluation, a domain that lacks a native verifier and has multiple competing objectives. The authors introduce EVA-RSI, a multi-agent framework that uses a constructed-target exam (with controlled edits like reversal and frame dropping) to provide supervision, and a Nash bargaining rule to balance progress across dimensions like physics, smoothness, and dynamics. Starting from Qwen3-VL-8B, the loop improves motion evaluation, with the largest gain on smoothness (from 0.461 to 0.888), and the gains transfer to unseen defect types and public benchmarks. If true, this would show that RSI can extend beyond math and coding to perceptual tasks where ground-truth verifiers are unavailable, and that the resulting evaluator can improve video generation when used as an RL reward or data filter.

What carries the argument. The central mechanism is a multi-agent RSI loop with three key components: (1) a constructed-target exam that pairs clean clips with clips containing controlled temporal defects (reversal, frame dropping, freezing, etc.) and assigns the clean clip as the pairwise target, providing supervision without human annotation; (2) dimension-specific LLM proposer agents that study the evaluator's errors and propose training recipes, coordinated by a leader agent; (3) a Nash bargaining acceptance rule that accepts a candidate only if all main dimensions improve (or hold once saturated) and no dimension falls below its floor, preventing gains on one dimension from causing losses on another. The exam is fixed within each stage, making candidate scores comparable, and humans redesign the exam between stages to address saturation points.

Core claim. The paper claims that EVA-RSI, a multi-agent recursive self-improvement framework, can improve a video evaluator's temporal quality judgments without a native verifier and across multiple competing dimensions. The system builds a fixed proxy exam with construction-defined targets (controlled edits like reversal, frame dropping, object disappearance) that supply supervision without human annotation, plus a small set of human-labeled real videos to anchor the exam to human judgment. Each dimension has an LLM proposer agent that studies errors and proposes fixes; a leader coordinates proposals; a Nash bargaining rule selects updates that balance progress across dimensions. Starting from Qwen3-VL-8B, the loop produces eight champions over 88 rounds, raising the main mean on a hidden test bank from 0.706 to 0.906, with the largest gain on smoothness (0.461 to 0.888). The gains carry over to defect types never seen during the run, and on the dynamics subsets of public benchmarks the evaluator achieves the highest observed scores among 21 tested models. As an RL reward for Wan2.1, it increases generated motion and earns human preference; a data-selection pipeline incorporating it also increases motion.

The premise it rests on. The constructed-target exam with controlled edits (reversal, frame dropping, etc.) provides a valid proxy for real motion quality that generalizes to real generated videos and human perceptual judgments.

If the paper is right (Pith's inferences, not the authors')

If the EVA-RSI approach is correct, then recursive self-improvement can be applied to perceptual evaluation tasks that lack a native verifier, extending RSI beyond math and coding domains.

The resulting evaluator can serve as an effective RL reward for video generators, increasing motion and earning human preference, which could improve the quality of generated videos.

The evaluator can also be used as a data filter in training pipelines, selecting clips that lead to more motion in the generated outputs.

The Nash bargaining rule provides a principled way to coordinate improvement across multiple competing objectives, which could be applied to other multi-dimensional evaluation tasks.

The gains transfer to unseen defect types and public benchmarks, suggesting that the learned motion judgment generalizes beyond the synthetic exam.

The paper's reliance on a constructed-target exam with controlled edits as a proxy for real motion quality is the load-bearing assumption. If this exam does not capture the distribution of real generated video defects (where defects interact with scene content and intent), the RSI loop may optimize for the wrong signal. The authors acknowledge this limitation explicitly.

The approach requires significant human effort in designing the exam, reviewing candidate changes, and revising the curriculum between stages. Scaling this to new domains or larger models may require automating more of the curriculum design.

The paper shows that the evaluator improves motion as an RL reward, but also notes that adherence (prompt following) can degrade with longer training. This trade-off between motion and adherence may need to be addressed for practical deployment.

The comparison against proprietary models on public benchmarks shows EVA-RSI leading on dynamics subsets but trailing on overall mean rank score. This suggests that the gains are concentrated on motion-related metrics, and other dimensions (like text alignment) may not benefit as much.

Check this review

What was read

The passes were given all 96,782 characters of the text of the paper's PDF.

Fingerprint of that text (SHA-256): cd23fa0c810efb42ba65824311b8eb66a5f1e85de3db6ad756416233ee80763b

Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:

PYTHONPATH=. python scripts/review_input_check.py 2611.23890

The passes, in the order they ran

  1. Reader deepseek-v4-flash · medium effort · $0.011 · instructions of the Reader, fingerprint f9ead7300382…
  2. Skeptic deepseek-v4-flash · medium effort · $0.0032 · instructions of the Skeptic, fingerprint 8f88c0f61488…
  3. Referee deepseek-v4-flash · medium effort · $0.0032 · instructions of the Referee, fingerprint a852d99aea0d…
  4. Check against the paper deepseek-v4-flash · medium effort · $0.0050 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
  5. Editor deepseek-v4-flash · medium effort · $0.0033 · instructions of the Editor, fingerprint e7211f6a3d27…
  6. Circularity check deepseek-v4-flash · medium effort · $0.0037 · instructions of the Circularity check, fingerprint e9946e39d67b…
  7. Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0035 · instructions of the Summary for the page, fingerprint e4f56bf568e7…

The passes cost $0.032 in model calls together.

Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00

Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.

curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum

prints the fingerprint of the reader's instructions. The same works for every pass.

The receipt

The review's fingerprint was taken on 19 Nov 2026. The review you are reading still matches it.

Fingerprint of the review (SHA-256): 38579597157974828614a9fc10b6677c636e7643599c6246b9ac3cf8a6e8c152

curl -s 'https://pith.science/api/v1/verdicts/2611.23890?form=canonical' | sha256sum

prints the fingerprint of the review as it stands now. The rule, in four steps.

A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, from the text of the paper's PDF.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-19

The authors' main claim, as the classifier picked it out 0.83

“Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.35 · human participants 0.61 · controlled comparison 0.63 · randomized assignment 0.02 · states quantitative result 0.84 · compares with prior results 0.94 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. background 0.78 0.09 0.03 asserted 1.00
Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. background 1.00 0.04 0.02 asserted 1.00
Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. gap 0.52 0.10 0.03 asserted 1.00
We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. contribution 1.00 0.84 0.05 asserted 0.98
We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. method 0.99 0.94 0.03 asserted 1.00
These edits supply supervision without human annotation. method 0.99 0.78 0.02 asserted 1.00
A small set of human-labeled real videos anchors the exam to human judgment. method 0.99 0.83 0.02 asserted 1.00
Each dimension has an LLM agent that proposes improvements. method 0.99 0.81 0.02 asserted 1.00
A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. method 0.99 0.89 0.05 asserted 1.00
Humans design the exam, review candidate changes and revise the curriculum between stages. method 0.98 0.83 0.02 asserted 0.99
Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. result 1.00 0.96 0.55 asserted 1.00
On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. result 1.00 0.95 0.21 asserted 1.00
As an RL reward, it increases generated motion and earns human preference. result 0.98 0.88 0.13 asserted 1.00
A data-selection pipeline incorporating it also increases motion. result 0.94 0.88 0.07 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-19: 1 request, 9,893 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (5)
Figure 1 of the paper
Figure 1. A reward that misjudges motion trains generators that do not move. Top: physics ratings for a clip and for a copy in which the person freezes. Bottom: Wan2.1 after 50 RL updates with VideoReward (visual and motion quality, no penalties) and with EVA-RSI as the reward. Abstract Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of gener￾ated videos. Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses…
Figure 2 of the paper
Figure 2. The EVA-RSI system. ① A team of LLM agents proposes, implements and trains candidate evaluators from the current champion. ② A frozen verifier scores each candidate on an exam with fixed targets, and a Nash bargaining rule accepts or rejects it. An accepted candidate becomes the champion of the next round; every outcome enters a shared memory that the proposers read. Humans design the verifier and redesign the exam between stages. stage, making candidate scores comparable. Second, agents control the evaluator’s data, recipe, architecture and infer￾ence; the verifier remains frozen.…
Figure 3 of the paper
Figure 3. RSI improves the evaluator over rounds. (a) Normalized score of each champion on the hidden test bank. (b) Scored candidates (grey) and champions (blue) on the development banks, which change at the start of stage 2. (c) Mean rank score on 12 public benchmarks, against all evaluators except VideoReward (Appendix D). The development banks determine the final champion. Smoothness · camera shake fixed by RSI A✗ B✓ P(B) rating A / B Qwen3-VL-8B 0.39 4.82 / 4.37 EVA-RSI 1.00 2.50 / 3.98 Physics · objectsize jump fixed by RSI A✗ B✓ P(B) rating A / B Qwen3-VL-8B 0.09 2.66 / 2.80 EVA-RSI…
Figure 4 of the paper
Figure 4. Test-bank pairs before and after RSI. B is the correct answer in each pair. P is the probability that each model gives to B. Left: close-up of four consecutive frames; dashed lines mark a fixed position. Middle: objects suddenly change size. Right: people and boxes vanish and reappear. Dimension Qwen3-VL-8B After stage 1 EVA-RSI Gain Physics 0.739 0.881 0.858 +0.119 Consistency 0.877 0.977 0.997 +0.120 Smoothness 0.461 0.841 0.888 +0.427 Dynamics 0.769 0.904 0.923 +0.154 Real videos 0.684 0.755 0.863 +0.179 Semantics 0.988 0.992 0.992 +0.004 Quality 0.968 0.993 0.996 +0.028 Human…
Figure 5 of the paper
Figure 5. Same prompt and seed; only the reward changes (update 50). EVA-RSI keeps the scene and gives the most motion. VideoRe￾ward: visual and motion quality, without penalties. Corner: optical flow. More motion. All 32 motion tests pass after Holm cor￾rection: for both seeds, updates 50 and 100 and both prompt sets, EVA-RSI gives more motion than the start model and than Qwen3-VL-8B (Tab. 6, Appendix E.1). Object motion rises in every comparison after camera motion is removed. The gain tracks the loop: the stage-1 champion increases motion over Qwen3-VL-8B at update 50; EVA-RSI sustains a…

Figures are the authors', reproduced from the paper.

References (75)
  1. [1] Piyush Bagad, Makarand Tapaswi, and Cees G.M. Snoek. Test of Time: Instilling Video-Language Models with a Sense of Time. InCVPR, pages 2503–2516, 2023. 2 on Pith
  2. [2] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun... on PithPith's reviewarXiv
  3. [3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, H... on PithPith's reviewarXiv
  4. [4] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. VideoPhy: Evaluating Phys- ical Commonsense for Video Generation. InICLR, pages 102075–102121, 2025. 2 on Pith
  5. [5] Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Golden- berg, Aditya Grover, and Kai-Wei Chang. VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evalua- tion in Video Generation. InICLR, pages 118456–118470, on Pith
  6. [6] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training Diffusion Models with Reinforce- ment Learning. InICLR, pages 4965–4987, 2024. 1 on Pith
  7. [7] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets.arXiv prepri... on PithPith's reviewarXiv
  8. [8] Mu Cai, Reuben Tan, Jianrui Zhang, Bocheng Zou, Kai Zhang, Feng Yao, Fangrui Zhu, Jing Gu, Yiwu Zhong, Yuzhang Shang, Yao Dou, Jaden Park, Jianfeng Gao, Yong Jae Lee, and Jianwei Yang. TemporalBench: Bench- marking Fine-grained Temporal Understanding for Multi- modal Video Mod... on PithPith's reviewarXiv
  9. [9] Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. InICLR, pages 50466– ... on Pith
  10. [10] Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M Asano. Lost in Time: A New Temporal Benchmark for VideoLLMs. InBMVC, 2025. 2 on Pith
  11. [11] Yaxin Du, Xiyuan Yang, Zhifan Zhou, Yujie Ge, Cheng Wang, Jiajun Wang, Sijie Chen, Zehui Liu, Yuxin Zhang, Weicheng Gu, Julian Zhang, Zixing Lei, and Siheng Chen. RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement.arXiv preprint arXiv:2609.35561, on PithPith's reviewarXiv
  12. [12] Yaxin Du, Xiyuan Yang, Zhifan Zhou, Wanxu Liu, Zixing Lei, Zimeng Chen, Fenyi Liu, Haotian Wu, Yuzhu Cai, Zexi Liu, Xinyu Zhu, WenHao Wang, Linfeng Zhang, Chen Qian, and Siheng Chen. DataMaster: Data-Centric Autonomous AI Research.arXiv preprint arXiv:2605.10906, 2026. 2 on PithPith's reviewarXiv
  13. [13] Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. InProceedings of the 13th Scandina- vian Conference on Image Analysis, pages 363–370, 2003. 7 on Pith
  14. [14] Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shao- hui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, En- hong Chen, Caifeng Shan, Ran He, and Xing Sun. Video- MME: The Fir... on Pith
  15. [15] Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Kse- nia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolf- gang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. Reinforced Self-Training (ReST) for Language ... on PithPith's reviewarXiv
  16. [16] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda ... on Pith
  17. [17] Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bo- han Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Bill Yuchen Lin, and Wenhu Chen. VideoScore: Building Au... on Pith
  18. [18] Xuan He, Dongfu Jiang, Ping Nie, Minghao Liu, Zhengxuan Jiang, Mingyi Su, Wentao Ma, Junru Lin, Chun Ye, Yi Lu, Keming Wu, Benjamin Schneider, Quy Duc Do, Zhuofeng Li, Yiming Jia, Yuxuan Zhang, Guo Cheng, Haozhe Wang, Wangchunshu Zhou, Qunshu Lin, Yuanxing Zhang, Ge Zhang, Wen... on Pith
  19. [19] Vlad Hosu, Franz Hahn, Mohsen Jenadeleh, Hanhe Lin, Hui Men, Tamas Sziranyi, Shujun Li, and Dietmar Saupe. The Konstanz natural video database (KoNViD-1k). InQoMEX, pages 1–6, 2017. 18 on Pith
  20. [20] Shengran Hu, Cong Lu, and Jeff Clune. Automated Design of Agentic Systems. InICLR, pages 21344–21377, 2025. 2 on Pith
show 55 more
  1. [21] Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large Language Mod- els Can Self-Improve. InEMNLP, pages 1051–1068, 2023. 2 on Pith
  2. [22] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive Benchmark Suite for Video Generative Models.... on Pith
  3. [23] Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, and Wenhu Chen. GenAI Arena: An Open Evaluation Platform for Generative Models. InNeurIPS, pages 79889–79908, 2024. 18 on Pith
  4. [24] Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, and Ning Liu. Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment. InACM Multimedia, pages 7793–7802, on Pith
  5. [25] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Christo- pher Wilhelm, L... on Pith
  6. [26] Robert Lange, Yuki Imajuku, and Edoardo Cetin. ShinkaE- volve: Towards Open-Ended and Sample-Efficient Program Evolution. InICLR, pages 74026–74078, 2026. 2 on Pith
  7. [27] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, and Chunyuan Li. LLaV A-OneVision: Easy Visual Task Transfer.Transactions on Machine Learning Research, on Pith
  8. [28] Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. MVBench: A Comprehensive Multi- modal Video Understanding Benchmark. InCVPR, pages 22195–22206, 2024. 2, 3, 18 on Pith
  9. [29] Hanhe Lin, Vlad Hosu, and Dietmar Saupe. KADID-10k: A Large-scale Artificially Distorted IQA Database. InQoMEX, pages 1–3, 2019. 2 on Pith
  10. [30] Xinran Ling, Chen Zhu, Meiqi Wu, Hangyu Li, Xiaokun Feng, Cundian Yang, Aiming Hao, Jiashu Zhu, Jiahong Wu, and Xiangxiang Chu. VMBench: A Benchmark for Perception-Aligned Video Motion Generation. InICCV, pages 13087–13098, 2025. 2 on Pith
  11. [31] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di ZHANG, and Wanli Ouyang. Flow-GRPO: Training Flow Matching Models via Online RL. InNeurIPS, pages 45650–45685, 2025. 1 on Pith
  12. [32] Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, 10 Xintao Wang, Xiaohong Liu, Fei Yang, Pengfei Wan, Di ZHANG, Kun Gai, Yujiu Yang, and Wanli Ouyang. Improv- ing Video Generation with Human Feedback. InNeurIPS, pag... on Pith
  13. [33] Xialei Liu, Joost Van De Weijer, and Andrew D. Bagdanov. RankIQA: Learning from Rankings for No-Reference Image Quality Assessment. InICCV, pages 1040–1049, 2017. 2 on Pith
  14. [34] Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. EvalCrafter: Benchmarking and Eval- uating Large Video Generation Models. InCVPR, pages 22139–22149, 2024. 2 on Pith
  15. [35] Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Temp- Compass: Do Video LLMs Really Understand Videos? In Findings of ACL, pages 8731–8772, 2024. 2, 3 on Pith
  16. [36] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.arXiv preprint arXiv:2408.06292, 2024. 2 on PithPith's reviewarXiv
  17. [37] Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. HPSv3: Towards Wide-Spectrum Human Preference Score. InICCV, pages 15086–15095, 2025. 21 on Pith
  18. [38] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hal- linan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine: Iterat... on Pith
  19. [39] Fanqing Meng, Jiaqi Liao, Xinyu Tan, Quanfeng Lu, Wenqi Shao, Kaipeng Zhang, Yu Cheng, Dianqi Li, and Ping Luo. Towards World Simulator: Crafting Physical Commonsense- Based Benchmark for Video Generation. InICML, pages 43781–43806, 2025. 2 on Pith
  20. [40] Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do Generative Video Models Under- stand Physical Principles? InWACV, pages 948–958, 2026. 2 on Pith
  21. [41] Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. OpenVid-1M: A Large-Scale High-Quality Dataset for Text- to-video Generation. InICLR, pages 1045–1064, 2025. 1 on Pith
  22. [42] John F. Nash, Jr. The Bargaining Problem.Econometrica, 18(2):155–162, 1950. 2 on Pith
  23. [43] Alexander Novikov, Ngân V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shi- robokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaud- huri, George Holland, Alex Davies, Sebastian Nowozin, ... on PithPith's reviewarXiv
  24. [44] Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, L... on PithPith's reviewarXiv
  25. [45] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emi- lien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Peng- ming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with ... on Pith
  26. [46] Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2V-CompBench: A Compre- hensive Benchmark for Compositional Text-to-video Gener- ation. InCVPR, pages 8406–8416, 2025. 2 on Pith
  27. [47] Zachary Teed and Jia Deng. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. InECCV, pages 402–419, on Pith
  28. [48] Haibo Tong, Zhaoyang Wang, Zhaorun Chen, Haonian Ji, Shi Qiu, Siwei Han, Kexin Geng, Zhongkai Xue, Yiyang Zhou, Peng Xia, Mingyu Ding, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. MJ-Video: Benchmarking and Re- warding Video Generation with Fine-Grained Video Prefer- ence. I... on Pith
  29. [49] Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang, Pengfei Zhou, Wangbo Zhao, Fangyuan Kong, Pengfei Wan, Kun Gai, and Yang You. DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models.arXiv preprint arXiv:2610.12572, 2026. 7 on PithPith's reviewarXiv
  30. [50] Wan Team, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P... on PithPith's reviewarXiv
  31. [51] Haoru Wang, Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning Human Motion Gen- eration with Human Perceptions. InICLR, pages 77626– 77656, 2025. 2 on Pith
  32. [52] Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi-Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, and Xian Li. Self- Taught Evaluators.arXiv preprint arXiv:2408.02666, 2024. 2 on PithPith's reviewarXiv
  33. [53] Wenhao Wang and Yi Yang. VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models. InNeurIPS, pages 65618–65642, 2024. 7 on Pith
  34. [54] Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Sheng- long Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wen- hao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tia... on PithPith's reviewarXiv
  35. [55] Yibin Wang, Zhiyu Tan, Junyan Wang, Xiaomeng Yang, Cheng Jin, and Hao Li. LiFT: Leveraging Human Feed- back for Text-to-Video Model Alignment.arXiv preprint arXiv:2412.04814, 2024. 2 on PithPith's reviewarXiv
  36. [56] Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models. arXiv preprint arXiv:2406.16338, 2024. 18 on PithPith's reviewarXiv
  37. [57] Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified Reward Model for Multimodal Understand- ing and Generation.arXiv preprint arXiv:2503.05236, 2025. 2, 6, 19 on PithPith's reviewarXiv
  38. [58] Haoning Wu, Chaofeng Chen, Jingwen Hou, Liang Liao, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. FAST-VQA: Efficient End-to-End Video Quality Assess- ment with Fragment Sampling. InECCV, pages 538–554, on Pith
  39. [59] Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jing- wen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring Video Quality Assessment on User Gener- ated Contents from Aesthetic and Technical Perspectives. In ICCV, pages 20087–20097, 2023. 2 on Pith
  40. [60] Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-Instruct: Improving Low-level Visual Abil- ities for Multi-modality Foundation Models. InCVPR, page... on Pith
  41. [61] Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guang- tao Zhai, and Weisi Lin. Q-Align: Teaching LMMs for Vi- sual Scoring via Discrete Text-Defined Levels. InICML, pages 5... on Pith
  42. [62] Xinhao Xiang, Xiao Liu, Zizhong Li, Zhuosheng Liu, and Ji- awei Zhang. AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark. InVISAPP, pages 346–357, 2026. 18 on Pith
  43. [63] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. LLaV A- Critic: Learning to Evaluate Multimodal Models. InCVPR, pages 13618–13628, 2025. 2 on Pith
  44. [64] Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jia- jun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, Jiayan Teng, Zhuoyi Yang, Wendi Zheng, Xiao Liu, Dan Zhang, Ming Ding, Xiaohan Zhang, Shiyu Huang, Xiaotao Gu, Minlie Huang, Jie Tang, and Yuxiao Dong. Vi- ... on Pith
  45. [65] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large Lan- guage Models as Optimizers. InICLR, pages 12028–12068, on Pith
  46. [66] Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting Beyond Scores: Advancing Image Quality Assessment Through Multi-modal Language Models. InECCV, pages 259–276, 2024. 2 on Pith
  47. [67] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-Rewarding Language Models. InICML, pages 57905– 57923, 2024. 2 on Pith
  48. [68] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. STaR: Bootstrapping Reasoning With Reasoning. In NeurIPS, pages 15476–15488, 2022. 2 on Pith
  49. [69] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam T. Kalai. Self-Taught Optimizer (STOP): Recursively Self- Improving Code Generation. InCOLM, 2024. 2 on Pith
  50. [70] Jianrui Zhang, Mu Cai, and Yong Jae Lee. Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos.arXiv preprint arXiv:2410.02763, 2024. 2 on PithPith's reviewarXiv
  51. [71] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. InICLR, 2026. 2 on Pith
  52. [72] Jingdong Zhang, Jionghao Wang, Zhengming Yu, Yancheng Cai, Wei Chow, Chris Choy, Zhiyang Dou, Hang Hua, Ji- achen Li, Zhixuan Liu, Chen Wang, Songlin Wei, Siyuan Yang, Yu Yuan, Xiaohang Zhan, Yucheng Zhang, Shuhong Zheng, Xin Li, Lingjie Liu, Wojciech Matusik, Zhengzhong 12 Tu... on PithPith's reviewarXiv
  53. [73] Zicheng Zhang, Ziheng Jia, Haoning Wu, Chunyi Li, Zijian Chen, Yingjie Zhou, Wei Sun, Xiaohong Liu, Xiongkuo Min, Weisi Lin, and Guangtao Zhai. Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs. InCVPR, pages 3229–3239, 2025. 18 on Pith
  54. [74] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. InNeurIPS, pages 46595–46623, 2023. 2 on Pith
  55. [75] Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, and Yunsong Guo. Dream-RSI: Recursive Self-Improvement through Evolving Worlds... on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment