Artificial Intelligence · arXiv:2611.23890 v1 · submitted 18 Nov 2026 · 75 references
EVA-RSI: Recursive Self-Improvement of Video Evaluators for Judging Temporal Quality
The paper says
“Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos.”
more of the abstract
Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 19 Nov 2026 · read the full text (96,782 characters) · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. EVA-RSI shows that recursive self-improvement can work for video evaluation by using a constructed-target exam and multi-agent Nash bargaining, producing a stronger evaluator that transfers to real generation tasks. This line is wrong
no objections raised
How often objections from reviews like this one are right: not yet measured
The paper, first character to last
The reviewer was given the whole text, 96,782 characters. Not checked: figures, code, what the 75 references say.
What would settle it
If a human study showed that the EVA-RSI evaluator's scores on the constructed-target exam do not correlate with human judgments of motion quality on real generated videos (especially for complex, interacting defects), the central claim would be undermined. The paper acknowledges this limitation and reports that human-labeled real videos are included in the exam to mitigate it, but the exam's coverage of real failure modes remains the load-bearing premise.
from Pith's summary of the paper
How the review went
5 passes by deepseek-v4-flash · 19 Nov 2026
- Reader holds up, with conditions
- Skeptic concern withdrawn: Pith's reviewer misread the paper
- Referee no objections raised
- Check against the paper kept 0 of 1 objection
- Editor wrote the verdict above
Not run on this paper: the simulated authors' defence.
Pith's passes raised 1 objection. A check against the paper's text kept 0 and withdrew 1 (1 Pith's reviewer misread the paper).
The review in full
The review in brief, machine-written
This paper asks whether recursive self-improvement (RSI) can improve a video evaluator when there is no native verifier and when multiple competing objectives must be balanced. The answer is a qualified yes, and the evidence is substantial.
The core idea is to build a fixed proxy exam with construction-defined targets—controlled edits like reversal, frame dropping, and object disappearance—that supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to perceptual judgment. LLM agents propose improvements for each dimension, a leader coordinates them, and a Nash bargaining rule selects updates that balance progress across dimensions. The loop runs for 170 candidates over 88 rounds in two stages, starting from Qwen3-VL-8B.
The resulting evaluator, EVA-RSI, improves the main mean on a hidden test bank from 0.706 to 0.906, with the largest gain on smoothness (0.461 to 0.888). Gains transfer to unseen defect types and to public benchmarks, where EVA-RSI achieves the highest observed scores among 21 tested models on dynamics subsets. As an RL reward, it increases generated motion and earns human preference at update 50. A data-selection pipeline incorporating it also increases motion. The paper includes extensive controls: a non-recursive search, a Union SFT baseline, and a recipe replay that isolate the contribution of recursion.
The main soft spot is the load-bearing assumption that the constructed-target exam is a valid proxy for real motion quality. The authors acknowledge this limitation explicitly: controlled edits simplify motion failures and differ from the distribution of real generated videos. The exam's coverage and fidelity to real failures is what the entire RSI loop depends on. The reliance on Claude Opus 5.5 for agent proposals means the exact search is not reproducible with open models. The loss of the stage-1 state file limits full reproducibility. These are real but acknowledged limitations, and the paper is transparent about them.
This paper is for researchers working on video evaluation, recursive self-improvement, and multi-objective optimization. It deserves a serious referee. The experimental design is thorough, the controls are informative, and the results are clearly presented. The limitations are honestly stated and point to clear directions for future work.
Recommendation: send to peer review. The paper would benefit from addressing the reproducibility concerns (e.g., testing with open-source agents, releasing all available artifacts) and further validating the exam's generalization to complex, interacting failures, but these are directions for revision, not reasons for rejection.
The stress test's concern
Withdrawn: Pith's reviewer misread the paper.
The paper says
“The main limitation of EV A-RSI is its verifier exam. Controlled edits simplify motion failures and differ from the distribution of real generated videos, where defects interact with scene content and intent. Construction-defined targets describe individual edits. Human-labeled real videos broaden exam coverage; complex, interacting failures remain a target for expansion. Future work should build richer exams from generator failures, validate targets with human judgments, and test transfer across generators.”
The referee's report
If the results hold, this work demonstrates a practical approach to recursive self-improvement for video evaluation, a domain lacking native verifiers. The multi-agent coordination via Nash bargaining and the constructed-target exam are novel contributions. The extensive evaluation with controls, hidden test bank, and public benchmarks strengthens the claims. The paper is significant for advancing video evaluation and generative model training.
Pith's reading of the paper (a machine's summary, not the authors')
The paper is trying to establish that recursive self-improvement (RSI) can work for video evaluation, a domain that lacks a native verifier and has multiple competing objectives. The authors introduce EVA-RSI, a multi-agent framework that uses a constructed-target exam (with controlled edits like reversal and frame dropping) to provide supervision, and a Nash bargaining rule to balance progress across dimensions like physics, smoothness, and dynamics. Starting from Qwen3-VL-8B, the loop improves motion evaluation, with the largest gain on smoothness (from 0.461 to 0.888), and the gains transfer to unseen defect types and public benchmarks. If true, this would show that RSI can extend beyond math and coding to perceptual tasks where ground-truth verifiers are unavailable, and that the resulting evaluator can improve video generation when used as an RL reward or data filter.
What carries the argument. The central mechanism is a multi-agent RSI loop with three key components: (1) a constructed-target exam that pairs clean clips with clips containing controlled temporal defects (reversal, frame dropping, freezing, etc.) and assigns the clean clip as the pairwise target, providing supervision without human annotation; (2) dimension-specific LLM proposer agents that study the evaluator's errors and propose training recipes, coordinated by a leader agent; (3) a Nash bargaining acceptance rule that accepts a candidate only if all main dimensions improve (or hold once saturated) and no dimension falls below its floor, preventing gains on one dimension from causing losses on another. The exam is fixed within each stage, making candidate scores comparable, and humans redesign the exam between stages to address saturation points.
Core claim. The paper claims that EVA-RSI, a multi-agent recursive self-improvement framework, can improve a video evaluator's temporal quality judgments without a native verifier and across multiple competing dimensions. The system builds a fixed proxy exam with construction-defined targets (controlled edits like reversal, frame dropping, object disappearance) that supply supervision without human annotation, plus a small set of human-labeled real videos to anchor the exam to human judgment. Each dimension has an LLM proposer agent that studies errors and proposes fixes; a leader coordinates proposals; a Nash bargaining rule selects updates that balance progress across dimensions. Starting from Qwen3-VL-8B, the loop produces eight champions over 88 rounds, raising the main mean on a hidden test bank from 0.706 to 0.906, with the largest gain on smoothness (0.461 to 0.888). The gains carry over to defect types never seen during the run, and on the dynamics subsets of public benchmarks the evaluator achieves the highest observed scores among 21 tested models. As an RL reward for Wan2.1, it increases generated motion and earns human preference; a data-selection pipeline incorporating it also increases motion.
The premise it rests on. The constructed-target exam with controlled edits (reversal, frame dropping, etc.) provides a valid proxy for real motion quality that generalizes to real generated videos and human perceptual judgments.
If the paper is right (Pith's inferences, not the authors')
If the EVA-RSI approach is correct, then recursive self-improvement can be applied to perceptual evaluation tasks that lack a native verifier, extending RSI beyond math and coding domains.
The resulting evaluator can serve as an effective RL reward for video generators, increasing motion and earning human preference, which could improve the quality of generated videos.
The evaluator can also be used as a data filter in training pipelines, selecting clips that lead to more motion in the generated outputs.
The Nash bargaining rule provides a principled way to coordinate improvement across multiple competing objectives, which could be applied to other multi-dimensional evaluation tasks.
The gains transfer to unseen defect types and public benchmarks, suggesting that the learned motion judgment generalizes beyond the synthetic exam.
The paper's reliance on a constructed-target exam with controlled edits as a proxy for real motion quality is the load-bearing assumption. If this exam does not capture the distribution of real generated video defects (where defects interact with scene content and intent), the RSI loop may optimize for the wrong signal. The authors acknowledge this limitation explicitly.
The approach requires significant human effort in designing the exam, reviewing candidate changes, and revising the curriculum between stages. Scaling this to new domains or larger models may require automating more of the curriculum design.
The paper shows that the evaluator improves motion as an RL reward, but also notes that adherence (prompt following) can degrade with longer training. This trade-off between motion and adherence may need to be addressed for practical deployment.
The comparison against proprietary models on public benchmarks shows EVA-RSI leading on dynamics subsets but trailing on overall mean rank score. This suggests that the gains are concentrated on motion-related metrics, and other dimensions (like text alignment) may not benefit as much.
Check this review
What was read
The passes were given all 96,782 characters of the text of the paper's PDF.
Fingerprint of that text (SHA-256): cd23fa0c810efb42ba65824311b8eb66a5f1e85de3db6ad756416233ee80763b
Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:
PYTHONPATH=. python scripts/review_input_check.py 2611.23890
The passes, in the order they ran
- Reader deepseek-v4-flash · medium effort · $0.011 · instructions of the Reader, fingerprint f9ead7300382…
- Skeptic deepseek-v4-flash · medium effort · $0.0032 · instructions of the Skeptic, fingerprint 8f88c0f61488…
- Referee deepseek-v4-flash · medium effort · $0.0032 · instructions of the Referee, fingerprint a852d99aea0d…
- Check against the paper deepseek-v4-flash · medium effort · $0.0050 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
- Editor deepseek-v4-flash · medium effort · $0.0033 · instructions of the Editor, fingerprint e7211f6a3d27…
- Circularity check deepseek-v4-flash · medium effort · $0.0037 · instructions of the Circularity check, fingerprint e9946e39d67b…
- Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0035 · instructions of the Summary for the page, fingerprint e4f56bf568e7…
The passes cost $0.032 in model calls together.
Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00
Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.
curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum
prints the fingerprint of the reader's instructions. The same works for every pass.
The receipt
The review's fingerprint was taken on 19 Nov 2026. The review you are reading still matches it.
Fingerprint of the review (SHA-256): 38579597157974828614a9fc10b6677c636e7643599c6246b9ac3cf8a6e8c152
curl -s 'https://pith.science/api/v1/verdicts/2611.23890?form=canonical' | sha256sum
prints the fingerprint of the review as it stands now. The rule, in four steps.
A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, from the text of the paper's PDF.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-19
The authors' main claim, as the classifier picked it out 0.83
“Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.35 · human participants 0.61 · controlled comparison 0.63 · randomized assignment 0.02 · states quantitative result 0.84 · compares with prior results 0.94 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. | background 0.78 | 0.09 | 0.03 | asserted 1.00 |
| Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. | gap 0.52 | 0.10 | 0.03 | asserted 1.00 |
| We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. | contribution 1.00 | 0.84 | 0.05 | asserted 0.98 |
| We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. | method 0.99 | 0.94 | 0.03 | asserted 1.00 |
| These edits supply supervision without human annotation. | method 0.99 | 0.78 | 0.02 | asserted 1.00 |
| A small set of human-labeled real videos anchors the exam to human judgment. | method 0.99 | 0.83 | 0.02 | asserted 1.00 |
| Each dimension has an LLM agent that proposes improvements. | method 0.99 | 0.81 | 0.02 | asserted 1.00 |
| A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. | method 0.99 | 0.89 | 0.05 | asserted 1.00 |
| Humans design the exam, review candidate changes and revise the curriculum between stages. | method 0.98 | 0.83 | 0.02 | asserted 0.99 |
| Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. | result 1.00 | 0.96 | 0.55 | asserted 1.00 |
| On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. | result 1.00 | 0.95 | 0.21 | asserted 1.00 |
| As an RL reward, it increases generated motion and earns human preference. | result 0.98 | 0.88 | 0.13 | asserted 1.00 |
| A data-selection pipeline incorporating it also increases motion. | result 0.94 | 0.88 | 0.07 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.98
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-19: 1 request, 9,893 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (5)
Figures are the authors', reproduced from the paper.
References (75)
- [1] Piyush Bagad, Makarand Tapaswi, and Cees G.M. Snoek. Test of Time: Instilling Video-Language Models with a Sense of Time. InCVPR, pages 2503–2516, 2023. 2 on Pith
- [2] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun... on PithPith's reviewarXiv
- [3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhao- hai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, H... on PithPith's reviewarXiv
- [4] Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. VideoPhy: Evaluating Phys- ical Commonsense for Video Generation. InICLR, pages 102075–102121, 2025. 2 on Pith
- [5] Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Golden- berg, Aditya Grover, and Kai-Wei Chang. VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evalua- tion in Video Generation. InICLR, pages 118456–118470, on Pith
- [6] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training Diffusion Models with Reinforce- ment Learning. InICLR, pages 4965–4987, 2024. 1 on Pith
- [7] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, Varun Jampani, and Robin Rombach. Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets.arXiv prepri... on PithPith's reviewarXiv
- [8] Mu Cai, Reuben Tan, Jianrui Zhang, Bocheng Zou, Kai Zhang, Feng Yao, Fangrui Zhu, Jing Gu, Yiwu Zhong, Yuzhang Shang, Yao Dou, Jaden Park, Jianfeng Gao, Yong Jae Lee, and Jianwei Yang. TemporalBench: Bench- marking Fine-grained Temporal Understanding for Multi- modal Video Mod... on PithPith's reviewarXiv
- [9] Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. InICLR, pages 50466– ... on Pith
- [10] Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M Asano. Lost in Time: A New Temporal Benchmark for VideoLLMs. InBMVC, 2025. 2 on Pith
- [11] Yaxin Du, Xiyuan Yang, Zhifan Zhou, Yujie Ge, Cheng Wang, Jiajun Wang, Sijie Chen, Zehui Liu, Yuxin Zhang, Weicheng Gu, Julian Zhang, Zixing Lei, and Siheng Chen. RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement.arXiv preprint arXiv:2609.35561, on PithPith's reviewarXiv
- [12] Yaxin Du, Xiyuan Yang, Zhifan Zhou, Wanxu Liu, Zixing Lei, Zimeng Chen, Fenyi Liu, Haotian Wu, Yuzhu Cai, Zexi Liu, Xinyu Zhu, WenHao Wang, Linfeng Zhang, Chen Qian, and Siheng Chen. DataMaster: Data-Centric Autonomous AI Research.arXiv preprint arXiv:2605.10906, 2026. 2 on PithPith's reviewarXiv
- [13] Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. InProceedings of the 13th Scandina- vian Conference on Image Analysis, pages 363–370, 2003. 7 on Pith
- [14] Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shao- hui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, En- hong Chen, Caifeng Shan, Ran He, and Xing Sun. Video- MME: The Fir... on Pith
- [15] Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Kse- nia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolf- gang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. Reinforced Self-Training (ReST) for Language ... on PithPith's reviewarXiv
- [16] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda ... on Pith
- [17] Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, Kai Wang, Quy Duc Do, Yuansheng Ni, Bo- han Lyu, Yaswanth Narsupalli, Rongqi Fan, Zhiheng Lyu, Bill Yuchen Lin, and Wenhu Chen. VideoScore: Building Au... on Pith
- [18] Xuan He, Dongfu Jiang, Ping Nie, Minghao Liu, Zhengxuan Jiang, Mingyi Su, Wentao Ma, Junru Lin, Chun Ye, Yi Lu, Keming Wu, Benjamin Schneider, Quy Duc Do, Zhuofeng Li, Yiming Jia, Yuxuan Zhang, Guo Cheng, Haozhe Wang, Wangchunshu Zhou, Qunshu Lin, Yuanxing Zhang, Ge Zhang, Wen... on Pith
- [19] Vlad Hosu, Franz Hahn, Mohsen Jenadeleh, Hanhe Lin, Hui Men, Tamas Sziranyi, Shujun Li, and Dietmar Saupe. The Konstanz natural video database (KoNViD-1k). InQoMEX, pages 1–6, 2017. 18 on Pith
- [20] Shengran Hu, Cong Lu, and Jeff Clune. Automated Design of Agentic Systems. InICLR, pages 21344–21377, 2025. 2 on Pith
show 55 more
- [21] Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large Language Mod- els Can Self-Improve. InEMNLP, pages 1051–1068, 2023. 2 on Pith
- [22] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Com- prehensive Benchmark Suite for Video Generative Models.... on Pith
- [23] Dongfu Jiang, Max Ku, Tianle Li, Yuansheng Ni, Shizhuo Sun, Rongqi Fan, and Wenhu Chen. GenAI Arena: An Open Evaluation Platform for Generative Models. InNeurIPS, pages 79889–79908, 2024. 18 on Pith
- [24] Tengchuan Kou, Xiaohong Liu, Zicheng Zhang, Chunyi Li, Haoning Wu, Xiongkuo Min, Guangtao Zhai, and Ning Liu. Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment. InACM Multimedia, pages 7793–7802, on Pith
- [25] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Christo- pher Wilhelm, L... on Pith
- [26] Robert Lange, Yuki Imajuku, and Edoardo Cetin. ShinkaE- volve: Towards Open-Ended and Sample-Efficient Program Evolution. InICLR, pages 74026–74078, 2026. 2 on Pith
- [27] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, and Chunyuan Li. LLaV A-OneVision: Easy Visual Task Transfer.Transactions on Machine Learning Research, on Pith
- [28] Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, and Yu Qiao. MVBench: A Comprehensive Multi- modal Video Understanding Benchmark. InCVPR, pages 22195–22206, 2024. 2, 3, 18 on Pith
- [29] Hanhe Lin, Vlad Hosu, and Dietmar Saupe. KADID-10k: A Large-scale Artificially Distorted IQA Database. InQoMEX, pages 1–3, 2019. 2 on Pith
- [30] Xinran Ling, Chen Zhu, Meiqi Wu, Hangyu Li, Xiaokun Feng, Cundian Yang, Aiming Hao, Jiashu Zhu, Jiahong Wu, and Xiangxiang Chu. VMBench: A Benchmark for Perception-Aligned Video Motion Generation. InICCV, pages 13087–13098, 2025. 2 on Pith
- [31] Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di ZHANG, and Wanli Ouyang. Flow-GRPO: Training Flow Matching Models via Online RL. InNeurIPS, pages 45650–45685, 2025. 1 on Pith
- [32] Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, 10 Xintao Wang, Xiaohong Liu, Fei Yang, Pengfei Wan, Di ZHANG, Kun Gai, Yujiu Yang, and Wanli Ouyang. Improv- ing Video Generation with Human Feedback. InNeurIPS, pag... on Pith
- [33] Xialei Liu, Joost Van De Weijer, and Andrew D. Bagdanov. RankIQA: Learning from Rankings for No-Reference Image Quality Assessment. InICCV, pages 1040–1049, 2017. 2 on Pith
- [34] Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. EvalCrafter: Benchmarking and Eval- uating Large Video Generation Models. InCVPR, pages 22139–22149, 2024. 2 on Pith
- [35] Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Temp- Compass: Do Video LLMs Really Understand Videos? In Findings of ACL, pages 8731–8772, 2024. 2, 3 on Pith
- [36] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.arXiv preprint arXiv:2408.06292, 2024. 2 on PithPith's reviewarXiv
- [37] Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. HPSv3: Towards Wide-Spectrum Human Preference Score. InICCV, pages 15086–15095, 2025. 21 on Pith
- [38] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hal- linan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine: Iterat... on Pith
- [39] Fanqing Meng, Jiaqi Liao, Xinyu Tan, Quanfeng Lu, Wenqi Shao, Kaipeng Zhang, Yu Cheng, Dianqi Li, and Ping Luo. Towards World Simulator: Crafting Physical Commonsense- Based Benchmark for Video Generation. InICML, pages 43781–43806, 2025. 2 on Pith
- [40] Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do Generative Video Models Under- stand Physical Principles? InWACV, pages 948–958, 2026. 2 on Pith
- [41] Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. OpenVid-1M: A Large-Scale High-Quality Dataset for Text- to-video Generation. InICLR, pages 1045–1064, 2025. 1 on Pith
- [42] John F. Nash, Jr. The Bargaining Problem.Econometrica, 18(2):155–162, 1950. 2 on Pith
- [43] Alexander Novikov, Ngân V ˜u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shi- robokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaud- huri, George Holland, Alex Davies, Sebastian Nowozin, ... on PithPith's reviewarXiv
- [44] Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, L... on PithPith's reviewarXiv
- [45] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emi- lien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Peng- ming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with ... on Pith
- [46] Kaiyue Sun, Kaiyi Huang, Xian Liu, Yue Wu, Zihan Xu, Zhenguo Li, and Xihui Liu. T2V-CompBench: A Compre- hensive Benchmark for Compositional Text-to-video Gener- ation. InCVPR, pages 8406–8416, 2025. 2 on Pith
- [47] Zachary Teed and Jia Deng. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. InECCV, pages 402–419, on Pith
- [48] Haibo Tong, Zhaoyang Wang, Zhaorun Chen, Haonian Ji, Shi Qiu, Siwei Han, Kexin Geng, Zhongkai Xue, Yiyang Zhou, Peng Xia, Mingyu Ding, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. MJ-Video: Benchmarking and Re- warding Video Generation with Fine-Grained Video Prefer- ence. I... on Pith
- [49] Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang, Pengfei Zhou, Wangbo Zhao, Fangyuan Kong, Pengfei Wan, Kun Gai, and Yang You. DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models.arXiv preprint arXiv:2610.12572, 2026. 7 on PithPith's reviewarXiv
- [50] Wan Team, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, P... on PithPith's reviewarXiv
- [51] Haoru Wang, Wentao Zhu, Luyi Miao, Yishu Xu, Feng Gao, Qi Tian, and Yizhou Wang. Aligning Human Motion Gen- eration with Human Perceptions. InICLR, pages 77626– 77656, 2025. 2 on Pith
- [52] Tianlu Wang, Ilia Kulikov, Olga Golovneva, Ping Yu, Weizhe Yuan, Jane Dwivedi-Yu, Richard Yuanzhe Pang, Maryam Fazel-Zarandi, Jason Weston, and Xian Li. Self- Taught Evaluators.arXiv preprint arXiv:2408.02666, 2024. 2 on PithPith's reviewarXiv
- [53] Wenhao Wang and Yi Yang. VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Models. InNeurIPS, pages 65618–65642, 2024. 7 on Pith
- [54] Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Sheng- long Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wen- hao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tia... on PithPith's reviewarXiv
- [55] Yibin Wang, Zhiyu Tan, Junyan Wang, Xiaomeng Yang, Cheng Jin, and Hao Li. LiFT: Leveraging Human Feed- back for Text-to-Video Model Alignment.arXiv preprint arXiv:2412.04814, 2024. 2 on PithPith's reviewarXiv
- [56] Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, and Zilong Zheng. VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models. arXiv preprint arXiv:2406.16338, 2024. 18 on PithPith's reviewarXiv
- [57] Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified Reward Model for Multimodal Understand- ing and Generation.arXiv preprint arXiv:2503.05236, 2025. 2, 6, 19 on PithPith's reviewarXiv
- [58] Haoning Wu, Chaofeng Chen, Jingwen Hou, Liang Liao, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. FAST-VQA: Efficient End-to-End Video Quality Assess- ment with Fragment Sampling. InECCV, pages 538–554, on Pith
- [59] Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jing- wen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring Video Quality Assessment on User Gener- ated Contents from Aesthetic and Technical Perspectives. In ICCV, pages 20087–20097, 2023. 2 on Pith
- [60] Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang, Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-Instruct: Improving Low-level Visual Abil- ities for Multi-modality Foundation Models. InCVPR, page... on Pith
- [61] Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guang- tao Zhai, and Weisi Lin. Q-Align: Teaching LMMs for Vi- sual Scoring via Discrete Text-Defined Levels. InICML, pages 5... on Pith
- [62] Xinhao Xiang, Xiao Liu, Zizhong Li, Zhuosheng Liu, and Ji- awei Zhang. AIGVE-Tool: AI-Generated Video Evaluation Toolkit with Multifaceted Benchmark. InVISAPP, pages 346–357, 2026. 18 on Pith
- [63] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. LLaV A- Critic: Learning to Evaluate Multimodal Models. InCVPR, pages 13618–13628, 2025. 2 on Pith
- [64] Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jia- jun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, Jiayan Teng, Zhuoyi Yang, Wendi Zheng, Xiao Liu, Dan Zhang, Ming Ding, Xiaohan Zhang, Shiyu Huang, Xiaotao Gu, Minlie Huang, Jie Tang, and Yuxiao Dong. Vi- ... on Pith
- [65] Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large Lan- guage Models as Optimizers. InICLR, pages 12028–12068, on Pith
- [66] Zhiyuan You, Zheyuan Li, Jinjin Gu, Zhenfei Yin, Tianfan Xue, and Chao Dong. Depicting Beyond Scores: Advancing Image Quality Assessment Through Multi-modal Language Models. InECCV, pages 259–276, 2024. 2 on Pith
- [67] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E Weston. Self-Rewarding Language Models. InICML, pages 57905– 57923, 2024. 2 on Pith
- [68] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. STaR: Bootstrapping Reasoning With Reasoning. In NeurIPS, pages 15476–15488, 2022. 2 on Pith
- [69] Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam T. Kalai. Self-Taught Optimizer (STOP): Recursively Self- Improving Code Generation. InCOLM, 2024. 2 on Pith
- [70] Jianrui Zhang, Mu Cai, and Yong Jae Lee. Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos.arXiv preprint arXiv:2410.02763, 2024. 2 on PithPith's reviewarXiv
- [71] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. InICLR, 2026. 2 on Pith
- [72] Jingdong Zhang, Jionghao Wang, Zhengming Yu, Yancheng Cai, Wei Chow, Chris Choy, Zhiyang Dou, Hang Hua, Ji- achen Li, Zhixuan Liu, Chen Wang, Songlin Wei, Siyuan Yang, Yu Yuan, Xiaohang Zhan, Yucheng Zhang, Shuhong Zheng, Xin Li, Lingjie Liu, Wojciech Matusik, Zhengzhong 12 Tu... on PithPith's reviewarXiv
- [73] Zicheng Zhang, Ziheng Jia, Haoning Wu, Chunyi Li, Zijian Chen, Yingjie Zhou, Wei Sun, Xiaohong Liu, Xiongkuo Min, Weisi Lin, and Guangtao Zhai. Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs. InCVPR, pages 3229–3239, 2025. 18 on Pith
- [74] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph Gonzalez, and Ion Stoica. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. InNeurIPS, pages 46595–46623, 2023. 2 on Pith
- [75] Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, and Yunsong Guo. Dream-RSI: Recursive Self-Improvement through Evolving Worlds... on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment