Pith. sign in

REVIEW 4 major objections 5 minor 49 references

VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Video summaries ranked by two information-theoretic scores improved human task accuracy by up to 61% and cut response time by up to 76% in user studies.

desk verdict The label-leakage problem is real and undercuts the headline, but the Max-G results and the released code make this worth a revision. read the letter →

arxiv 2505.17423 v4 pith:DJY2XFNW submitted 2025-05-23 cs.CV cs.HCcs.ITmath.IT

Shenghui Chen , Po-han Li , Sandeep Chinchali , Ufuk Topcu This is my paper · ORCID
classification cs.CVcs.HCcs.ITmath.IT
keywords videosummarizationTLDRinformationbottleneckpointwisemutualvision-languagemodelsannotation-freeevaluationhumandecision-makingsummaryselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VIBE is an annotation-free method for scoring and selecting video-to-text summaries without retraining a vision-language model. Each candidate summary gets two numbers: a grounding score reflecting how well it aligns with the video, and a utility score reflecting how useful it is for a given task; both are computed by approximating pointwise mutual information through masked VLM inference. In between-subjects user studies on three video datasets, summaries selected for maximum utility raised human task accuracy by up to 61.23% and reduced response time by up to 75.77% relative to naive VLM summaries or raw video, while summaries selected for grounding alone, requiring no task labels, still gave consistent accuracy gains. The claim is that VIBE is a practical, training-free selector for decision-supportive video summaries, especially for long videos where summarization reduces cognitive load.

What carries the argument

The central object is the pair of pointwise mutual information estimates computed through masked inference. The grounding score uses a masked summary whose high-tf-idf keywords are replaced by a mask token, and measures how much the video helps reconstruct it; the utility score uses a masked video with content removed and measures how much the summary helps predict the task answer. Both rely on the next-token prediction probabilities of a VLM, and the selection rule is their weighted sum, $\arg\max_T \alpha \cdot \text{grounding} + \beta \cdot \text{utility}$ over sampled summaries.

What would settle it

Redact from each candidate summary any text that states or closely paraphrases the correct answer, recompute the utility scores on the redacted summaries, and rerun the same between-subjects user study. If the redacted VIBE-selected summaries no longer outperform naive summaries on accuracy, the original gain came from answer-key leakage rather than task-relevant informativeness.

Watch

Extended reading notes

Core claim

VIBE turns the two terms of the information bottleneck objective into evaluation scores for video summaries. The grounding score, approximating $I(V;T)$, is the log ratio of the probability that a VLM reconstructs a masked summary when shown the video over the probability when it is not. The utility score, approximating $I(T;Y)$, is the log ratio of the probability that the VLM answers the task question given the summary plus a masked video over the probability given only the masked video. Given several sampled summaries, VIBE selects the one maximizing a weighted sum of the two scores, and sweeping the trade-off weight produces a Pareto frontier that naive VLM summaries fall inside. Human studies on three datasets—conference talk videos, long instructional clips, and short traffic clips—showed that the top-utility summaries improved accuracy and response time over both unfiltered VLM summaries and raw video, with the largest gains on the longest videos.

Load-bearing premise

The utility score is computed from the ground-truth answer for each video and the same answers are used to pick the summaries shown to participants, so the reported accuracy gains assume that the summaries do not simply echo the correct answer; if that assumption fails, the gains would be an artifact of answer-key leakage rather than genuine task support.

Editorial extensions

If this is right

  • Video summarization can be evaluated and curated without gold-standard human captions, removing the annotation bottleneck for new domains.
  • A self-supervised grounding score alone improves human accuracy relative to unfiltered summaries, so task labels are not strictly required for gains.
  • The selection rule is model-agnostic: any VLM with next-token probability access can be used to rank summaries.
  • The benefits concentrate on long videos; for very short clips, summarization provides little time saving, consistent with the paper's traffic-clip results.
  • Utility-ranked summaries also reduce inverse efficiency score, meaning participants gained accuracy without a compensating slowdown.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The utility score requires the ground-truth label for selection; a natural extension is to test selection using utility scores computed on a held-out validation set, so the displayed summaries are chosen without access to the answer of the questions participants actually answer.
  • Because utility correlates with human accuracy, the same score could serve as a reward signal for fine-tuning summarizers, a direction the paper notes as future work.
  • The grounding score may transfer to general video-caption quality assessment even when no downstream task exists.
  • A direct test of answer-key leakage: compare VIBE-selected summaries to summaries that explicitly contain the correct answer; if both perform similarly, the mechanism is label exposure, not summary informativeness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. VIBE proposes an annotation-free, training-free framework for selecting video-to-text summaries that support human decision-making. It defines a grounding score (I(V;T)) and a utility score (I(T;Y)) approximated with VLM next-token probabilities, ranks randomly sampled VLM summaries by a weighted combination of the two, and evaluates the selected summaries in between-subjects user studies on LearningPaper24, LongVideoBench, and SUTD-TrafficQA. The paper claims that VIBE-selected summaries improve human accuracy by up to 61.23% and reduce response time by up to 75.77% compared with naive VLM summaries or raw video, with additional correlation and VLM-ablation analyses.

Significance. If the headline results held, VIBE would be a practical, low-cost way to select task-relevant video summaries without gold-standard captions or model retraining. The paper has notable strengths: a 243-participant human study across three diverse datasets, a new LearningPaper24 dataset, consistent findings across multiple VLM backbones in the ablation study, and a transparent limitations section. However, the central quantitative claim is compromised by an answer-key leakage in the Max-U evaluation protocol, and some advertised numbers are not reproducible from the reported tables. The useful core idea, especially the label-free Max-G variant, may be salvageable, but the current validation does not support the strongest claims.

major comments (4)
  1. [Sec. 4, Eq. (4); Sec. 5.2; Table 1; Appendix A] The Max-U condition uses the ground-truth answer label Y to rank summaries via the utility score log P(Y|T,V_masked)/P(Y|V_masked), and Section 5.2 then measures human accuracy on the same multiple-choice questions whose correct answers were used for selection. This is direct answer-key leakage: the selected summary is optimized to help a VLM predict Y from text, not necessarily to be a generally better decision-support summary. Appendix A illustrates the mechanism—the Max-U summary contains the phrase 'human-machine interactions and AI safety,' which maps almost verbatim to the correct area '(I) Human-AI Interaction and Ethics,' while the Naive summary points toward a wrong area. The reported accuracy gains over Naive VLM therefore do not establish that VIBE selects better summaries in general; they may only establish that summaries containing the correct answer help humans answer that same question. The authors should validate utility-based selection on held-out tasks or questions, or restrict the headline claim to Max-G, which does not use Y.
  2. [Abstract; Sec. 1 (Contributions); Table 1] The advertised 'accuracy by up to 61.23%' cannot be found in Table 1. The largest relative accuracy gain over Naive VLM in Table 1 is 40.0% (LongVideoBench: 65.00 vs. 46.43), and no reported comparison yields 61.23%. Moreover, on LongVideoBench the Video Only condition achieves 74.44% accuracy while Max-U achieves 65.00%, so the abstract's claim of improvement 'compared to ... raw video' is contradicted by the paper's own table. The authors must either supply the analysis that produces the 61.23% figure or correct the abstract, and the accuracy claims should be qualified as dataset-dependent rather than stated as a universal improvement.
  3. [Appendix I; Sec. 4, Eqs. (3) and (4)] The derivation of Eq. (3) in Appendix I assumes that T_masked is independent of both V and T. Since T_masked is obtained by tf-idf masking of T, it is a deterministic function of T and cannot be independent of T; the cancellation P(T,V,T_masked) = P(T,V)P(T_masked) is therefore not justified. The analogous independence assumption for V_masked in Eq. (4)—that V_masked is independent of T and Y—is not stated or proven, and V_masked is a random crop of V that may retain information about Y. The scores may still be useful heuristics, but the paper's claim that they approximate pointwise mutual information is not established by the supplied derivation. The authors should either provide a valid derivation under weaker assumptions or explicitly present these quantities as heuristic estimates.
  4. [Title; Abstract; Sec. 4; Sec. 5.2] The paper is framed as 'annotation-free' throughout, but the utility score in Eq. (4) requires ground-truth task labels Y, and the Max-U arm of the user study uses those labels for both selection and evaluation. The paper does acknowledge in Section 4 that 'calculating the utility score requires access to task labels Y,' but the title, abstract, and headline human-study results emphasize the label-dependent Max-U variant. The 'annotation-free' claim should be reserved for Max-G or clearly qualified, and the abstract should not lead with a label-dependent selection result without explicitly flagging that utility-based selection requires task labels.
minor comments (5)
  1. [Figure 3] The axis labels appear to be swapped: the x-axis is labeled with the utility expression and the y-axis with the grounding expression, which is the opposite of the definitions in Eqs. (3) and (4).
  2. [Sec. 5.3; Appendix F] The correlation analyses include CoT responses, which were generated with access to the answer options (as stated in Appendix F). Including these outputs can inflate the reported correlation between utility score and human accuracy, so the correlations should also be reported with CoT excluded.
  3. [Sec. 5.4; Figure 6] The claim that VIBE 'generalizes to various VLMs' is supported only by visual inspection of Pareto plots; the paper should report a quantitative agreement metric, such as rank correlation or score agreement, across models.
  4. [Sec. 1] The 'Critique and Open Problems' paragraph in the introduction is written as a self-evaluation and does not fit the standard structure of a research paper; the open directions should be integrated into the conclusion or removed from the introduction.
  5. [Sec. 5.2; Table 5] The CoT condition is listed in Table 5 but is not defined among the independent variables in Section 5.2; the text should state explicitly that CoT was an additional exploratory condition and why it is excluded from the main comparisons.

Circularity Check

1 steps flagged · score 6.0 of 10

Max-U accuracy gains are an in-sample effect: utility selection uses the ground-truth answer Y, and human accuracy is measured on the same Y.

  1. fitted input called prediction [Eq. (4) (Utility Score), Section 4; Eq. (5) (TL;DR Selection with VIBE); Section 5.2 (Independent Variables); Table 1]
    "IP (T;Y) = log P(Y|T, Vmasked) / P(Y|V masked) .(Utility Score) (4) ... Notably, calculating the utility score requires access to task labels Y , but not gold-standard, human-annotated labels for summary T as in previous works. ... In the remaining conditions, they view only a VLM-generated summary: a randomly selected VLM summary (Naive), the top-ranked summary from k response candidates by utility score (Max-U)."

    The utility score is a function of the correct answer Y: it measures how much the summary improves the VLM's probability of the ground-truth label. Eq. (5) selects the summary that maximizes this quantity (the Max-U arm), and the user study in Section 5.2 then measures human accuracy on the same Y. Thus the Max-U arm is not an independent evaluation of decision support: the selected summary was chosen precisely because it contains cues that let a model recover Y. The paper itself excludes CoT from scatter plots because it 'has access to answer options of the task, making direct comparisons unfair and potentially misleading' (Appendix F); Max-U has access to the correct label itself. The headline 61.23% accuracy gain is therefore an in-sample, answer-hint effect rather than a prediction.

full rationale

The Max-U evaluation is partially circular: the utility score in Eq. (4) is defined through the ground-truth task label Y, and Eq. (5) selects the summary maximizing this score for the Max-U arm. The user study then measures human accuracy on the same Y, so the headline accuracy gains are an in-sample, answer-hint effect rather than an independent prediction of decision support. The paper discloses that utility requires task labels and even excludes CoT from scatter plots because it has access to answer options, calling such comparisons unfair; Max-U has access to the correct label itself. The Max-G arm is not affected, since grounding does not use Y, and it provides independent albeit weaker evidence. No load-bearing self-citation or uniqueness import was found. Overall partial circularity: the Max-U headline is compromised, while the grounding-based claim stands.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entity. It introduces a new dataset (LearningPaper24) and a new metric family, but these are not 'invented entities' in the sense of postulating a new force, particle, or mediator. The load-bearing assumptions are the independence conditions for the PMI approximations and the validity of using task labels in selection.

free parameters (4)
  • tf-idf masking thresholds = per-dataset: min tf-idf 0.0025/0.01/0.006; discard n-grams appearing in >10%/30%/50% of responses
    Hyperparameters chosen by hand per dataset to construct T_masked and V_masked; the resulting scores are sensitive to masking choices, as the paper's Limitations section notes.
  • alpha and beta in Eq (5) = alpha in {0, 0.05, ..., 1}, beta = 1 - alpha; user study uses (0,1) and (1,0)
    Trade-off weights for grounding vs utility. They are swept rather than fitted to human data, but the user-study conditions use the extremes, which is a modeling choice.
  • frame sampling and cropping = 20 or 32 frames; random 1/16 cropping for LongVideoBench and SUTD-TrafficQA
    Preprocessing choices affect V_masked and the grounding/utility estimates; the number of frames and cropping are not derived from any principle.
  • number of candidate summaries k and sampling temperature = k = 5, 'various temperatures' (exact values not given)
    The selection pool size is a free choice; the temperature values are unspecified, making exact reproduction difficult.
assumptions (4)
  • ad hoc to paper T_masked is independent of both V and T
    Appendix I states this to derive eq (3), but T_masked is a masked version of T, so it is a function of T and cannot be independent of it. The stated condition for the approximation does not hold.
  • ad hoc to paper V_masked is independent of T and Y
    Eq (4) requires the masked video to be uninformative about the task and summary, but the paper does not justify this. Random 1/16 cropping retains most visual content, so the assumption is dubious.
  • domain assumption VLM next-token probabilities are calibrated estimators of P(T|V) and P(Y|T,V_masked)
    The scores are only as valid as the VLM's probability estimates. The paper acknowledges model-induced bias in the Limitations section.
  • domain assumption Uniform frame sampling represents video content for grounding
    Grounding is computed from 20 or 32 uniformly sampled frames, which may miss key visual evidence in long or fast-moving videos.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR." pith.science (2026). https://pith.science/paper/DJY2XFNW

@misc{pith2026250517423,
  author       = {Pith},
  title        = {Pith review of: VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJY2XFNW}},
  note         = {Machine review of arXiv:2505.17423}
}
read the original abstract

Many decision-making tasks, where both accuracy and efficiency matter, still require human supervision. For example, tasks like traffic officers reviewing hour-long dashcam footage or researchers screening conference videos can benefit from concise summaries that reduce cognitive load and save time. Yet current vision-language models (VLMs) often produce verbose, redundant outputs that hinder task performance. Existing video caption evaluation depends on costly human annotations and overlooks the summaries' utility in downstream tasks. We address these gaps with Video-to-text Information Bottleneck Evaluation (VIBE), an annotation-free method that scores VLM outputs using two metrics: grounding (how well the summary aligns with visual content) and utility (how informative it is for the task). VIBE selects from randomly sampled VLM outputs by ranking them according to the two scores to support effective human decision-making. Human studies on LearningPaper24, SUTD-TrafficQA, and LongVideoBench show that summaries selected by VIBE consistently improve performance-boosting task accuracy by up to 61.23% and reducing response time by 75.77% compared to naive VLM summaries or raw video.

Figures

Figures reproduced from arXiv: 2505.17423 by the authors.

Figure 1
Figure 1. VIBE for Video-to-Text Summary Selection. Given a video, a task, and VLM-generated summaries, VIBE ranks the summaries using the proposed grounding and utility scores, which assess video alignment and task relevance. It selects the summary most conducive to helping human users achieve higher task accuracy and lower response time compared to watching the full video. of generating summaries or task answers. By compari… view at source ↗
Figure 2
Figure 2. Computing VIBE Scores via Masked Inference. VIBE estimates grounding and utility scores using the next-token prediction mechanism of VLMs. The grounding score measures how well the video helps reconstruct a masked summary, while the utility score captures how much the summary improves task prediction given a masked video. We now formally describe our video-to-text information bottleneck evaluation (VIBE) method. Ins… view at source ↗
Figure 3
Figure 3. Pareto front of VIBE. Summaries selected by VIBE form a Pareto frontier across different (α, β) per Equation (5), demonstrating optimal trade-offs between grounding and utility. In contrast, Naive VLM summaries, author-written TLDRs, and abstracts from OpenReview in LearningPaper24 fall inside the frontier, indicating suboptimal scores. 5 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (a1, b1) Accuracy versus inverse response time. Each point represents an individual participant; large markers indicate group means with standard error of the mean. Shaded areas denote 2D kernel density estimates (threshold = 0.45). (a2, b2) Correlation between accurac…
Figure 5
Figure 5. Figure 5: Keyframe thumbnails from the qualitative example SlidesLive talk. The correct answer is [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: VIBE generalizes to various VLMs. Summaries selected by VIBE form a Pareto frontier not only across various (α, β) but also across various VLMs from different sources. We conduct an ablation study to examine how different VLM variants affect the reliability of VIBE. Sp…
Figure 7
Figure 7. Figure 7: Distribution of papers in LearningPaper24 by consolidated primary area. OpenReview’s primary areas are often too specific or overlapping, so we remap them into 12 broader and semantically coherent categories. For example, “self-supervised learning” and “representation …
Figure 8
Figure 8. Figure 8: Prolific recruitment interfaces for LearningPaper24 dataset showing video condition (top) and text conditions (bottom). Similar interfaces are used for the other two datasets. Instructions for LongVideoBench In this study, you will (watch/read) 10 (short videos/short s…
Figure 9
Figure 9. Figure 9: Scatter plots and Spearman correlations across all datasets. Trendlines shown for [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 41 canonical work pages

  1. [1]

    Guidelines for Human-AI Interaction

    S. Amershi, D. Weld, M. V orvoreanu, et al. “Guidelines for Human-AI Interaction”. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19. Glasgow, Scotland Uk: Association for Computing Machinery, 2019, pp. 1–13.ISBN: 9781450359702

    work page 2019

  2. [2]

    Interactive machine learning for health informatics: when do we need the human-in-the-loop?

    A. Holzinger. “Interactive machine learning for health informatics: when do we need the human-in-the-loop?” In:Brain informatics3.2 (2016), pp. 119–131

    work page 2016

  3. [3]

    Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities

    D. Xin, L. Ma, J. Liu, et al. “Accelerating Human-in-the-loop Machine Learning: Challenges and Opportunities”. In:Proceedings of the Second Workshop on Data Management for End-To- End Machine Learning. DEEM’18. Houston, TX, USA: Association for Computing Machinery, 2018.ISBN: 9781450358286

    work page 2018

  4. [4]

    Is human oversight to AI systems still possible?

    A. Holzinger, K. Zatloukal, and H. Müller. “Is human oversight to AI systems still possible?” In:New Biotechnology85 (2025), pp. 59–62.ISSN: 1871-6784

    work page 2025

  5. [5]

    Msr-vtt: A large video description dataset for bridging video and language

    J. Xu, T. Mei, T. Yao, and Y . Rui. “Msr-vtt: A large video description dataset for bridging video and language”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 5288–5296

    work page 2016

  6. [6]

    VaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research

    X. Wang, J. Wu, J. Chen, et al. “VaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research”. In:The IEEE International Conference on Computer Vision (ICCV). Oct. 2019

    work page 2019

  7. [7]

    AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

    W. Chai, E. Song, Y . Du, et al. “AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark”. In:The Thirteenth International Conference on Learning Representa- tions. 2025

    work page 2025

  8. [8]

    Spoken moments: Learning joint audio-visual representations from video descriptions

    M. Monfort, S. Jin, A. Liu, et al. “Spoken moments: Learning joint audio-visual representations from video descriptions”. In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, pp. 14871–14881

    work page 2021

Show all 49 references
  1. [9]

    Tishby, F

    N. Tishby, F. C. Pereira, and W. Bialek.The information bottleneck method. 2000. arXiv: physics/0004057 [physics.data-an]

    2000 arXiv

  2. [10]

    Longvideobench: A benchmark for long-context interleaved video-language understanding

    H. Wu, D. Li, B. Chen, and J. Li. “Longvideobench: A benchmark for long-context interleaved video-language understanding”. In:Advances in Neural Information Processing Systems37 (2024), pp. 28828–28857

    2024

  3. [11]

    Sutd-trafficqa: A question answering benchmark and an efficient network for video reasoning over traffic events

    L. Xu, H. Huang, and J. Liu. “Sutd-trafficqa: A question answering benchmark and an efficient network for video reasoning over traffic events”. In:Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021, pp. 9878–9888

    2021

  4. [12]

    J. T. Townsend and F. G. Ashby.Stochastic modeling of elementary psychological processes. CUP Archive, 1983

    1983

  5. [13]

    Video Question Answering via Gradually Refined Attention over Appearance and Motion

    D. Xu, Z. Zhao, J. Xiao, et al. “Video Question Answering via Gradually Refined Attention over Appearance and Motion”. In:ACM Multimedia. 2017

    2017

  6. [14]

    Zhong, J

    Y . Zhong, J. Xiao, W. Ji, et al.Video Question Answering: Datasets, Algorithms and Challenges

  7. [15]

    Any2Any: Incomplete Multimodal Retrieval with Confor- mal Prediction

    P.-h. Li, Y . Yang, M. Omama, et al. “Any2Any: Incomplete Multimodal Retrieval with Confor- mal Prediction”. In:arXiv preprint arXiv:2411.10513(2024)

    2024 arXiv

  8. [16]

    Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval

    M. Omama, P.-h. Li, and S. P. Chinchali. “Exploiting Distribution Constraints for Scalable and Efficient Image Retrieval”. In:The Thirteenth International Conference on Learning Representations. 2025

    2025

  9. [17]

    Rouge: A package for automatic evaluation of summaries

    C.-Y . Lin. “Rouge: A package for automatic evaluation of summaries”. In:Text summarization branches out. 2004, pp. 74–81

    2004

  10. [18]

    Bleu: a method for automatic evaluation of machine translation

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. “Bleu: a method for automatic evaluation of machine translation”. In:Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 2002, pp. 311–318

    2002

  11. [19]

    Cider: Consensus-based image descrip- tion evaluation

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh. “Cider: Consensus-based image descrip- tion evaluation”. In:Proceedings of the IEEE conference on computer vision and pattern recognition. 2015, pp. 4566–4575

    2015

  12. [20]

    Information-Theoretic Distillation for Reference-less Summa- rization

    J. Jung, X. Lu, L. Jiang, et al. “Information-Theoretic Distillation for Reference-less Summa- rization”. In:First Conference on Language Modeling. 2024. 11

    2024

  13. [21]

    Human-agent cooperation in games under incomplete information through natural language communication

    S. Chen, D. Fried, and U. Topcu. “Human-agent cooperation in games under incomplete information through natural language communication”. In:International Joint Conference on Artificial Intelligence, Human-Centred AI(2024)

    2024

  14. [22]

    Comparing automatic and human evaluation of NLG systems

    A. Belz and E. Reiter. “Comparing automatic and human evaluation of NLG systems”. In: 11th conference of the european chapter of the association for computational linguistics. 2006, pp. 313–320

    2006

  15. [23]

    Can machine translation systems be evaluated by the crowd alone

    Y . Graham, T. Baldwin, A. Moffat, and J. Zobel. “Can machine translation systems be evaluated by the crowd alone”. In:Natural Language Engineering23.1 (2017), pp. 3–30

    2017

  16. [24]

    Automatic summarization

    A. Nenkova, K. McKeown, et al. “Automatic summarization”. In:Foundations and Trends® in Information Retrieval5.2–3 (2011), pp. 103–233

    2011

  17. [25]

    Is summary useful or not? an extrinsic human evaluation of text summaries on downstream tasks

    X. Pu, M. Gao, and X. Wan. “Is summary useful or not? an extrinsic human evaluation of text summaries on downstream tasks”. In:arXiv preprint arXiv:2305.15044(2023)

    2023 arXiv

  18. [26]

    The speed-accuracy tradeoff: history, physiology, methodology, and behavior

    R. P. Heitz. “The speed-accuracy tradeoff: history, physiology, methodology, and behavior”. In:Frontiers in neuroscience8 (2014), p. 150

    2014

  19. [27]

    A cortical information bottleneck during decision- making

    M. Kleinman, T. Wang, D. Xiao, et al. “A cortical information bottleneck during decision- making”. In:bioRxiv(2023)

    2023

  20. [28]

    Bottlesum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle

    P. West, A. Holtzman, J. Buys, and Y . Choi. “Bottlesum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle”. In:arXiv preprint arXiv:1909.07405(2019)

    2019 arXiv

  21. [29]

    Improving the adversarial robustness of NLP models by information bottleneck

    C. Zhang, X. Zhou, Y . Wan, et al. “Improving the adversarial robustness of NLP models by information bottleneck”. In:arXiv preprint arXiv:2206.05511(2022)

    2022 arXiv

  22. [30]

    Applying the information bottleneck principle to unsu- pervised clustering of discrete and continuous image representations

    Gordon, Greenspan, and Goldberger. “Applying the information bottleneck principle to unsu- pervised clustering of discrete and continuous image representations”. In:Proceedings Ninth IEEE International Conference on Computer Vision. 2003, 370–377 vol.1

    2003

  23. [31]

    Deep learning and the information bottleneck principle

    N. Tishby and N. Zaslavsky. “Deep learning and the information bottleneck principle”. In: 2015 IEEE Information Theory Workshop (ITW). 2015, pp. 1–5

    2015

  24. [32]

    How does information bottleneck help deep learning?

    K. Kawaguchi, Z. Deng, X. Ji, and J. Huang. “How does information bottleneck help deep learning?” In:International Conference on Machine Learning. PMLR. 2023, pp. 16049–16096

    2023

  25. [33]

    The information bottleneck problem and its applications in machine learning

    Z. Goldfeld and Y . Polyanskiy. “The information bottleneck problem and its applications in machine learning”. In:IEEE Journal on Selected Areas in Information Theory1.1 (2020), pp. 19–38

    2020

  26. [34]

    Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations

    S. Mai, Y . Zeng, and H. Hu. “Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations”. In:IEEE Transactions on Multimedia25 (2023), pp. 4121–4134

    2023

  27. [35]

    Representation Learning In Deep RL Via Discrete Information Bottleneck

    R. Islam, H. Zang, M. Tomar, et al. “Representation Learning In Deep RL Via Discrete Information Bottleneck”. In:AISTATS 2023. May 2023

    2023

  28. [36]

    Normalized (pointwise) mutual information in collocation extraction

    G. Bouma. “Normalized (pointwise) mutual information in collocation extraction”. In:Pro- ceedings of GSCL30 (2009), pp. 31–40

    2009

  29. [37]

    Term-weighting approaches in automatic text retrieval

    G. Salton and C. Buckley. “Term-weighting approaches in automatic text retrieval”. In:Infor- mation Processing & Management24.5 (1988), pp. 513–523.ISSN: 0306-4573

    1988

  30. [38]

    S. P. Boyd and L. Vandenberghe.Convex optimization. Cambridge university press, 2004

    2004

  31. [39]

    https : / / github

    OpenAI.OpenAI Python Library. https : / / github . com / openai / openai - python. Accessed: 2025-04-28. 2025

    2025

  32. [40]

    S. Bai, K. Chen, X. Liu, et al.Qwen2.5-VL Technical Report. 2025. arXiv: 2502.13923 [cs.CV]

    2025 arXiv

  33. [41]

    https://github

    JaidedAI.EasyOCR: Ready-to-use OCR with 80+ supported languages. https://github. com/JaidedAI/EasyOCR. Accessed: 2025-05-15. 2020

    2025

  34. [42]

    Prolific. ac—A subject pool for online experiments

    S. Palan and C. Schitter. “Prolific. ac—A subject pool for online experiments”. In:Journal of Behavioral and Experimental Finance17 (2018), pp. 22–27

    2018

  35. [43]

    The proof and measurement of association between two things

    C. Spearman. “The proof and measurement of association between two things.” In: (1961)

    1961

  36. [44]

    RouteLLM: Learning to Route LLMs from Preference Data

    I. Ong, A. Almahairi, V . Wu, et al. “RouteLLM: Learning to Route LLMs from Preference Data”. In:The Thirteenth International Conference on Learning Representations. 2025

    2025

  37. [45]

    Mixture-of-experts with expert choice routing

    Y . Zhou, T. Lei, H. Liu, et al. “Mixture-of-experts with expert choice routing”. In:Advances in Neural Information Processing Systems35 (2022), pp. 7103–7114. 12

    2022

  38. [46]

    P.-h. Li, O. S. Toprak, A. Narayanan, et al.Online Foundation Model Selection in Robotics. arXiv:2402.08570 [cs]. Feb. 2024

    2024 arXiv

  39. [47]

    Z. Chen, W. Wang, Y . Cao, et al.Expanding Performance Boundaries of Open-Source Multi- modal Models with Model, Data, and Test-Time Scaling. 2025. arXiv:2412.05271 [cs.CV]

    2025 arXiv

  40. [48]

    self-supervised learning

    J. Zhu, W. Wang, Z. Chen, et al.InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. 2025. arXiv:2504.10479 [cs.CV]. 13 Appendix A Qualitative Result To illustrate differences across summary generation methods, we present a qualitativ...

    2025 arXiv

  41. [2022]

    arXiv:2203.01225 [cs.CV]

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.