Pith. sign in

REVIEW 5 cited by

HCQA @ Ego4D EgoSchema Challenge 2024

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.15771 v2 pith:BDHQXFVJ submitted 2024-06-22 cs.CV cs.AI

Haoyu Zhang , Yuquan Xie , Yisen Feng , Zaijing Li , Meng Liu , Liqiang Nie This is my paper
classification cs.CVcs.AI
keywords hcqaansweringegoschemainformationquestioncaptionchallengecontext-driven
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this report, we present our champion solution for Ego4D EgoSchema Challenge in CVPR 2024. To deeply integrate the powerful egocentric captioning model and question reasoning model, we propose a novel Hierarchical Comprehension scheme for egocentric video Question Answering, named HCQA. It consists of three stages: Fine-grained Caption Generation, Context-driven Summarization, and Inference-guided Answering. Given a long-form video, HCQA captures local detailed visual information and global summarised visual information via Fine-grained Caption Generation and Context-driven Summarization, respectively. Then in Inference-guided Answering, HCQA utilizes this hierarchical information to reason and answer given question. On the EgoSchema blind test set, HCQA achieves 75% accuracy in answering over 5,000 human curated multiple-choice questions. Our code will be released at https://github.com/Hyu-Zhang/HCQA.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DIVE: Deep-search Iterative Video Exploration A Technical Report for the CVRR Challenge at CVPR 2025

    cs.CV 2025-06 conditional novelty 5.0 of 10

    DIVE, an iterative question-decomposition system with intent estimation and object-centric video summarization, achieves 81.44% on CVRR-ES.

  2. OSGNet @ Ego4D Episodic Memory Challenge 2025

    cs.CV 2025-06 conditional novelty 4.0 of 10

    OSGNet, an early-fusion grounding model, wins all three Ego4D Episodic Memory Challenge tracks by converting localization tasks into retrieval problems.

  3. Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A three-stage pipeline using the EgoVideo-V encoder, a verb-noun co-occurrence reranker, SAM2 hand-object features, and a fine-tuned Llama 2 model took first place in the Ego4D 2025 long-term action anticipation challenge.

  4. HCQA-1.5 @ Ego4D EgoSchema Challenge 2025

    cs.CV 2025-05 conditional novelty 4.0 of 10

    An ensemble of LLMs with confidence filtering and low-confidence re-reasoning reaches 77% accuracy on the EgoSchema benchmark, up from 75% for the prior HCQA system.

  5. Four Eyes Are Better Than Two: Harnessing the Collaborative Potential of Large Models via Differentiated Thinking and Complementary Ensembles

    cs.CV 2025-05 reject novelty 4.0 of 10

    A training-free ensemble of commercial VLMs with prompt and chain-of-thought engineering reaches 79% on EgoSchema, ranking 2nd in the CVPR 2025 challenge, but the ensemble weights are fit to the test labels.

Pith tools