Video-BrowseComp: Benchmarking Agentic Video Research on Open Web
Zhengyang Liang1,♣, Yan Shu2,♣, Xiangrui Liu3, Minghao Qin3, Kaixin Liang4, Paolo Rota2, Nicu Sebe2, Zheng Liu3, Lizi Liao1
1Singapore Management University, 2University of Trento, 3Beijing Academy of Artificial Intelligence, 4Beijing University of Posts and Telecommunications
The evolution of autonomous agents is redefining information seeking, transitioning from passive retrieval to proactive, open-ended web research. However, while textual and static multimodal agents have seen rapid progress, a significant modality gap remains in the web's most dynamic modality: video. Existing video benchmarks predominantly focus on passive perception, feeding curated clips to models without requiring external retrieval. They fail to evaluate agentic video research, which necessitates actively interrogating video timelines, cross-referencing dispersed evidence, and verifying claims against the open web.
To bridge this gap, we present Video-BrowseComp, a challenging benchmark comprising 210 questions tailored for open-web agentic video reasoning. Unlike prior benchmarks, Video-BrowseComp enforces a mandatory dependency on temporal visual evidence, ensuring that answers cannot be derived solely through text search but require navigating video timelines to verify external claims. Our evaluation of state-of-the-art models reveals a critical bottleneck: even advanced search-augmented models like GPT-5.1 (w/ Search) achieve only 23.81% accuracy. Our analysis reveals that these models largely rely on textual proxies, excelling in metadata-rich domains (e.g., TV shows with plot summaries) but collapsing in metadata-sparse, dynamic environments (e.g., sports, gameplay) where visual grounding is essential. As the first open-web video research benchmark, Video-BrowseComp advances the field beyond passive perception toward proactive video reasoning.
Leaderboard
We evaluate state-of-the-art models on Video-BrowseComp. The accuracy (%) is reported for Overall (OA) and three difficulty levels (Level 1, Level 2, Level 3).
Model
Overall Acc (%)
Level 1 (%)
Level 2 (%)
Level 3 (%)
Calibration Error (%)
Tool-Free Models
Qwen3-VL-8B-Thinking
7.14
12.00
0.00
0.00
52.49
Qwen3-VL-235B-A22B-Instruct
13.33
22.40
0.00
0.00
77.64
GLM-4.6V
10.95
16.80
3.23
0.00
44.40
gpt-4o-2024-11-20
17.62
28.00
3.23
0.00
58.81
gpt-4o-mini-2024-07-18
9.52
16.00
0.00
0.00
63.55
gpt-5-mini-2025-08-07
15.71
26.40
0.00
0.00
37.47
gemini-2.5-flash-2025-06
16.67
27.20
1.61
0.00
77.79
gemini-2.5-pro-2025-06
19.52
31.20
3.23
0.00
79.18
Search Models
gemini-2.5-flash-2025-06 (w/ Search)
20.95
32.80
4.84
0.00
35.98
gemini-2.5-pro-2025-06 (w/ Search)
23.81
37.60
4.84
0.00
31.45
gpt-5.1-2025-11-13 (w/ Search)
15.24
21.60
6.45
4.35
30.20
o4-mini-deep-research-2025-06-26
22.86
30.40
12.90
8.70
42.55
Citation
@article{liang2025video,
title={Video-BrowseComp: Benchmarking Agentic Video Research on Open Web},
author={Liang, Zhengyang and Shu, Yan and Liu, Xiangrui and Qin, Minghao and Liang, Kaixin and Rota, Paolo and Sebe, Nicu and Liu, Zheng and Liao, Lizi},
journal={arXiv preprint},
year={2025}
}