近年来，大型视频语言模型（LVLMs）在多模态理解方面取得了显著进展，然而如何对其视频语境中的事实性进行有效评估仍是一个亟待解决的关键挑战。为填补这一空白，我们提出了Video SimpleQA，这是首个专门用于评估LVLMs事实性的综合基准。我们的工作通过以下几个关键特征区别于现有的视频基准：1）所需知识：要求整合超越显式叙述的外部知识；2）事实导向问题：针对客观、无争议的事件或关系，避免主观解读；3）明确且简短的答案：答案以简洁且无歧义的方式呈现，能够通过LLM-as-a-judge框架实现自动化评估，并尽量减少评分方差；4）外部来源验证：所有标注均经过与权威外部参考的严格验证，确保可靠性；5）时间推理需求：标注的问题类型涵盖静态单帧理解和动态时间推理，明确评估LVLMs在长上下文依赖下的事实性。我们对41个最先进的LVLMs进行了广泛评估，并总结了以下主要发现：1）当前LVLMs在事实遵守方面存在明显不足，尤其是开源模型表现更差，性能最佳的Gemini-1.5-Pro模型仅达到54.4%的F分数；2）测试时计算范式显示出微不足道的性能提升，揭示了通过后处理计算增强事实性的根本局限性；3）检索增强生成方法在提高性能的同时带来了额外的推理时间开销，呈现出重要的效率-性能权衡。

Recent advancements in Large Video Language Models (LVLMs) have highlighted their potential for multi-modal understanding, yet evaluating their factual grounding in video contexts remains a critical unsolved challenge. To address this gap, we introduce Video SimpleQA, the first comprehensive benchmark tailored for factuality evaluation of LVLMs. Our work distinguishes from existing video benchmarks through the following key features: 1) Knowledge required: demanding integration of external knowledge beyond the explicit narrative; 2) Fact-seeking question: targeting objective, undisputed events or relationships, avoiding subjective interpretation; 3) Definitive & short-form answer: Answers are crafted as unambiguous and definitively correct in a short format, enabling automated evaluation through LLM-as-a-judge frameworks with minimal scoring variance; 4) External-source verified: All annotations undergo rigorous validation against authoritative external references to ensure the reliability; 5) Temporal reasoning required: The annotated question types encompass both static single-frame understanding and dynamic temporal reasoning, explicitly evaluating LVLMs factuality under the long-context dependencies. We extensively evaluate 41 state-of-the-art LVLMs and summarize key findings as follows: 1) Current LVLMs exhibit notable deficiencies in factual adherence, particularly for open-source models. The best-performing model Gemini-1.5-Pro achieves merely an F-score of 54.4%; 2) Test-time compute paradigms show insignificant performance gains, revealing fundamental constraints for enhancing factuality through post-hoc computation; 3) Retrieval-Augmented Generation demonstrates consistent improvements at the cost of additional inference time overhead, presenting a critical efficiency-performance trade-off.