SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge

SOK-Bench

Our work aims to delve deeper into reasoning evaluations, specifically within dynamic, open-world, and structured context knowledge. SOK-Bench consists of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving.

Preview

SOK-Bench consists of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving.

Data Example

Data Download

Data Overview

44K Situated Questions

10K Situation Video Clips

Situation Commonsense Knowledge Graphs

Annotation Statistics

Data Download

Data Usage

Dataset Readme

Questions, Answers, Situated Commonsense Graphs, and Rationals

Data Download json

Situation Video Sources

Video Source Data

Paper

Link to Paper

@inproceedings{SOK-Bench, author = {Wang*, Andong and Wu*, Bo and Chen, Sunli and Chen, Zhenfang and Guan, Haotian and Lee, Wei-Ning and Li, Erran Li and Tenenbaum, Joshua B and Gan, Chuang}, title = {SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge}, booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year = {2024} }