SOK-Bench
Our work aims to delve deeper into reasoning evaluations, specifically within dynamic, open-world, and structured context knowledge. SOK-Bench consists of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving.
Preview
SOK-Bench consists of 44K questions and 10K situations with instance-level annotations depicted in the videos. The reasoning process is required to understand and apply situated knowledge and general knowledge for problem-solving.
![](imgs/fig_overview.png)
Data Example
![](imgs/fig_qa_examples.png)
Data Download
Data Overview
44K Situated Questions
10K Situation Video Clips
Situation Commonsense Knowledge Graphs
Annotation Statistics
Data Download
Data Usage
Dataset ReadmeQuestions, Answers, Situated Commonsense Graphs, and Rationals
Data Download jsonSituation Video Sources
Video Source DataPaper
![](imgs/paper.png)