NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding
Abstract
Fine-grained questions about long videos ask how many times an action occurs, in what order events unfold, or exactly when a state changes. Answering them can require finding brief moments scattered across hours of footage. Agentic models that decide when and where to look promise to find these moments, yet this ability is hard both to measure and to learn. Sparse supporting evidence may establish that an action occurred without establishing its total count; verifying timeline-wide claims requires checking for missed occurrences and double counting. Answer-only supervision, in turn, does not explicitly supervise the search for evidence. We present NexSight, which builds verifiable tasks to both evaluate and train agentic long-video understanding. Its agentic pipeline, SightForge, accepts a task only if a verifier blind to the draft answer can reconstruct it from evidence densely reacquired from the raw video and counterexample checks fail to refute it. This yields SightLine, 310 evidence-traceable tasks from 108 long videos, evaluated through SightLoop, a shared multiscale harness supporting temporal exploration, local inspection, evidence integration, and revisiting while recording returned observations. On disjoint videos, solver trajectories on verified tasks become step-level supervision for observation decisions. The strongest model reaches only 54.52% on SightLine, yet 79.35% when given the annotated evidence. With under 6K examples, Sight-35B nearly doubles its base model's interactive accuracy (21.29% to 42.26%), outperforms the 397B model of its family, begins using explicit spatial inspection after post-training, which its base model never uses, and gains over 8 points on two external long-video benchmarks.
Community
๐ Excited to share NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding!
๐ฌ SightForge builds verifiable tasks, SightLine benchmarks fine-grained reasoning, and SightLoop enables interactive, evidence-driven video understanding.
๐ With fewer than 6K training examples, Sight-35B improves interactive accuracy from 21.29% to 42.26%.
๐ Towards more reliable and evidence-grounded video agents. Feedback is welcome!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Open-o3-Video-v2: Seek, Ground, and Reason with Verifiable Spatio-Temporal Evidence (2026)
- 4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents (2026)
- Supervising the Visible State: Separating Representation from View Selection in Streaming Video (2026)
- OmniFysics-Captioner Technical Report: Grounding Omni-Modal Understanding in the Physical World for Better Captioning (2026)
- ASSEMBLE: Atomic Skills for Evidence-Grounded Video Reasoning (2026)
- VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding (2026)
- VideoEvoHarness: Self Evolving Agent Harness from Unlabeled Videos for Video Understanding (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.22790 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper