OpenCodePapers

moment-retrieval-on-charades-sta

Visual GroundingVideo GroundingMoment Retrieval
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
PaperCodeR@1 IoU=0.5R@1 IoU=0.7R@5 IoU=0.5R@5 IoU=0.7R@1 IoU=0.3mIoUModelNameReleaseDate
Universal Video Temporal Grounding with Generative Multi-modal Large Language Models74.3353.71UniTime-SP2025-06-23
TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning for Video LLMs✓ Link74.152.983.662.1TempSamp-R1 (CoT)2025-09-22
SARL-STG: A Spatially Aware Reinforcement Learning Framework for Refining MLLMs in Spatio-Temporal Video Grounding73.649.161.3SARL-STG2026-06-03
The Devil is in the Spurious Correlations: Boosting Moment Retrieval with Dynamic Learning✓ Link73.4953.01TD-DETR (InternVideo2) (extra data)2025-01-13
Invert4TVG: A Temporal Video Grounding Framework with Inversion Tasks Preserving Action Understanding Ability✓ Link72.551.483.0Invert4TVG 7B (extra data)2025-08-10
TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM✓ Link72.250.182.860.9Time-R1-7B (fine-tuned) (extra data)2025-03-17
Sparse-Dense Side-Tuner for efficient Video Temporal Grounding✓ Link72.052.661.2SDST (InternVideo2)2025-07-10
See More, Store Less: Memory-Efficient Resolution for Video Moment Retrieval71.2649.81SMORE2026-01-14
Saliency-Guided DETR for Moment Retrieval and Highlight Detection✓ Link71.1052.80SG-DETR (w/ PT)2024-10-02
LLaVA-MR: Large Language-and-Vision Assistant for Video Moment Retrieval✓ Link70.6549.58LLaVA-MR2024-11-21
Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning✓ Link70.445.781.959.1VideoReasoner2026-06-03
FlashVTG: Feature Layering and Adaptive Score Handling Network for Video Temporal Grounding✓ Link70.3249.87FlashVTG2024-12-18
Saliency-Guided DETR for Moment Retrieval and Highlight Detection✓ Link70.2049.50SG-DETR2024-10-02
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding✓ Link70.0348.95InternVideo2-6B2024-03-22
VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understanding✓ Link70.048.3VideoLoom-8B2026-01-12
DTOS: Dynamic Time Object Sensing with Large Multimodal Model✓ Link69.8143.281.2958.0DTOS-TCS2025-06-11
DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos✓ Link68.7947.5591.5372.96DeCaf-Grounder (extra data)2025-05-22
Measure Twice, Cut Once: Grasping Video Structures and Event Semantics with LLMs for Video Temporal Localization✓ Link68.541.682.3MeCo (QWen2VL 7B) (extra data)2025-03-12
Knowing Where to Focus: Event-aware Transformer for Video Grounding✓ Link68.4744.92EaTR2023-08-14
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding✓ Link68.3645.03InternVideo2-1B2024-03-22
Temporal-Aware Reasoning Optimization for Video Temporal Grounding✓ Link67.841.983.2TaRO 8B (extra data)2026-06-08
SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding67.646.779.558.3SlotVTG 7B (extra data)2026-03-26
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning✓ Link67.143.0VideoChat-T (FT)2024-10-25
VideoTG-R1: Boosting Video Temporal Grounding via Curriculum Reinforcement Learning on Reflected Boundary Annotations✓ Link64.839.279.555.9VideoTG-R1 (extra data)2025-10-27
UniMD: Towards Unifying Moment Retrieval and Temporal Action Detection✓ Link63.9844.4691.9467.72UniMD+Sync.2024-04-07
SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability✓ Link63.638.5SpaceVLLM 7B (extra data)2025-03-18
OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding✓ Link63.237.078.3OmniVTG-7B (zero-shot) (extra data)2026-04-28
CVA: Context-aware Video-text Alignment for Video Temporal Grounding62.6140.7874.1953.35CVA (extra data)2026-03-26
LD-DETR: Loop Decoder DEtection TRansformer for Video Moment Retrieval and Highlight Detection✓ Link62.5841.5673.92 53.44LD-DETR2025-01-18
VideoLights: Feature Refinement and Cross-Task Alignment Transformer for Joint Video Highlight Detection and Moment Retrieval✓ Link61.9641.0573.3352.94VideoLights-B-pt2024-12-02
GIRL-DETR: Gradient-Isolated Reinforcement Learning for Video Moment Retrieval✓ Link61.7537.47GIRL-DETR2026-05-30
TRACE: Temporal Grounding Video LLM via Causal Event Modeling✓ Link61.741.4TRACE (extra data)2024-10-08
Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding✓ Link61.0833.1776.9152.31SemVID (Qwen3-VL-4B, 25% visual-token retention)2026-03-05
HERO: Hierarchical Embedding-Refinement for Open-Vocabulary Temporal Sentence Grounding in Videos61.0541.29HERO (extra data)2026-03-06
UnLoc: A Unified Framework for Video Localization Tasks✓ Link60.838.488.261.1UnLoc-L2023-08-21
T2SGrid: Temporal-to-Spatial Gridification for Video Temporal Grounding60.635.976.953.2Qwen2-VL-7B + T2SGrid-FT (extra data)2026-03-07
Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Grounding✓ Link60.238.071.651.6SRAM-Base2024-08-29
SE-DETR: Explicit Semantic Exploration for Generalizability and Distinguishability in Video Temporal Grounding✓ Link60.239.771.151.2SE-DETR+2026-09-08
UniVTG: Towards Unified Video-Language Temporal Grounding✓ Link60.1938.5572.6352.17UniVTG w/ PT (extra data)2023-07-31
BAM-DETR: Boundary-Aligned Moment Detection Transformer for Temporal Sentence Grounding in Videos✓ Link59.9539.38BAM-DETR2023-11-30
$R^2$-Tuning: Efficient Image-to-Video Transfer Learning for Video Temporal Grounding✓ Link59.7837.0270.9150.86R2-Tuning2024-03-31
Background-aware Moment Detection for Video Moment Retrieval✓ Link59.4838.33BM-DETR2023-06-05
Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection✓ Link59.2536.64UVCOM2023-11-28
VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning✓ Link59.131.273.550.2VideoMind 7B (zero-shot) (extra data)2025-03-17
Moment Quantization for Video Temporal Grounding✓ Link58.8438.8470.9751.10MQVTG (extra data)2025-04-03
DSE-VTG: Dual-Side Enhancement for Training-Free Video Temporal Grounding58.5832.2375.6551.30DSE-VTG2026-09-08
Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding✓ Link58.4436.34CG-DETR2023-11-15
Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding✓ Link58.430.178.952.4PTD Qwen3-VL-4B2026-08-28
Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval✓ Link58.3136.49LLMEPET2024-07-21
HawkEye: Training Video-Text LLMs for Grounding Text in Videos✓ Link58.328.872.549.3HawkEye 7B (fine-tuned) (extra data)2024-03-15
MomentSeg: Moment-Centric Sampling for Enhanced Referring Video Object Segmentation58.225.876.150.2MomentSeg-3B2026-09-08
UnLoc: A Unified Framework for Video Localization Tasks✓ Link58.135.487.459.1UnLoc-B2023-08-21
VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding✓ Link57.833.9VTG-LLM (τ=0.1) (extra data)2024-05-22
GroundVTS: Visual Token Sampling in Multimodal Large Language Models for Video Temporal Grounding✓ Link57.534.271.550.1GroundVTS-Q (extra data)2026-04-02
Query-Dependent Video Representation for Moment Retrieval and Highlight Detection✓ Link57.3132.55QD-DETR (Only Video)2023-03-24
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding✓ Link57.1836.05video-mamba-suite2024-03-14
Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos✓ Link57.1034.54MIGCN (I3D)2021-10-12
Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection✓ Link56.5133.66TaskWeave (VGG)2024-04-14
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link56.0Penguin-VL 8B2026-03-06
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models✓ Link55.933.270.249.4Grounded-VideoLLM Phi-3.5 (task-tuned) (extra data)2024-10-04
MomentSeg: Moment-Centric Sampling for Enhanced Video Pixel Understanding✓ Link55.925.378.950.7MomentSeg-3B (θ=0.3) (extra data)2025-10-10
QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries✓ Link55.6534.17Moment-DETR w/ PT (on 10K HowTo100M videos)2021-07-20
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link54.5Penguin-VL 2B2026-03-06
QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries✓ Link53.6331.37Moment-DETR2021-07-20
Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs✓ Link52.532.674.751.0UG-Ground (InternVideo2.5-8B)2025-10-01
DART: Difficulty-Adaptive Routing for Zero-Shot Video Temporal Grounding52.0429.4570.9848.93DART2026-07-01
Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding✓ Link50.425.771.347.2F2G (Qwen3-VL-8B, zero-shot) (extra data)2026-05-21
UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection✓ Link49.3526.1689.4154.95UMT (VO)2022-03-23
Explore-And-Match: Bridging Proposal-Based and Proposal-Free With Transformer for Sentence Grounding in Videos✓ Link49.1126.5988.555.9947.13LVTR-CLIP (extra data)2022-01-25
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning✓ Link48.724.045.43VideoChat-T (ZS)2024-10-25
UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection✓ Link48.3129.2588.7956.08UMT (VA)2022-03-23
SnAG: Scalable and Accurate Video Grounding✓ Link46.2633.5292.5571.94SnAG2024-04-02
LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding✓ Link44.823.463.142.4LLaVA-ST (extra data)2025-01-14
SimVTP: Simple Video Text Pre-training with Masked Autoencoders44.726.383.755.1SimVTP2022-12-07
ARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shorts✓ Link54.8ARC-Hunyuan-Video-7B (extra data)2025-07-28
Text-Visual Prompting for Efficient 2D Temporal Video Grounding✓ Link44.3921.5165.92TVP (extra data)2023-03-09
Text-Visual Prompting for Efficient 2D Temporal Video Grounding✓ Link44.39TVP2023-03-09
VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT✓ Link43.6825.9459.4839.81VTG-GPT (zero-shot) (extra data)2024-03-04
VideoLLM Knows When to Speak: Enhancing Time-Sensitive Video Comprehension with Video-Text Duet Interaction Format✓ Link42.418.0MMDuet (zero-shot) (extra data)2024-11-27
Number it: Temporal Grounding Videos like Flipping Manga✓ Link42.020.663.841.4LongVA-7B-DPO + NumPro-FT (extra data)2024-11-15
D3G: Exploring Gaussian Prior for Temporal Sentence Grounding with Glance Annotation✓ Link41.6419.6079.2549.30D3G (glance-supervised)2023-08-08
Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language✓ Link39.8123.2579.3352.152D-TAN (Conv)2019-12-08
Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrieval39.721.062.741.9Self-SiMS2026-07-21
Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Grounding31.213.7TimeChat + Seq2Time (extra data)2024-11-25
Joint Moment Retrieval and Highlight Detection Via Natural Language Queries✓ Link25.9713.2584.8740.38VisionaryVid (video + audio)2023-05-08
VideoTemp-o3: Harmonizing Temporal Grounding and Video Understanding in Agentic Thinking-with-Videos33.057.8VideoTemp-o3-7B-RL (extra data)2026-02-08
Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency✓ Link58.0Temporal-RLT (4096 video tokens) (extra data)2025-06-02
TimeChat-Captioner: Scripting Multi-Scene Videos with Time-Aware and Structural Audio-Visual Captions✓ Link58.8TimeChat-Captioner (7B, GRPO)2026-02-09
VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice✓ Link63.7VideoAuto-R1 (Qwen3-VL-8B)2026-01-08
VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting✓ Link42.7VisionCoach-7B2026-03-15
ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning✓ Link42.0ResAdapt (Qwen2.5-VL-7B, 128 frames)2026-03-30
ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning✓ Link50.1ParaVT (Qwen3-VL-8B)2026-05-19
DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models✓ Link43.8DRIFT (ET-Chat)2026-06-04
Less from More: Reinforcing Sparse Video Reasoning from Dense References✓ Link54.8SAVER-4B (extra data)2026-10-07
EventLens: Event-Structure Reinforcement Learning for Video Understanding✓ Link48.25EventLens-8B2026-10-12