← Back to challenges

Object Tracking in the Real World

Train with partially paired modalities; deploy with full or missing sets using modality-agnostic representations.

Challenge objectives

  • Conducting RGBT object tracking under different modality-missing conditions.
  • Evaluation is conducted using the official scoring protocol, which emphasizes tracker performance under both normal and modality-missing conditions.

Introduction

Existing RGBT tracking methods are predominantly developed and evaluated under the assumption that both RGB and thermal modalities are always available and complete. However, this assumption is often violated in real-world applications. In practical deployments, sensors may suffer from temporary failures, transmission interruptions, environmental interference, or hardware degradation, leading to partial or complete absence of one modality. This challenge specifically focuses on modality missing rather than general degradation, aiming to expose the limitations of current RGBT tracking methods that rely heavily on complete RGBT inputs. By systematically simulating modality-missing conditions, the benchmark assesses a tracker's robustness, adaptability, and capability to maintain stable target localization when one modality is unavailable.

Dataset Description

This challenge addresses RGBT tracking in real-world scenarios with prevalent modality-missing conditions. We construct a new modality-missing tracking test dataset based on ground and aerial platforms, comprising a total of 90 modality-missing tracking sequences. Among these, 60 sequences are used for the first phase of public evaluation, while the remaining 30 sequences are reserved for the second phase of testing (all sequences will be made publicly available after the workshop).

Specifically, the entire dataset contains targets from 14 object categories, comprising a total of 37,385 frames, with a maximum sequence length of 2,977 frames and an average sequence length of 415.4 frames. In addition, the dataset includes five missing-modality patterns: random_missing, long_term_missing, swap_missing, long_term_and_random_missing, and swap_and_random_missing. For detailed definitions of these missing patterns, please refer to the paper "Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks".

For more details on dataset usage and evaluation protocols, please visit: https://www.codabench.org/competitions/13582/

Evaluation Metrics

We introduce a new evaluation score to assess tracker performance under missing-modality scenarios. We adopt Precision Rate (PR) and Success Rate (SR) as the primary evaluation metrics. The final score is jointly determined by the PRt and SRt over the complete sequences, as well as the PRm and SRm under modality-missing scenarios.

Challenge Deadlines

Phase 1 - Validation Phase

February 1 - March 3, 2026

Training and validation data will be released, validation server will be active.

Phase 2 - Test Phase

March 4 - March 10, 2026

Participants are required to submit the model, code, and a description of the solution for the organizers to conduct testing on unpublished data, in order to determine the final ranking.

Phase 3 - Paper Submission Phase

March 12 - March 30, 2026

The top-ranking participants will be invited for co-authoring the challenge report. Please note that all the teams have full freedom to decide whether they also wish to submit a separate full workshop paper.