Evaluation

Metrics, Leaderboards & Submissions

Methods are ranked by the RbS-Score, which balances clean Spring performance and corrupted RobustSpring performance with equal weight.

Metric

RbS-Score

Each quantitative task is ranked using the Robustness-balanced Spring Score (RbS-Score). The score combines a method's accuracy on the clean Spring benchmark (S) with its accuracy on the corrupted RobustSpring benchmark (R). Both components receive equal weight, and lower is better.

RbS-Scoret(m) = 0.5 · ESt(m) / BSt + 0.5 · ERt(m) / BRt
  • m: the evaluated method.
  • t: the quantitative task, such as optical flow, stereo, or scene flow.
  • ESt(m): the method's endpoint-error-style metric on the clean Spring data.
  • ERt(m): the same error metric evaluated on RobustSpring, averaged over all included corruption types.
  • BSt, BRt: fixed task-specific baseline errors used to normalize the clean and corrupted results.

Task-specific error metrics

The same task-specific error definition is used for Spring and RobustSpring. RobustSpring therefore measures prediction accuracy on corrupted inputs rather than only the change between clean and corrupted predictions.

  • Optical flow: mean endpoint error (EPE).
  • Stereo: mean absolute disparity error (Abs).
  • Scene flow: a balanced combination of the errors for reference-frame disparity d1, target-frame disparity d2, and optical flow. Each component is normalized by its own baseline error before the three components are averaged:
    RbS-ScoreSF(m) = 0.5 · [ ( Absd1,S(m) / Bd1,S + Absd2,S(m) / Bd2,S + EPEflow,S(m) / Bflow,S ) / 3 ] + 0.5 · [ ( Absd1,R(m) / Bd1,R + Absd2,R(m) / Bd2,R + EPEflow,R(m) / Bflow,R ) / 3 ]

A score of 1.0 indicates that the method matches the selected baselines on average. Scores below 1.0 indicate better normalized performance, while scores above 1.0 indicate worse normalized performance.

Tie-breaking

If two methods obtain the same RbS-Score, ties are resolved in the following order:

  1. Lower normalized RobustSpring term
  2. Lower normalized Spring term
Leaderboards

Public rankings

Optical Flow Leaderboard Coming soon Stereo Matching Leaderboard Coming soon Scene Flow Leaderboard Coming soon Cross-Task Leaderboard Coming soon Exploration Track Coming soon
Submissions

Submission workflow

  1. Generate predictions for the selected task
  2. Package archive in the required format
  3. Validate archive locally using the starting kit
  4. Submit to the benchmark server using your unique team ID
  5. Check leaderboard for results
Workshop Paper

Paper requirements

Each quantitative or exploration entry must submit a 4–6 page single-blind workshop paper. The paper should describe the method, implementation details, training and pretraining data, compute budget, results, ablations, and limitations.

Papers will be reviewed by the organizers and additional external reviewers with expertise in optical flow, stereo, scene flow, robustness, and benchmark design. Acceptance will depend on technical soundness, clarity, reproducibility, and relevance to robust dense correspondence, not only on leaderboard rank. All accepted papers will be forwarded for the official PMLR volume of NeurIPS workshop proceedings.

Reproducibility package

With the camera-ready deadline, each entry must submit a reproducibility package together with the paper. Award eligibility requires an accepted workshop paper and a complete package that meets the requirements below.

  • Dataset declaration. List every dataset used for training, pretraining, fine-tuning, and validation (not only Spring and RobustSpring).
  • Training scripts. Provide runnable scripts or commands that reproduce training end to end.
  • Evaluation scripts. Provide runnable scripts that reproduce the metrics and results reported in the paper on the declared data splits.
  • Resource requirements. Document hardware and runtime needs for both training and evaluation, including GPU type, GPU count, peak memory, and wall-clock time.
  • README. Include a detailed, concise setup guide that explains how to install dependencies and rerun training and evaluation. Pin the environment either with a pyproject.toml following the Python packaging guide, or with explicit conda instructions that list every library and version required to recreate the environment.