Physical Reasoning from Video under Distribution Shift
This is an unofficial IOAI-style practice task created by LibrAI. The simulation foundation is adapted from Silicon23/mujoco-kinematics-probing under the MIT License; the competition splits, rendering randomization, and scoring are separate additions.
You are given the first half of a short video showing rigid objects moving under forces and occasionally colliding. Predict what happens during the hidden second half. For every future frame, your system must predict the target object's normalized image-plane position. It must also infer whether a scored collision will occur, the first collision frame, the image-plane acceleration, and the effective coefficient of restitution when that quantity is observable. The test set contains both familiar examples and unseen physical or visual conditions. A successful solution must recover motion from pixels and remain reliable under distribution shift.
Each sequence contains 64 frames at 16 FPS. Only frames 0–31 are provided for test clips; frames 32–63 are hidden and scored.
- train_*.mp4: observed training clips.
- test_*.mp4: observed test clips.
- train.csv: future-frame labels for training clips.
- test.csv: identifiers for the future rows to predict.
- sample_submission.csv: the required submission structure.
Submit one row for every row_id with these columns:
row_id,x,y,will_collide,first_collision_frame,accel_x,accel_y,restitution
will_collide is a probability. Clip-level quantities are repeated for all 32 future rows belonging to the same clip. All submitted values must be finite.
The score ranges from 0 to 100, and higher is better. It combines: - future trajectory accuracy; - collision probability and timing; - acceleration estimation; - restitution estimation when observable; and - performance across in-distribution, parameter-shift, and scene-shift clips.
Out-of-distribution clips receive most of the final weight. Submissions receive a score from 0 to 100; higher is better. The scientific score combines: | Component | Nominal weight | |---|---| | Future trajectory (ADE/FDE) | 50% | | Collision probability and time | 20% | | Image-plane acceleration | 15% | | Observable restitution | 10% | | Valid output | 5% |
Scores are first macro-averaged by clip. They are then computed within three hidden strata and combined as:
25% ID + 35% Parameter OOD + 40% Appearance/Scene OOD
The stratum of each test example is private. Missing IDs, duplicate IDs, incorrect columns, or non-finite values invalidate a submission.
You may use classical computer vision, learned video models, tracking, filtering, system identification, or hybrid physics/ML approaches. No specific foundation model is required.