{"ID":6536518,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-14T20:11:26.822169759Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.10470","arxiv_id":"2607.10470","title":"On the Real-World Generalisability of Optical Flow Models","abstract":"Real-world deployment of vision models to broadly benefit society is arguably a main research objective. In optical flow, however, the difficulty to obtain the ground truth has focused research mainly on synthetic data and domain-specific benchmarks. Here, we investigate the severity of this mismatch. We study how well modern optical flow estimation models generalise to real-world video and question if accuracy on synthetic benchmark proxies actually predicts accuracy on real-world optical flow. To address this, we build a real-world evaluation benchmark and evaluate the real-world generalisability of a broad set of recent optical flow models using standard checkpoints. Our benchmark contains 8,204 frame pairs across TAP-Flow, Slow Flow, and our own dataset FlowFactor. FlowFactor is a manually annotated real-world benchmark of 1,000 HD frame pairs organised into four confounding factors: large displacements, repetitive textures, occlusions, and lighting variation. Each setting mainly varies only one factor, enabling diagnostic, confounder-specific analysis. Using FlowFactor, we reveal that performance on varying lighting and large displacements correlates most strongly with real-world accuracy, and that improvements on large-motion regimes can trade off against robustness in small-motion, stationary scenes. Our experiments show that progress on Sintel, KITTI and Spring only weakly predicts accuracy on real-world data, highlighting the need for a broad real-world optical flow benchmark. Interestingly, scaling up the amount of training data does not necessarily resolve the gap, calling for new innovative research instead of simply scaling data and compute.","short_abstract":"Real-world deployment of vision models to broadly benefit society is arguably a main research objective. In optical flow, however, the difficulty to obtain the ground truth has focused research mainly on synthetic data and domain-specific benchmarks. Here, we investigate the severity of this mismatch. We study how well...","url_abs":"https://arxiv.org/abs/2607.10470","url_pdf":"https://arxiv.org/pdf/2607.10470v1","authors":"[\"Petter Reijalt\",\"Sander Gielisse\",\"Rickard Karlsson\",\"Jan van Gemert\"]","published":"2026-07-11T20:18:23Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"eess.IV\"]","methods":"[\"Generative Adversarial Network\"]","has_code":false}