Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling
DescriptionSparse observations and coarse-resolution climate models limit regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, high-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence-scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 32,768 GPUs, achieving up to 1.8 ExaFLOPS sustained throughput and 92%–98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. At 7 km resolution, ORBIT-2 achieves high accuracy with $R^2$ scores in the range of 0.98–0.99 against observational data.
Authors
Event Type
ACM Gordon Bell Climate Modeling Finalist
ACM Gordon Bell Finalist
Awards and Award Talks
TimeTuesday, 18 November 20252:37pm - 3:00pm CST
Location261-262-265-266
BP
GBC