BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201806Z
LOCATION:261-262-265-266
DTSTART;TZID=America/Chicago:20251119T103000
DTEND;TZID=America/Chicago:20251119T105200
UID:submissions.supercomputing.org_SC25_sess183_pap725@linklings.com
SUMMARY:Distributed Cross-Channel Hierarchical Aggregation for Foundation 
 Models
DESCRIPTION:Aristeidis Tsaris, Isaac Lyngaas, and John Lagergren (Oak Ridg
 e National Laboratory (ORNL)); Mohamed Wahib (RIKEN Center for Computation
 al Science (R-CCS)); and Larry York, Prasanna Balaprakash, Dan Lu, Feiyi W
 ang, and Xiao Wang (Oak Ridge National Laboratory (ORNL))\n\nVision-based 
 scientific foundation models hold significant promise for advancing scient
 ific discovery and innovation. This potential stems from their ability to 
 aggregate images from diverse sources—such as varying physical groundings 
 or data acquisition systems—and to learn spatio-temporal correlations usin
 g transformer architectures. However, tokenizing and aggregating images ca
 n be compute-intensive, a challenge not fully addressed by current distrib
 uted methods. In this work, we introduce the Distributed Cross-Channel Hie
 rarchical Aggregation (D-CHAG) approach designed for datasets with a large
  number of channels across image modalities. Our method is compatible with
  any model-parallel strategy and any type of vision transformer architectu
 re, significantly improving computational efficiency. We evaluated D-CHAG 
 on hyperspectral imaging and weather forecasting tasks. When integrated wi
 th tensor parallelism and model sharding, our approach achieved up to a 75
 % reduction in memory usage and more than doubled sustained throughput on 
 up to 1,024 AMD GPUs on the Frontier supercomputer.\n\nTag: Applications, 
 Architectures & Networks, HPC for Machine Learning\n\nRecording: Livestrea
 med, Recorded\n\nRegistration Category: Technical Program Reg Pass\n\nSess
 ion Chair: Bogdan Nicolae (Argonne National Laboratory (ANL), Illinois Ins
 titute of Technology)\n\n
END:VEVENT
END:VCALENDAR