BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201258Z
LOCATION:Second Floor Atrium
DTSTART;TZID=America/Chicago:20251121T080000
DTEND;TZID=America/Chicago:20251121T120000
UID:submissions.supercomputing.org_SC25_sess620_post197@linklings.com
SUMMARY:Optimizing the GPU All-Reduce Using Multiple Processes Per GPU
DESCRIPTION:Michael Adams and Amanda Bienz (University of New Mexico)\n\nL
 arge inter-GPU all-reduce operations, prevalent throughout deep learning, 
 are bottlenecked by communication costs. Emerging heterogeneous architectu
 res are comprised of complex nodes, often containing four GPUs and dozens 
 to hundreds of CPU cores per node. Parallel applications are typically acc
 elerated on the available GPUs, using only a single CPU core per GPU while
  the remaining cores sit idle. This poster presents novel optimizations to
  large GPU-aware all-reduce operations, extending lane-aware reductions to
  the GPUs, and notably using multiple CPU cores per GPU to accelerate thes
 e operations. These multi-CPU-accelerated GPU-aware lane all-reduces using
  an intermediate host buffer yield speedup of up to 2.45x for large MPI al
 l-reduces across the NVIDIA A100 GPUs of NCSA's Delta supercomputer. Final
 ly, the approach is extended to GPUDirect RDMA communication, yielding spe
 edup of 1.17x for large all-reduces.\n\nTag: Research & ACM SRC Posters\n\
 nRegistration Category: Technical Program Reg Pass\n\nSession Chairs: Kent
 o Sato (RIKEN Center for Computational Science (R-CCS)); Anja Gerbes (Geor
 g-August-Universität Göttingen); and Chris Schlipalius (Pawsey Supercomput
 ing Research Centre; Commonwealth Scientific and Industrial Research Organ
 isation (CSIRO), Australia)\n\n
END:VEVENT
END:VCALENDAR