BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201805Z
LOCATION:266
DTSTART;TZID=America/Chicago:20251117T161500
DTEND;TZID=America/Chicago:20251117T164500
UID:submissions.supercomputing.org_SC25_sess218_ws_waccpd104@linklings.com
SUMMARY:Scalable Neural Network Training: Distributed Data-Parallel Approa
 ches
DESCRIPTION:Fernando Vazquez-Novoa (Barcelona Supercomputing Center (BSC))
 , Pedro López and José Flich (Universidad Politecnica de Valencia), and Ro
 sa M. Badia (Barcelona Supercomputing Center (BSC))\n\nTraining large neur
 al networks is computationally demanding and often limited by synchronizat
 ion overhead in distributed environments. Traditional data-parallel framew
 orks, such as Horovod or PyTorch DDP, average gradients at every batch, wh
 ich can limit scalability due to communication bottlenecks. \nIn this work
 , we propose two novel data-parallel strategies that reduce synchronizatio
 n by averaging weights and biases only at the end of each epoch. These met
 hods are implemented using the PyCOMPSs task-based programming model and i
 ntegrated into dislib, enabled by a new distributed tensor abstraction (ds
 -tensor) that supports multidimensional data structures suitable for deep 
 learning workloads. \n\nWe evaluate our approach on classification and reg
 ression tasks using real-world datasets and federated learning scenarios. 
 Results show up to 95% training time reduction and strong scalability up t
 o 64 workers, while maintaining or improving model accuracy. Our strategie
 s enable asynchronous, communication-efficient training and are well-suite
 d for heterogeneous and large-scale HPC systems.\n\nRecording: Livestreame
 d, Recorded\n\nRegistration Category: Technical Program Reg Pass, Workshop
  Reg Pass\n\nSession Chairs: Andreas Herten (Forschungszentrum Jülich, Jül
 ich Supercomputing Centre (JSC)); Rabab Alomairy (Massachusetts Institute 
 of Technology (MIT), King Abdullah University of Science and Technology (K
 AUST)); and Jorge Luis Galvez Vallejo (Australian National University)\n\n
END:VEVENT
END:VCALENDAR