BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201803Z
LOCATION:267
DTSTART;TZID=America/Chicago:20251117T143000
DTEND;TZID=America/Chicago:20251117T150000
UID:submissions.supercomputing.org_SC25_sess198_ws_pmbsf104@linklings.com
SUMMARY:Pretraining LLMs at Scale: Tuning Strategies and Performance Porta
 bility.
DESCRIPTION:Adrián Pérez Diéguez, Àlex Batlle Casellas, Aleix Torres-Camps
 , Harris Teague, and Jordi Ros-Giralt (Qualcomm)\n\nTraining large languag
 e models (LLMs) at scale presents challenges that demand careful co-design
  across software, hardware, and parallelization strategies. In this work, 
 we introduce a communication-aware tuning methodology for optimizing LLM p
 retraining, and extend the performance portability metric to evaluate LLM-
 training efficiency across our systems. Our methodology, validated through
  LLM pretraining workloads at a leading global technology enterprise, deli
 vered up to 1.6x speedup over default configurations. We further provide s
 ix key insights that challenge prevailing assumptions in LLM training perf
 ormance, including the trade-offs between ZeRO stages, the default DeepSpe
 ed communication collectives, and the critical role of batch size choices.
  Our findings highlight the need for platform-specific tuning and advocate
  for a shift toward end-to-end co-design to unlock performance efficiency 
 in LLM training.\n\nRecording: Livestreamed, Recorded\n\nRegistration Cate
 gory: Technical Program Reg Pass, Workshop Reg Pass\n\nSession Chairs: Ste
 ven A. Wright (University of York, England); Simon Hammond (National Nucle
 ar Security Administration (NNSA)); and Sascha Hunold (Technical Universit
 y of Vienna)\n\n
END:VEVENT
END:VCALENDAR