BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201805Z
LOCATION:267
DTSTART;TZID=America/Chicago:20251117T140000
DTEND;TZID=America/Chicago:20251117T143000
UID:submissions.supercomputing.org_SC25_sess198_ws_pmbsf121@linklings.com
SUMMARY:MoE-Inference-Bench: Performance Evaluation of Mixture of Expert L
 arge Language and Vision Models
DESCRIPTION:Krishna Teja Chitty-Venkata (Argonne National Laboratory (ANL)
 ); Sylvia Howland, Golara Azar, Daria Soboleva, and Natalia Vassilieva (Ce
 rebras); Siddhisanket Raskar (Pacific Northwest National Laboratory (PNNL)
 ); and Murali Emani and Venkatram Vishwanath (Argonne National Laboratory 
 (ANL))\n\nMixture of Experts (MoE) models have enabled the scaling of Larg
 e Language Models (LLMs) and Vision Language Models (VLMs) by achieving ma
 ssive parameter counts while maintaining computational efficiency. However
 , MoEs introduce several inference-time challenges, including load imbalan
 ce across experts and the additional routing computational overhead. To ad
 dress these challenges and fully harness the benefits of MoE, a systematic
  evaluation of hardware acceleration techniques is essential. We present M
 oE-Inference-Bench, a comprehensive study to evaluate MoE performance acro
 ss diverse scenarios. We analyze the impact of batch size, sequence length
 , and critical MoE hyperparameters such as FFN dimensions and number of ex
 perts on throughput. We evaluate several optimization techniques on Nvidia
  H100 GPUs, including pruning, Fused MoE operations, speculative decoding,
  quantization, and various parallelization strategies. Our evaluation incl
 udes MoEs from the Mixtral, DeepSeek, OLMoE and Qwen families. The results
  reveal performance differences across configurations and provide insights
  for the efficient deployment of MoEs.\n\nRecording: Livestreamed, Recorde
 d\n\nRegistration Category: Technical Program Reg Pass, Workshop Reg Pass\
 n\nSession Chairs: Steven A. Wright (University of York, England); Simon H
 ammond (National Nuclear Security Administration (NNSA)); and Sascha Hunol
 d (Technical University of Vienna)\n\n
END:VEVENT
END:VCALENDAR