BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260202T201300Z
LOCATION:Second Floor Atrium
DTSTART;TZID=America/Chicago:20251121T080000
DTEND;TZID=America/Chicago:20251121T120000
UID:submissions.supercomputing.org_SC25_sess620_post167@linklings.com
SUMMARY:An Efficient GEMM Acceleration Method for LLM Inference with Varia
 ble-Length Sequences
DESCRIPTION:Yu Zhang and Lu Lu (South China University of Technology)\n\nT
 ransformer-based large language models (LLMs) have demonstrated remarkable
  capabilities in natural language processing (NLP) tasks. The transformer 
 layer in LLM involves substantial general matrix multiplication (GEMM). Ho
 wever, the sequence length variability leads to redundant computation and 
 hardware resource overhead in the GEMM with a uniform-size padding approac
 h, leading to reduced inference speed. \n\nThis work proposes an efficient
  GEMM acceleration method for LLM inference with variable-length sequences
 . First, a fused parallel prefix scan design is developed to capture the m
 atrix dimension distribution. Second, an efficient various-size tile kerne
 l is implemented based on Matrix Core, with an analysis of the hardware re
 source requirements in the computation process. Third, a hardware-aware ti
 ling algorithm is designed to select the optimal tiling scheme based on th
 read parallelism and hardware resources. The experimental results show tha
 t the proposed approach achieves performance improvements of 3.10x and 2.9
 9x (up to 4.44x and 4.27x) over hipBLAS and rocBLAS.\n\nTag: Research & AC
 M SRC Posters\n\nRegistration Category: Technical Program Reg Pass\n\nSess
 ion Chairs: Kento Sato (RIKEN Center for Computational Science (R-CCS)); A
 nja Gerbes (Georg-August-Universität Göttingen); and Chris Schlipalius (Pa
 wsey Supercomputing Research Centre; Commonwealth Scientific and Industria
 l Research Organisation (CSIRO), Australia)\n\n
END:VEVENT
END:VCALENDAR