Presentation
High-Performance Sparse Attention on Tensor Cores: Fused3S and Beyond
DescriptionSparse attention is a core building block in many leading neural network models, from graph-structured learning to sparse sequence modeling. It can be decomposed into a sequence of three sparse matrix operations (3S): sampled dense-dense matrix multiplication (SDDMM), softmax normalization, and sparse matrix multiplication (SpMM). Efficiently executing the 3S computational pattern on modern GPUs remains challenging due to (a) the mismatch between unstructured sparsity and tensor cores optimized for dense operations, and (b) the high cost of data movement.
Previous works have optimized these sparse operations individually or addressed one of these challenges. This poster introduces Fused3S, the first fused 3S algorithm that jointly maximizes tensor core utilization and minimizes data movement. Across real-world graph datasets, Fused3S achieves significant speedup over state-of-the-art kernels on H100 and A30 GPUs.
Previous works have optimized these sparse operations individually or addressed one of these challenges. This poster introduces Fused3S, the first fused 3S algorithm that jointly maximizes tensor core utilization and minimizes data movement. Across real-world graph datasets, Fused3S achieves significant speedup over state-of-the-art kernels on H100 and A30 GPUs.
Event Type
Research and ACM SRC Posters
TimeTuesday, 18 November 20258:00am - 5:00pm CST
LocationSecond Floor Atrium
Archive
view