Presentation
A Formal Characterization of Non-Monotonicity in Tensor Cores
DescriptionModern high performance computing increasingly relies on hardware accelerators like NVIDIA Tensor Cores, which employ non-standard internal arithmetic that can evolve between hardware generations. This non-standard approach can violate the fundamental mathematical property of monotonicity, leading to incorrect outputs where adding a larger number produces a smaller result. To address this, we introduce a formal framework using satisfiability modulo theories to analyze the hardware design space by systematically varying hardware features (e.g., number of terms [𝑛], internal padding bits [𝑝]) within a custom bitvector encoding. We derive a precise condition for guaranteed monotonicity, proving that non-monotonicity can only occur when 𝑝 ≤ ⌊log2 (𝑛 − 1) − 2⌋. We also derive a formula for the maximum magnitude of error when non-monotonicity can occur. Our results provide hardware architects with provably correct design parameters to eliminate such anomalies, ensuring greater numerical stability.
Event Type
Best Poster Presentations (Research, ACM SRC Grad/Undergrad)
TimeWednesday, 19 November 20252:00pm - 2:15pm CST
Location230
Archive
view