Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
What To Support When You’re Compressing: The State of Practice, Gaps, and Opportunities for Scientific Data Compression
DescriptionOver the last nearly 20 years, lossy compression has become an essential aspect of HPC applications' data pipelines, allowing them to overcome limitations in storage capacity and bandwidth and, in some cases, increase computational throughput and capacity. However, with the adoption of lossy compression comes the requirement to assess and control the impact lossy compression has on scientific outcomes.
In this work, we take a major step forward in describing the state of practice and characterizing workloads. We examine applications' needs and compressors' capabilities across nine different supercomputing application domains. We present 25 takeaways that provide best practices for applications, operational impacts for facilities achieving compressed data, and gaps in application needs not addressed by production compressors that point towards opportunities for future compression research.
In this work, we take a major step forward in describing the state of practice and characterizing workloads. We examine applications' needs and compressors' capabilities across nine different supercomputing application domains. We present 25 takeaways that provide best practices for applications, operational impacts for facilities achieving compressed data, and gaps in application needs not addressed by production compressors that point towards opportunities for future compression research.
Authors
Event Type
Paper
TimeThursday, 20 November 20251:30pm - 1:52pm CST
Location261-262-265-266
Algorithms
Applications
State of the Practice
Livestreamed
Recorded
TP