Presentation
Why Read It All? Just Read What Matters: A Path to Faster Scientific Visualization
DescriptionAs HPC simulations generate ever-larger datasets, reducing the volume of data that must be loaded into compute node memory for analysis has become essential for unlocking insights efficiently. In-storage analysis achieves this by processing data directly at the storage servers, allowing them to return only compact results that match regions of interest instead of raw datasets that may be orders of magnitude larger—significantly reducing data footprint.
This e-poster showcases a novel compute-near-storage architecture based on pNFS that enables secure in-storage analysis of scientific data with industry-standard software, including Arrow, Parquet, Substrait, and DuckDB. Using a real-world asteroid impact dataset, we present a live demonstration of a VTK visualization pipeline modified to offload analysis to pNFS data servers, tracing the aftermath of the impact over time and rendering the results on the client as 3D visuals. We show substantial data reduction by pushing down analysis and transmitting only insight-relevant information.
This e-poster showcases a novel compute-near-storage architecture based on pNFS that enables secure in-storage analysis of scientific data with industry-standard software, including Arrow, Parquet, Substrait, and DuckDB. Using a real-world asteroid impact dataset, we present a live demonstration of a VTK visualization pipeline modified to offload analysis to pNFS data servers, tracing the aftermath of the impact over time and rendering the results on the client as 3D visuals. We show substantial data reduction by pushing down analysis and transmitting only insight-relevant information.
Event Type
Doctoral Showcase
Interactive Research e-Poster
TimeThursday, 20 November 202510:00am - 5:00pm CST
LocationSecond Floor Atrium
Archive
view