Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
Secure In-Storage Execution of VTK Workloads on Modern Parallel NFS Data Servers
DescriptionAs data volumes grow, the cost of moving large datasets increasingly limits scientific visualization performance. One promising solution is to analyze data where it is stored. This paper presents a pushdown architecture for pNFS-based storage systems that offloads early stages of a VTK pipeline—such as reading and filtering—to the pNFS data servers that hold the data. Using a FUSE-based interface, a pNFS client triggers remote processing and retrieves results by writing and reading special command and result files. Our design leverages pNFS clients' ability to locate file-resident servers, along with a recent Linux enhancement that enables efficient local access to pNFS data without exposing filesystem internals. Offloaded code runs with the user's credentials, preserving standard permission checks. Experiments with two real-world scientific datasets show up to 6.1× speedup in end-to-end visualization runtime and up to 7.1× in data loading, thanks to early data filtering that significantly reduces data movement.
Event Type
Workshop
TimeMonday, 17 November 20253:30pm - 4:00pm CST
Location230
Data Analytics
High Performance I/O, Storage, Archive, & File Systems
Storage
Livestreamed
Recorded
TP
W