Presentation
SIGN IN TO VIEW THIS PRESENTATION Sign In
ModuLair: Streamlining Python Virtual Environment Management for HPC
DescriptionManaging Python environments on high-performance computing (HPC) systems presents unique challenges due to complex toolchains, file system constraints, and diverse user needs. We present ModuLair, a modular, metadata-driven Python virtual environment framework designed to simplify environment creation, activation, and management in HPC contexts. ModuLair supports both EasyBuild and non-EasyBuild module systems, automatically detecting explicit specification of toolchains to ensure reproducibility and compatibility across workflows. The framework integrates seamlessly with command-line and graphical interfaces, including the improved User Dashboard, Job Composer, and JupyterLab, enabling visual, intuitive environment management for both novice and experienced users.
We validate ModuLair through usage metrics collected over five months across three HPC clusters, demonstrating sustained adoption by active Python users and integration into ongoing research workflows in both CLI and GUI contexts. These results show that ModuLair reduces setup complexity, lowers the barrier to entry, and promotes best practices in environment configuration and job submission.
We validate ModuLair through usage metrics collected over five months across three HPC clusters, demonstrating sustained adoption by active Python users and integration into ongoing research workflows in both CLI and GUI contexts. These results show that ModuLair reduces setup complexity, lowers the barrier to entry, and promotes best practices in environment configuration and job submission.
