Presentation
Mastering AI Workflows Using ACCESS Pegasus
DescriptionThis tutorial is designed for both users and facilitators who want to deepen their understanding of modeling AI pipelines in a portable, reproducible way using scientific workflows and application containers. Scientific workflows are essential for managing complex computations: they define the dependencies between steps in data analysis and simulation pipelines, automate execution, and capture provenance information critical for verifying results and ensuring reproducibility. Workflows also promote sharing and reuse. Participants will learn to use Pegasus, a leading scientific workflow management system now integrated into the ACCESS Support offerings (https://support.access-ci.org/pegasus). ACCESS Pegasus provides a fully hosted environment built on Open OnDemand and Jupyter, enabling users to develop and run workflows directly from a web browser. Workflow execution is powered by HTCondor Annex, allowing jobs to run across multiple ACCESS resources, including PSC Bridges-2, SDSC Expanse, Purdue Anvil, NCSA Delta, and IU Jetstream2. Through hands-on exercises in a hosted Jupyter Notebook, participants will work through an example LLM-RAG (large language model retrieval-augmented generation) workflow that leverages GPUs across ACCESS resources. Along the way, the tutorial will address key challenges and best practices across the entire workflow life cycle.
Note for Attendees
The participants will be expected to bring in their own laptops with a web browser. We assume familiarity with working in a Linux environment. The laptops should be able to connect to the internet over WI-FI. Attendees will use a hosted Jupyter notebook environment, and thus only a web browser is required.
As part of this tutorial, attendees will learn how to compose workflows using ACCESS Pegasus (http://pegasus.access-ci.org). In order to login and do the hands-on exercises, you will need to have an ACCESS account. An ACCESS account is required for the hands-on portion of the tutorial. If you already have an account, use that one.
If you have any trouble creating an ACCESS account, please contact organizers at pegasus-support@isi.edu.
As part of this tutorial, attendees will learn how to compose workflows using ACCESS Pegasus (http://pegasus.access-ci.org). In order to login and do the hands-on exercises, you will need to have an ACCESS account. An ACCESS account is required for the hands-on portion of the tutorial. If you already have an account, use that one.
- A new account can be requested here (https://operations.access-ci.org/identity/new-user). We strongly recommend going with the Option 1 “Register with an existing identity”.
- Note that the account does not have to be part of an allocation. Having an account is sufficient.
- ACCESS is enforcing to have an institutional email address associated with the account, and matching the listed institution. We recommend you sign up for ACCESS with your institution ID, and your institution email address.
If you have any trouble creating an ACCESS account, please contact organizers at pegasus-support@isi.edu.
Event Type
Tutorial
TimeMonday, 17 November 20258:30am - 12:00pm CST
Location126
Livestreamed
Recorded