Dagster-slurm: Bringing Modern Data Orchestration to Slurm-Managed
dagster-slurm is an open-source Python integration that allows data scientists and research software engineers to run Dagster pipeline assets on both a laptop and Slurm-managed HPC …
Discovering the SUPER in computing - dagster-slurm for reproducible research on HPC
Dagster is a modern data orchestrator that emphasises reproducibility, observability, and a strong developer experience. In parallel, most high-performance computing (HPC) centres …
Metaxy + Dagster-Slurm for Efficient Multimodal Pipelines
How to combine Metaxy, Dagster-Slurm, Docling, and Ray to run incremental multimodal pipelines on sovereign AI infrastructure.
Dagster + Slurm
Run and orchestrate Dagster workloads on Slurm-based HPC environments with production-oriented patterns.
•
1 min read
Rediscovering the SUPER in Supercomputing
Improving developer experience on HPC systems with dagster.