Dagster + Slurm
Run and orchestrate Dagster workloads on Slurm-based HPC environments with production-oriented patterns.
A selection of projects spanning data engineering, geospatial analytics, and applied machine learning.
Run and orchestrate Dagster workloads on Slurm-based HPC environments with production-oriented patterns.
Open ELO-style ranking of analytics and OLAP engines based on public benchmark evidence.
Metaxy is a next-generation data processing engine and framework focused on scalable, high-performance analytics pipelines.
Conda forge offers effortless installation of various well tested python packages. I am a maintainer of H3 and H3-py on conda-forge.
Reverse engineering old data pipelines ; ) and analyzing huge quantities of spatial data.
Individual cost based classification model outperforms classical processes.