TLDR: Spark-based data PaaS solutions are convenient. But they come with their own set of challenges such as a high vendor lock-in and obscured costs. We show how to use a dedicated orchestrator (dagster-pipes). It can not only make Databricks an implementation detail but also save cost. Also, it improves developer productivity. It allows you to take back control.
Sep 2, 2024
May 20, 2024
Feb 8, 2023
Jul 16, 2022
Jul 16, 2022
Find the full analysis here: https://www.csh.ac.at/lockdown-for-unvaccinated-mobility-in-austria/ and here: https://www.csh.ac.at/wp-content/uploads/2021/11/2021-11-26-CSH-Policy-Brief-Mobilitat-Herbst-2021-final.pdf
Nov 26, 2021
This publication is currently still a preprint under review.
Oct 1, 2021
Aug 21, 2021
WWTF COVID project summary
May 12, 2021
Apr 18, 2021