TLDR: Spark-based data PaaS solutions are convenient. But they come with their own set of challenges such as a high vendor lock-in and obscured costs. We show how to use a dedicated orchestrator (dagster-pipes). It can not only make Databricks an implementation detail but also save cost. Also, it improves developer productivity. It allows you to take back control.
Sep 2, 2024
May 20, 2024
Jul 16, 2022
Jul 16, 2022
Mar 14, 2022
Figure description: (a) Probability $p(s|c)$ to find a supply link, sij , given that there exists a communication link, cij, between firms i and j for communication links exceeding a given call duration, dij. Error bars denote the quartiles of a bootstrap simulation described in SI Text 1.
Oct 13, 2021
This publication is currently still a preprint under review.
Oct 1, 2021
Aug 21, 2021
Accepted by Nature Scientific Reports. Published now: 10.1038/s41598-021-97394-1
Oct 23, 2020
Management summary Mobility data from mobile phones are used to investigate mobility changes during Covid- 19 lock-down measures in Austria. Here we focus on relative changes between different sub-groups of the population by using the tools from compositional data analysis. This allows to gain much deeper insight into differences between the groups, and how these differences evolved over time.
Sep 4, 2020