Cost efficient alternative to databricks lock-in
Save money 💰 and increase developer productivity 👩💻👨💻 by limiting scope-creep of Spark-based data PaaS solutions: 🌐 turn them into an implementation detail 🔧.
Dagster, dbt, duckdb as new local MDS
Lean and efficient MDS experience: Delivers better software engineering practices to the data ecosystem with the new local MDS stack comprised of Dagster, dbt and DuckDB which …
Securing Secrets with Mozilla SopS and AGE: A Powerful Combo
🔐 Exploring the power duo of Mozilla's sops & AGE for secret management! Dive deep into their benefits: simplicity, version control compatibility & robust encryption. Secure your …
Unlocking Advanced Metadata Extraction with the New DBT API in Dagster
📊 Unleash the power of metadata extraction in your data engineering pipelines with the new DBT API in Dagster! 🚀 Learn how to seamlessly integrate and leverage DBT transformations, …
Making BigData small again (and green)
Towards simpler and perhaps more energy efficient data platforms with increased developer productivity.
Comparing SQL-based streaming approaches
Comparing established and up-and-coming streaming approaches for an integrated real-time data model
SFTP sensor
Way too many data pipelines still work with SFTP file transfer. Even a modern data orchestrator needs to interface here well.
Connector goodness from Airbyte E2E lineage
Simplify data ingestion with the plentiful connectors of Airbyte without compromising on data lineage
Scalable data pipelines from dagster with pyspark
Getting started with simple dagster pipelines.
Tame your notebooks
Include jupyter notebooks into reliable data pipelines.