Scaling data pipelines @Telekom

Join the Vienna Data Engineering Meetup on the 9th of April.

Scaling data pipelines @Telekom

Aleks and Georg share how Magenta Telekom is handling their data challenges by turning data dependencies, tooling silos and development from being a problem into a helpful component of Magenta`s data platform. They start by covering the basic principles underpinning the data platform, discuss the current challenges in the domain of data engineering and share how these are solved for Magenta. They finish by sharing how building around the orchestrator - makes it easy to build for scale.

• Aleksandar Milicevic is a Senior Data Engineer at Magenta. Aleks is deeply involved in the data engineering community and worked previously in consulting.
• Georg Heiler is a Senior Data Expert at Magenta. He is a speaker and meetup organizer. After completing his PhD in Complexity Science he is now defining the data architecture of Magenta.

Content

Video recording of the talk:

Post recording discussion about certain frequently asked questions:

Shorts

Tackling the fear of scaling data to the needs of the business

Asset- vs. Task-based orchestration (short)

Asset- vs. Task-based orchestration (long)

Orchestrators and streaming

Impressions from the event

Thank you

This was a very special event tonight for me. Many thanks to all the helping hands! Many thanks especially to Rigerta for organizing the great lineup and to the other speakers. Many thanks to Aleks for co-presenting and all the great support to build what we are talking about - and the whole team who have supported in bringing this to life.

Tonight is special in many ways:

• Firstly, it was the largest Meetup which I have hosted so far - around 170 people on site! It was an incredible crowd. I still remember when we started with the first events many years ago in a small room with limited support (very grassroots)
• Secondly, with the kind support of some Magenta colleagues we had these awesome rockets! I hope they will allow you to bring some of the spirit of tonight`s event into your work places
• Thirdly, these rockets are an important symbol for us @Magenta: Our programme for the data-cloudification is called “project Pluto”. These rockets resemble that we have landed there - finally. As the first data assets have landed in production already recently.

There were so many great talks and discussions tonight! Thank you to everyone who mande this possible. We are already looking forward to a next time.

We Aleks and I hope that the content provided here on this page allows to spread the ideas and concepts further.