https://georgheiler.com/tags/data-science/Data-ScienceHugoBlox Kit (https://hugoblox.com)en-usFri, 29 May 2026 18:00:00 +0000https://georgheiler.com/media/icon_hu_2b4e40339850646.pnghttps://georgheiler.com/tags/data-science/https://georgheiler.com/event/vdsg-optimizing-multimodal-ai-pipelines-with-metaxy/Fri, 29 May 2026 18:00:00 +0000https://georgheiler.com/event/vdsg-optimizing-multimodal-ai-pipelines-with-metaxy/<h2 id="abstract">Abstract</h2> <p>The AI era has changed the economics of data pipelines. Multimodal workflows often fan out into transcription, image understanding, embeddings, classification, extraction, review, and downstream analytics. Without precise metadata, a small change can invalidate too much of the pipeline and force costly reruns.</p> <p>This talk introduces , an open source Python framework for sample-level metadata versioning and field-level provenance. Metaxy acts as a control layer for incremental data pipelines: it records which fields depend on which upstream fields, computes what became stale, and lets the execution layer process only the affected records.</p> <p>We focus on practical examples across startup, enterprise, and research settings:</p> <ul> <li>avoiding wasteful recomputation in multimodal AI workflows,</li> <li>using field-level lineage to decide what can be skipped,</li> <li>keeping provenance queryable across document, audio, image, and tabular data,</li> <li>connecting Metaxy with orchestrators and compute engines such as Dagster, Ray, and Slurm.</li> </ul> <p>The core idea is simple: if an audio file changes, recompute transcription. Do not rerun face recognition if it only depends on the video stream.</p> <h2 id="links">Links</h2> <ul> <li>Deck: </li> <li>Event: </li> <li>Metaxy docs: </li> </ul>