Bringing robust DevOps practices to cloud data platforms is crucial for delivering scalable, reliable solutions. In this post, I’ll walk through how we implemented a streamlined CI/CD pipeline for Azure Synapse Analytics using Azure DevOps. This setup enables our team to safely promote changes from development to production…
In the previous post, we explored the foundational difference between SMP and MPP systems, and why MPP is better suited for large-scale analytics. Now, let’s dive into how Azure Synapse Analytics builds on that MPP architecture to become one of the most powerful and flexible cloud data platforms available…
Notes from the edge of data engineering — pipelines, platforms, and the occasional Proxmox reboot. For a while now, I’ve wanted a space to share the things I’ve been learning — and breaking — as I work through the realities of building and maintaining modern data systems. The ETL Dispatch is…
Intro Following a dreadful stint in retail and food service, one of my first “real” jobs was manually keying addresses for the United States Postal Service. Today, I’m a data engineer working with Azure Synapse, ADF, dbt, and Snowflake — building scalable cloud pipelines and leading technical projects. But my…
When working with data at scale, one of the most important concepts to understand is the distinction between SMP (Symmetric Multiprocessing) and MPP (Massively Parallel Processing). These two architectures form the backbone of how modern data systems are designed and perform—and they're especially important when evaluating platforms…
As data engineers, we often find ourselves navigating a rapidly evolving ecosystem of tools and platforms. One platform that continues to gain traction in the cloud data space is Azure Synapse Analytics. It's powerful, flexible, and packed with features—but it also has a learning curve. That’s…
I’ve been using ChatGPT for just about everything lately. Although I spun up the infrastructure to host this blog months ago, it’s unlikely I would have done anything with it without ChatGPT’s help. For some tasks, it’s invaluable. For others... it’s borderline useless (and no,…