melbdataguyReal-Time End-to-End Data Streaming with Postgres, Kafka, MinIO, Prometheus, and GrafanaAs data engineering evolves, proficiency in both batch and streaming architectures is increasingly essential. Batch processing handles…Jul 9Jul 9
melbdataguyCI/CD and DevOps for Snowflake: A Comprehensive GuideIn the world of data engineering and database management, seamlessly integrating Continuous Integration/Continuous Deployment (CI/CD)…Jun 133Jun 133
melbdataguy10 Things I Like About Snowflake — from a SQL Server MigrantAs a data engineer, I’ve spent much of my career working with SQL Server, managing data pipelines and extracting insights within its…May 91May 91
melbdataguyMetadata-Driven Development in Azure Data FactoryIn the realm of data engineering, harnessing the power of metadata-driven development is not merely a revolutionary concept — it’s a…Mar 24Mar 24
melbdataguyCDC 101 in SQL Server for Data EngineersWhile Change Data Capture (CDC) is a concept not limited to SQL Server, this article will focus on its implementation within the SQL Server…Mar 12Mar 12
melbdataguyDisplaying Self-Hosted Agents on Classic Release Pipeline (Azure DevOps)Recently, in my role as a data engineer, I undertook the task of converting our Azure SQL database deployment pipeline from YAML to the…Mar 1Mar 1
melbdataguyDeploying Python Applications for Free using RenderDeploying Python applications can be a daunting task, especially if you’re on a tight budget. While Heroku is a popular choice for many…Sep 22, 20231Sep 22, 20231
melbdataguySQL Server — Identifying Empty TablesIn the realm of data engineering, one critical task involves running data pipelines and ensuring their successful execution. A crucial…Sep 21, 2023Sep 21, 2023
