Real-Time End-to-End Data Streaming with Postgres, Kafka, MinIO, Prometheus, and GrafanaAs data engineering evolves, proficiency in both batch and streaming architectures is increasingly essential. Batch processing handles…Jul 9, 2024Jul 9, 2024
CI/CD and DevOps for Snowflake: A Comprehensive GuideIn the world of data engineering and database management, seamlessly integrating Continuous Integration/Continuous Deployment (CI/CD)…Jun 13, 20243Jun 13, 20243
10 Things I Like About Snowflake — from a SQL Server MigrantAs a data engineer, I’ve spent much of my career working with SQL Server, managing data pipelines and extracting insights within its…May 9, 20241May 9, 20241
Metadata-Driven Development in Azure Data FactoryIn the realm of data engineering, harnessing the power of metadata-driven development is not merely a revolutionary concept — it’s a…Mar 24, 2024Mar 24, 2024
CDC 101 in SQL Server for Data EngineersWhile Change Data Capture (CDC) is a concept not limited to SQL Server, this article will focus on its implementation within the SQL Server…Mar 12, 2024Mar 12, 2024
Displaying Self-Hosted Agents on Classic Release Pipeline (Azure DevOps)Recently, in my role as a data engineer, I undertook the task of converting our Azure SQL database deployment pipeline from YAML to the…Mar 1, 2024Mar 1, 2024
Deploying Python Applications for Free using RenderDeploying Python applications can be a daunting task, especially if you’re on a tight budget. While Heroku is a popular choice for many…Sep 22, 20231Sep 22, 20231
SQL Server — Identifying Empty TablesIn the realm of data engineering, one critical task involves running data pipelines and ensuring their successful execution. A crucial…Sep 21, 2023Sep 21, 2023
