Engineering Projects
A collection of data engineering and cloud architecture projects spanning Microsoft Fabric, Azure, SQL, PySpark, and enterprise integration — each built to solve real-world data challenges at scale.
6
Projects
5+
Years Exp.
Azure
Cloud
Unified Lakehouse Analytics Platform
End-to-end data platform on Microsoft Fabric
Designed and implemented a fully unified Lakehouse architecture on Microsoft Fabric, consolidating disparate data sources into a single OneLake. The platform ingests structured and semi-structured data from REST APIs, SQL databases, and flat files, transforming them through medallion layers (Bronze → Silver → Gold) using PySpark Notebooks and Dataflows Gen2.
Architecture
Medallion Lakehouse (Bronze / Silver / Gold) on OneLake. Ingestion via Data Factory pipelines, transformation in Spark Notebooks, semantic layer exposed through Power BI Direct Lake mode.
Enterprise ETL Orchestration Framework
Metadata-driven pipeline orchestration on ADF
Built a reusable, metadata-driven ETL framework on Azure Data Factory that dynamically generates and executes pipelines based on configuration stored in Azure SQL. The framework supports 50+ source-to-target mappings, handles full and incremental loads, and includes built-in retry logic, alerting, and audit logging.
Architecture
Control table pattern in Azure SQL drives ADF ForEach loops. Linked services parameterized for multi-environment deployment. Azure Monitor + Logic Apps handle alerting. All pipeline runs logged to a central audit table.
Real-Time Event Stream Processing
Streaming analytics with Databricks & Eventstream
Developed a real-time event processing solution using Microsoft Fabric Eventstream and Azure Databricks Structured Streaming. The pipeline ingests high-volume IoT and transactional events, applies windowed aggregations and anomaly detection logic, and writes enriched results to Delta tables for downstream BI consumption.
Architecture
Eventstream captures events from Azure Event Hubs. Databricks Structured Streaming reads from Kafka-compatible endpoint, applies sliding window aggregations and ML-based anomaly scoring, writes to Delta Live Tables. Power BI streams from Gold Delta layer.
Enterprise Data Warehouse Modernization
Dimensional modeling & SQL optimization at scale
Led the modernization of a legacy on-premise data warehouse to Azure Synapse Analytics. Redesigned the dimensional model using Kimball methodology, migrated 200+ stored procedures to optimized T-SQL, implemented columnstore indexes, and introduced a dbt-based transformation layer for version-controlled, testable SQL transformations.
Architecture
Azure Synapse dedicated SQL pool as the warehouse layer. dbt models define transformation logic with full lineage. Staging → ODS → DWH → Data Mart layers. Azure DevOps CI/CD deploys dbt models on merge to main.
Microsoft Purview Data Catalog Implementation
Enterprise data governance & lineage tracking
Implemented Microsoft Purview as the enterprise data governance layer across Azure Data Lake, Synapse, and Fabric workspaces. Configured automated scanning, classification policies, sensitivity labels, and end-to-end data lineage tracking. Built a custom data dictionary integrated with Purview APIs for business glossary management.
Architecture
Purview scans Azure Data Lake Gen2, Synapse, and Fabric Lakehouses on a scheduled basis. Custom classification rules tag PII and sensitive data. Lineage captured from ADF and Fabric pipelines. REST API integration surfaces catalog data in internal data portal.
SAP-to-Azure Data Integration Pipeline
Enterprise SAP ABAP extraction to Azure cloud
Engineered a robust SAP-to-Azure data integration pipeline extracting data from SAP ECC using custom ABAP programs (WRICEF) and RFC-enabled function modules. Data lands in Azure Data Lake via ADF SAP connectors, undergoes transformation through PySpark notebooks, and feeds downstream analytics and reporting workloads.
Architecture
SAP ABAP extractors push delta records to staging tables. ADF SAP Table connector pulls via RFC. Raw data lands in ADLS Gen2 Bronze zone. PySpark notebooks cleanse and conform data to Silver layer. Synapse SQL serves reporting layer.
Interested in working together?
I'm open to data engineering consulting, architecture reviews, and full-time opportunities in Microsoft Fabric and Azure data platforms.