Data Engineer

Senior Data Engineer – Databricks & Fabric Engineering

We are seeking an experienced Senior Data Engineer to lead strategic data modernization initiatives for multiple clients. This role focuses on designing, building, and governing enterprise data platforms using Databricks, Microsoft Fabric, SQL, Spark, and Delta Lake.

Key Responsibilities:

• Pipeline Engineering: Design scalable solutions for ingestion, transformation, and orchestration across the full lifecycle.
• Automation & QA: Build automated frameworks for validation, testing, monitoring, deployment, and reconciliation of enterprise datasets.
• Micrsosoft Fabric Expertise: Develop semantic models, lakehouses, warehouses, data marts, and implement Row-Level Security (RLS).
• Data Quality & Governance: Ensure business rule compliance, metadata management, and performance tuning of Spark workloads.
• DevOps Integration: Implement CI/CD pipelines using Azure DevOps or GitHub with Infrastructure-as-Code practices.

Required Skills (5+ Years):

• Databricks & Apache Spark (PySpark)
• Microsoft Fabric Components (Data Factory, Lakehouse, Warehouse, Semantic Models)
• Data Warehousing and ETL/ELT Solutions
• Azure DevOps / GitHub for CI/CD

Preferred Skills:

• Terraform, Bicep, ARM Templates (IaC)
• Power BI semantic models and DAX
• DataOps and MLOps processes