Senior Data Engineer

Senior Data Engineer – Databricks & Fabric

We are seeking an experienced Senior Data Engineer with deep expertise in Databricks, Microsoft Fabric, SQL, and Spark to join our team. This role is pivotal for designing, automating, and governing enterprise data platforms that ensure reliability, accuracy, and security.

Key Responsibilities:

• Pipeline Engineering: Design scalable solutions using Databricks, Fabric, SQL, Spark, and Delta Lake to ingest, transform, and orchestrate enterprise data.
  
• Automation & QA: Build frameworks for validation, testing, monitoring, and deployment. Establish automated reconciliation processes.

• Data Modeling: Construct semantic models, lakehouses, warehouses, and data marts within the Microsoft Fabric ecosystem.

• Governance & Security: Implement Row-Level Security (RLS), governance standards, and CI/CD pipelines using Azure DevOps or GitHub.
  

Required Skills:

You will need 5+ years of experience with Databricks Workflows, Apache Spark (PySpark), ETL/ELT solutions, and Lakehouse Architectures. Proficiency in performance tuning for Spark workloads is essential.

Preferred Qualifications:

• Experience with Infrastructure as Code (Terraform).

• Knowledge of Power BI semantic models and DAX.
  
• Familiarity with DataOps/MLOps processes and cloud platforms like Azure, AWS, or GCP.

About the Team

Join Datavail's elite team of over 1,000 professionals. Our experts provide end-to-end data services—from initial consulting to ongoing management—across leading technologies. We align technical solutions with your business objectives, ensuring project success at every stage.