Data Engineer

Position Summary

We are seeking a Data Engineer with hands-on expertise in Azure Databricks, Azure Data Factory and Azure Data Lake Storage. The role will build scalable, governed data pipelines and Lakehouse data products for BI, analytics, machine learning and AI-enabled self-service.

• Data Pipelines & Integration: Design batch/streaming pipelines to integrate diverse data into the Databricks Lakehouse.
• Lakehouse Data Products: Build reusable datasets using Delta Lake for analytics and AI use cases.
• Databricks Genie Enablement: Configure agents using trusted datasets and business terminology.
• Semantic Models & Metrics: Translate KPIs into consistent dimensions and measures.
• Data Quality & Observability: Implement automated quality checks and monitor pipeline health.
• Governance & Security: Use Unity Catalog for metadata, lineage, and role-based access control.
• Performance & Cost: Optimize Spark workloads and clusters for scalability and efficiency.
• DataOps & Support: Apply Git testing, CI/CD practices, and incident management.

Required Skills

• Bachelor's degree in CS or related field with 2-4 years of experience.
• Hands-on expertise: Azure Databricks, Delta Lake, Data Factory, Storage.
• Languages: Advanced SQL, Python, PySpark.