Senior Data Engineer

About Inchcape

At Inchcape, our mission is to connect the world and help our customers trade with confidence across every port globally. We embrace change, value local agent expertise, and are committed to positive industry evolution through technology and innovation.

Main Purpose of Job

The Senior Data Engineer will design, build, and maintain data ingestion and transformation pipelines that populate Inchcape's central platforms using Microsoft Fabric Lakehouse (OneLake) with Delta Lake/Parquet. You will curate bronze and silver layers in our medallion architecture, ensuring high-quality datasets for modeling, analysis, reporting, and business intelligence.

Key Accountabilities

• Develop ETL/ELT pipelines to ingest data from ERP, CRM, financial, operational, and third-party sources into Microsoft Fabric using Delta Lake.
• Implement robust data quality controls, incremental loading strategies (CDC), and error handling mechanisms.
• Create Bronze tables for raw ingestion, Silver tables for curated/cleansed data, and Gold tables for business-ready assets with full lineage documentation.
• Optimize performance through partitioning, clustering, and tuning of SQL/Spark workloads on MS Fabric to reduce operational costs while maintaining NFR requirements (resilience, availability, security).
• Collaborate with Data Architects, Modellers, and BAs to design scalable schemas and integration patterns.
• Build reusable automation scripts using Fabric Dataflows Gen2 or Spark notebooks.
• Maintain metadata and sensitivity labels via Purview ensuring compliance.

Key Deliverables

• Automated, monitored ingestion pipelines with alerting capabilities.
• High-quality conforming data assets ready for self-service analysis.
• Performance benchmarks demonstrating efficient resource usage.
• Templates and best practice guides to enable team knowledge sharing.

Essential Requirements

- 8+ years of experience in Data Engineering/Integration with SQL, Python, or Scala.
- Proven expertise with ETL/ELT patterns (CDC, incremental loading) using Microsoft Fabric Lakehouse architecture.
- Strong understanding of data governance frameworks, version control (CI/CD), and observability tools.

Desirable Skills

• Streaming data experience (Kafka, Event Hubs).
• Mastery Data Management and Governance practices.
- Shipping or Maritime industry domain knowledge.

Educational & Professional Qualifications

Bachelor's degree in Computer Science, Engineering, or a related field. Microsoft Certified Azure Data Engineer Associate certification is mandatory. AWS data platform certifications are highly desirable.