NTT DATA is seeking a skilled Data Engineer Advisor in Bengaluru, India, to lead the design and execution of SAP data migration strategies to Azure Databricks. This role involves leveraging tools such as Qlik Replicate, Azure Data Factory (ADF), and Airflow to facilitate seamless data transfer and transformation.
Key Responsibilities
- Design and implement comprehensive migration plans for SAP data into Azure Databricks environments.
- Convert existing SAP BW data models and ETL jobs into efficient PySpark workflows on Databricks.
- Develop and maintain data layers (Bronze, Silver, Gold) adhering to the Medallion Architecture for structured data ingestion and transformation.
- Ensure precise mapping of SAP tables and business objects to Databricks Delta tables.
- Implement robust data quality checks, lineage tracking, and governance mechanisms using Unity Catalog.
- Optimize ingestion pipelines for large SAP datasets to enhance latency and throughput.
- Perform performance tuning for queries and data transformations to ensure efficiency.
- Collaborate closely with SAP Subject Matter Experts (SMEs), data architects, and business analysts to validate requirements.
- Produce detailed documentation for migrated jobs and provide essential support during User Acceptance Testing (UAT) and hypercare phases.
- Ensure adherence to enterprise security policies and manage Role-Based Access Control (RBAC) for Databricks workspaces.
Required Skills and Experience
- Extensive knowledge of SAP BW, ECC, and related modules.
- Proficiency in Databricks, PySpark, and Delta Lake technologies.
- Familiarity with Azure Data Lake Storage (ADLS).
- Experience with orchestration tools such as Airflow.
- Hands-on experience with ETL/ELT tools including Qlik Replicate, Informatica, or similar.
- Understanding of cloud data governance frameworks, including Unity Catalog and RBAC.
This is a full-time opportunity for an experienced professional to make a significant impact on critical data migration projects.
