Data Engineer

About the Role

As a Senior Data Engineer, you will be responsible for developing, optimizing, and maintaining business intelligence and data warehouse systems. Your work ensures secure, efficient data storage/retrieval, enables self-service exploration, and supports stakeholders with insightful reporting.

Key Accountabilities:

• Data Pipeline: Develop scalable pipelines and new API integrations to handle increasing data volume/complexity.
• Data Integration: Connect offline and online data sources to improve customer behavior understanding for personalization.
• Data Pre-processing & Quality Management: Collect, parse, manage, analyze large datasets; cleanse data and drive governance strategies.
• Data Transformation: Use ETL/ELT processes to transform data into proper storage structures for querying and analysis.
• Data Enablement: Ensure enterprise-wide accessibility of usable data operations.

Qualifications & Requirements:

• Bachelor's/Master's degree in Engineering, CS, Math, or Statistics (Master's/PhD preferred).
• 4-7 years of relevant experience required; Stakeholder management is an advantage.
• Technical Skills:
• Languages: Python/Pyspark/SAS fluency.
• Distributed Systems: Strong experience with Hadoop, Hive, Spark optimization (Databricks/Delta Lake preferred).
• Cloud Platform: Azure expertise required (ADF, ADLS Storage, DevOps).
• DevOps/Tools: Docker, CI/CD, Kubernetes, Terraform, Octopus.
• Databases & ETL: Advanced SQL, Data Modeling; Experience with Ab Initio, Informatica, or DataStage.
• Good to have: Power BI experience, Cloud migration projects, and relevant certifications (Cloud/Data Engineering).