Senior Data Engineer

Deutsche Telekom Digital Labs is seeking a Senior Data Engineer with 4-7 years of experience to design, build, and manage scalable data pipelines and platforms on Google Cloud Platform (GCP). This role requires hands-on expertise in data engineering, cloud-based data processing, and handling large datasets to deliver critical analytics and business insights for our digital products in Gurgaon.

Responsibilities:

• Design, build, and maintain scalable data ingestion, transformation, and processing pipelines.
• Utilize GCP services including BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
• Develop and optimize ETL/ELT pipelines for diverse data sources.
• Ensure high data quality, integrity, and reliability.
• Collaborate with data scientists, analysts, and product teams to provide datasets for analytics and machine learning.
• Implement data warehousing solutions using BigQuery.
• Monitor and troubleshoot data pipelines for operational efficiency.
• Optimize performance and cost-effectiveness of GCP data processing.
• Maintain comprehensive documentation for data pipelines and architecture.

Requirements:

• 4-7 years of experience in Data Engineering.
• Strong proficiency in Google Cloud Platform (GCP) services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage).
• Expertise in Python or SQL for data processing and pipeline development.
• Experience with ETL/ELT frameworks and orchestration tools.
• Solid understanding of data warehousing and dimensional modeling.
• Experience with Apache Spark, Beam, or Hadoop is advantageous.
• Familiarity with workflow orchestration tools like Airflow or Cloud Composer.
• Proficiency with version control systems (Git).
• Strong analytical and problem-solving capabilities.
• Experience with streaming data pipelines.
• Exposure to CI/CD pipelines and DevOps practices.
• Knowledge of data governance, security, and compliance.
• Experience in Agile development environments.