Data Engineer

Data Engineer with Google Cloud Platform (GCP) Expertise - Gurgaon

Deutsche Telekom Digital Labs seeks a Data Engineer with 2-4 years of experience to design, build, and manage scalable data pipelines and platforms on Google Cloud Platform (GCP). This role is crucial for enabling advanced analytics and deriving business insights from large datasets.

Key Responsibilities:

• Architect, develop, and maintain robust data pipelines for efficient data ingestion, transformation, and processing.
• Leverage GCP services including BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
• Create and optimize ETL/ELT processes for diverse structured and unstructured data sources.
• Uphold high standards of data quality, integrity, and reliability throughout the data ecosystem.
• Collaborate closely with data scientists, analysts, and product teams to deliver high-quality datasets for analytics and machine learning initiatives.
• Implement and manage data warehousing solutions, primarily using BigQuery.
• Proactively monitor and resolve issues within data pipelines and workflows.
• Optimize data processing job performance and cost-effectiveness on GCP.
• Ensure comprehensive documentation of data pipelines, workflows, and system architecture.

Essential Requirements:

• 2-4 years of hands-on experience in Data Engineering or a similar technical role.
• Demonstrated expertise in Google Cloud Platform (GCP) services such as BigQuery, Dataflow, Dataproc, Pub/Sub, and Cloud Storage.
• Strong proficiency in Python or SQL for data manipulation and pipeline development.
• Experience with ETL/ELT methodologies and data pipeline orchestration.
• Solid understanding of data warehousing principles and dimensional modelling.
• Familiarity with workflow orchestration tools like Airflow or Cloud Composer.
• Proficiency in version control systems, particularly Git.
• Excellent analytical and problem-solving capabilities.
• Experience with streaming data pipelines is a significant advantage.
• Exposure to CI/CD practices and DevOps principles is beneficial.
• Knowledge of data governance, security, and compliance frameworks is a plus.
• Experience working within Agile development methodologies.