Data Engineer II

Join Baazi Games as a Data Engineer II in Bawana, Delhi, India, and play a pivotal role in constructing and refining sophisticated, scalable data pipelines and lakehouse systems. These systems are crucial for delivering analytics and product insights across the organization. You will actively engage with our AWS-based technology stack, develop resilient data workflows, and uphold stringent standards for data quality, performance, and reliability.

Responsibilities

• Design and optimize scalable data pipelines and lakehouse components, leveraging technologies like Apache Iceberg or Hudi.
• Develop robust ETL/ELT workflows within the AWS ecosystem, utilizing services such as AWS Glue, EMR, Lambda, Redshift, and other relevant platform components.
• Write high-quality, modular, and reusable code in PySpark, Python, and SQL.
• Manage and enhance orchestration workflows using Airflow to ensure operational reliability and scalability.
• Collaborate effectively with analytics, product, and engineering teams to establish and maintain unified and consistent data models.
• Engage in performance tuning, cost optimization initiatives, and continuous improvements of the AWS data infrastructure.
• Implement and adhere to best practices for data quality, data cataloging, and metadata management.
• Actively participate in code reviews and engineering discussions to maintain elevated technical standards.

Requirements

• Possess 2-4 years of dedicated experience in data engineering, with substantial exposure to large-scale data systems.
• Demonstrate 2+ years of hands-on experience with PySpark.
• Exhibit a solid understanding of the AWS data ecosystem, including services like AWS Glue, EMR, S3, Lambda, Redshift, and CloudWatch.
• Have practical experience working with data lakehouse technologies such as Apache Iceberg or Hudi, with a preference for Iceberg.
• Showcase strong programming proficiency in Python and PySpark, complemented by a solid command of SQL.
• Familiarity with using Apache Airflow for scheduling and orchestrating data workflows.
• Understand core principles of distributed systems, data modeling, and data governance.
• Exposure to containerized environments like Kubernetes and Docker is considered a valuable asset.
• Exhibit the ability to collaborate closely with both business and technical stakeholders to deliver scalable and effective data solutions.