Data Engineer

Shuru seeks a highly skilled Senior Data Engineer to join our dynamic engineering team in Bangalore. This is a hands-on role focused on designing, building, and maintaining robust data pipelines critical for our product, analytics, and machine learning initiatives. You will collaborate with cross-functional teams to ensure data quality, reliability, and accessibility across our platform.

Responsibilities:

• Design and implement scalable data pipelines for ingestion, validation, cleanup, and normalization.
• Construct and manage ETL/ELT processes for both batch and real-time data processing.
• Develop data quality frameworks and monitoring systems to ensure data integrity.
• Optimize pipeline performance and resolve data flow issues.
• Deploy and manage data infrastructure on AWS and GCP platforms.
• Implement Infrastructure as Code (IaC) using Terraform for repeatable deployments.
• Design and maintain data lakes, data warehouses, and streaming architectures.
• Ensure security, compliance, and cost optimization for cloud resources.
• Implement event-driven architectures utilizing technologies such as Kafka, Kinesis, or Pub/Sub.

Requirements:

• Over 5 years of experience in data engineering or a related domain.
• Demonstrated hands-on experience in data pipeline development and maintenance.
• Proficiency in Python, SQL, and at least one other programming language (Java, Scala, Go).
• Extensive experience with AWS services including S3, Redshift, EMR, Kinesis, Lambda, and Glue.
• Solid experience with GCP services such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage.
• Proven track record of using Infrastructure as Code with Terraform.
• Strong background in streaming technologies like Apache Kafka, Amazon Kinesis, or Google Pub/Sub.