Lead Data Engineer

As a Lead Data Engineer in Bengaluru, you will lead the design, development, and maintenance of our organization's data architecture and infrastructure. Your role will be pivotal in building a robust data platform capable of efficiently processing large data volumes, ensuring informed business decisions through reliable, high-quality information. We seek a candidate with extensive data engineering experience, strong leadership capabilities, and a proven history of managing complex data projects.

Key Responsibilities:

• Data Architecture & Design: Architect and implement scalable, efficient data solutions aligned with business objectives, collaborating with cross-functional teams.
• ETL Development: Guide the creation of reliable ETL processes for data integration into our data warehouse, enforcing data quality and integrity best practices.
• Big Data Technology: Stay abreast of big data and analytics trends, optimizing technologies like Hadoop, Spark, and Kafka for large-scale data processing and analysis.
• Cloud Integration: Partner with IT infrastructure to integrate data engineering solutions with cloud platforms such as AWS, Azure, or Google Cloud, prioritizing scalability, security, and performance.
• Performance Monitoring & Optimization: Deploy monitoring tools for data pipeline performance and proactively resolve any arising issues.
• Documentation: Maintain thorough documentation for all data engineering processes, data models, and system architectures.
• Collaboration & Communication: Work closely with data scientists, analysts, and stakeholders to understand and fulfill their data requirements. Communicate project progress and challenges effectively to both technical and non-technical audiences.

Qualifications:

• Education: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Experience: 6-8 years of professional data engineering experience.
• Technical Skills:
  • Proficiency in data modeling, ETL processes, and data warehousing.
  • Extensive experience building data warehouses with Snowflake.
  • Hands-on experience with data ingestion, data lakes, data mesh, and data governance.
  • Strong programming skills in Python.
  • Solid understanding of big data technologies including Hadoop, Spark, and Kafka.
  • Experience with cloud platforms like AWS, Azure, or Google Cloud.
  • Familiarity with diverse database systems (SQL, NoSQL) and data pipeline orchestration tools.
• Soft Skills: Exceptional problem-solving and analytical abilities, excellent communication and interpersonal skills, and a demonstrated capacity for collaborative work in a fast-paced, dynamic environment.

This is a permanent position based in Bengaluru.