Sr IT Data Engineer

About The Role:

We are seeking a highly skilled Senior Data Engineer to join our dynamic team at the Hyderabad office. In this role, you will be instrumental in accelerating the publication of data to our core Azure data lake, ensuring that data pipelines are robust, efficient, and aligned with organizational standards. Your primary responsibility will be to work with established cloud data platforms and internal tooling to build, enhance, and maintain data pipelines that support analytics and decision-making processes. The role emphasizes executing and refining existing data patterns and standards to deliver high-quality, governed data for various business needs. You will collaborate closely with data asset managers, architects, and development leads to ensure technical solutions meet both business and technical requirements. This position offers an exciting opportunity to work on large-scale data projects, optimize performance, and contribute to the organization’s data-driven culture.

Qualifications:

The ideal candidate will possess strong technical skills and relevant certifications, including proficiency in Python, SQL, Delta Lake, Databricks, Spark/PySpark, Github, and Azure cloud services. Attainment or maintenance of certifications such as Databricks and Azure certifications is expected. Candidates should have experience implementing CI/CD pipelines using tools like Github Actions, SonarQube, and Snyk. A solid understanding of distributed computing concepts, big data processing, and data engineering platforms is essential. Additional qualifications include a good grasp of data platform capabilities, experience working outside of P&G or D&A environments, and the ability to adapt to evolving technologies. Strong problem-solving skills, collaboration abilities, and a proactive approach to learning are highly valued.

Responsibilities:

• Productionize and maintain data pipelines for large, complex datasets that meet both technical and business requirements.
• Collaborate with data asset managers, architects, and development teams to design and implement robust technical solutions.
• Adhere to and contribute to coding standards and best practices to ensure efficiency, robustness, and reusability of pipelines and components.
• Identify opportunities for process improvements and implement innovative solutions to optimize data workflows.
• Optimize Spark jobs for performance and cost, leveraging advanced techniques such as broadcast joins, caching, and partitioning.
• Ensure data quality, reliability, and performance through data validation, monitoring, and proactive troubleshooting.
• Monitor data pipelines and workflows, troubleshooting issues to ensure seamless operation and minimal downtime.
• Stay informed of the latest features, tools, and industry trends in Databricks and data engineering to continuously enhance practices.
• Apply a strong understanding of distributed computing and big data processing to solve complex data challenges.
• Mentor and coach junior engineers, fostering a collaborative and knowledge-sharing environment.

Benefits:

Procter & Gamble offers a comprehensive benefits package designed to support the well-being and professional growth of our employees. You will receive continuous mentorship and training opportunities, including formal programs and day-to-day guidance from experienced managers. Our work environment promotes agility, work/life balance, and a culture where every individual is valued and encouraged to contribute ideas. Employees have access to health and wellness programs, competitive compensation, and opportunities for career advancement. We also support initiatives that foster diversity and inclusion, ensuring a welcoming and respectful workplace for all.

Equal Opportunity:

Procter & Gamble is an equal-opportunity employer committed to creating a diverse and inclusive workplace. We do not discriminate based on race, color, gender, age, national origin, religion, sexual orientation, gender identity or expression, marital status, citizenship, disability, HIV/AIDS status, or any other legally protected characteristic. Our hiring process is personalized to ensure fairness and equal opportunity for all candidates. We uphold the highest standards of ethics and corporate responsibility in all our operations, fostering a culture where everyone can succeed and thrive.