Senior Data Engineer

NielsenIQ is seeking a Senior Data Engineer to join their Forecasting team in Pune, MH, India. This role is ideal for a passionate data engineer focused on building scalable, production-grade data pipelines to support machine learning and AI initiatives. The successful candidate will be instrumental in solving complex data challenges and ensuring the robustness of data pipelines crucial for predictive modeling.

As a Senior Data Engineer, you will be responsible for the end-to-end design, implementation, and maintenance of data pipelines that power deep neural network models for global sales and revenue forecasting. You will collaborate closely with ML engineers, data scientists, and product stakeholders, leveraging GCP-native services to build automated, secure, and cost-efficient pipelines.

Key responsibilities include:

• Designing and building scalable, maintainable data pipelines for deep learning model training and inference in forecasting and other ML algorithms.
• Developing and orchestrating workflows using GCP tools such as Cloud Workflows, Cloud Run, BigQuery, Cloud Functions, and Pub/Sub.
• Automating and managing the entire pipeline lifecycle, including ETL/ELT processes and model run triggers.
• Monitoring and optimizing pipeline performance, cost, and data freshness.
• Collaborating with data scientists to productionize AI/ML models and integrate them into business workflows.
• Owning data ingestion, validation, and transformation logic for forecasting models, ensuring high-quality input data.
• Managing deployment, versioning, and configuration using Docker, Git, and CI/CD practices.
• Implementing logging, monitoring, and alerting for reliability and traceability in a high-scale environment.
• Conducting code reviews, mentoring junior engineers, and contributing to best practices in the data engineering stack.

Required qualifications:

• 7+ years of experience in data engineering or backend engineering.
• Strong proficiency in Python and SQL, with proven experience in building production data pipelines.
• Solid understanding of Docker, Git, and shell scripting in Linux.
• Hands-on experience with GCP services.
• Experience in building, deploying, and maintaining data workflows for AI/ML models.
• Familiarity with ML pipeline infrastructure and model lifecycle management.
• Proficiency in setting up CI/CD pipelines for data and ML systems (e.g., GitLab CI, Cloud Build).
• Exposure to Java for backend services or pipeline components.
• Proactive, collaborative, and strong communication skills.

Nice to have:

• Experience with forecasting or time series modeling pipelines.
• Familiarity with event-driven architectures.
• Knowledge of infrastructure-as-code tools like Terraform.
• Understanding of data quality frameworks and observability tools.
• Experience with model versioning and experiment tracking systems.

Join NielsenIQ to work on a high-impact platform for global demand forecasting and critical business decisions. Be part of a collaborative and innovative Forecasting & AI team within a leading global data science organization. Benefit from top-tier infrastructure, GCP services, challenging real-world problems, flexible working hours, a remote-friendly culture, and opportunities for professional growth. Competitive compensation and performance-based bonuses are offered, along with benefits like flexible working, volunteer time off, LinkedIn Learning, and an Employee Assistance Program.