Data Engineer
Responsibilities
Qualifications & Requirements
Experience Level: Senior Level
Full Job Description
Join NielsenIQ's Connect Technology team in Chennai, TN, India, to work on the new Connect platform, an open, global data ecosystem powered by Microsoft Azure. As a Senior Data Engineer, you will develop, test, troubleshoot, and enhance applications using core technologies like Spark, Pyspark, Scala, Pandas, Databricks, Airflow, and SQL. You will deploy application components via CI/CD pipelines, build monitoring and automation utilities, and collaborate with Agile cross-functional teams, including Data Science for ML model productionization. This role involves participation in a rotational support schedule for customer queries and bug fixes.
Key technologies include Spark, Pyspark, Scala, Databricks, Airflow, SQL, Docker, and Kubernetes, deployed using Azure, Kubernetes, Jenkins, and Bitbucket/GIT Hub.
We are looking for candidates with 4-8 years of software engineering experience, strong fundamentals in Big Data technologies (Spark, Pyspark, Scala, Pandas, Databricks, Airflow, SQL), and experience with cloud technologies, particularly Microsoft Azure. Experience in performance optimization of Spark workloads is essential. Familiarity with DevOps technologies like GIT Hub, Kubernetes, Jenkins, and Docker, as well as relational databases like PostgreSQL, is advantageous. Excellent English communication skills and a minimum B.S. degree in Computer Science, Computer Engineering, or a related field are required.
NIQ offers a flexible and rewarding work environment with opportunities for career development, wellness plans, and financial wellness tools.
Company
NielsenIQ
NIQ, a world leader in consumer intelligence, provides a comprehensive understanding of consumer buying behavior and identifies new growth opportunities. In 2023, NIQ merged with GfK, uniting two indu...