Senior Data Engineer
Responsibilities
Qualifications & Requirements
Experience Level: Senior Level
Full Job Description
Join NielsenIQ (NIQ) in Chennai, TN, as a Senior Data Engineer and contribute to building technology that empowers global businesses. You will be part of the Connect Technology team, working on our unified, global, open data ecosystem powered by Microsoft Azure. Our clients worldwide depend on Connect data and insights for innovation and growth.
As a Senior Data Engineer, you will collaborate with a team of skilled technologists passionate about cutting-edge technologies including Spark, Scala, Pyspark, Databricks, Airflow, SQL, Docker, and Kubernetes. These tools are deployed using DevOps pipelines on Azure, Kubernetes, Jenkins, and Bitbucket/GIT Hub.
Key Responsibilities:
- Develop, test, troubleshoot, debug, and enhance applications using Spark, Pyspark, Scala, Pandas, Databricks, Airflow, and SQL.
- Deploy application components via CI/CD pipelines.
- Build utilities for monitoring and automating repetitive tasks.
- Collaborate with Agile cross-functional teams, including internal and external clients, Operations, Infrastructure, and Tech Ops.
- Partner with the Data Science team to productionize ML Models.
- Participate in a rotational support schedule to address customer queries and deploy timely bug fixes.
Qualifications:
- 8-10 years of relevant software engineering experience.
- Strong fundamentals in Big Data technologies such as Spark, Pyspark, Scala, Pandas, Databricks, Airflow, and SQL.
- Proven experience with cloud technologies, preferably Microsoft Azure.
- Experience in performance optimization of Spark workloads.
- Experience with DevOps technologies like GIT Hub, Kubernetes, Jenkins, and Docker is a plus.
- Knowledge of relational databases, preferably PostgreSQL, is desirable.
- Excellent English communication skills for effective collaboration with cross-functional technology and business teams.
- Minimum B.S. degree in Computer Science, Computer Engineering, or a related field.
Benefits include a flexible working environment, volunteer time off, LinkedIn Learning, and an Employee Assistance Program (EAP).
Company
NielsenIQ
NIQ is a global leader in consumer intelligence, providing comprehensive insights into consumer buying behavior to drive growth. In 2023, NIQ merged with GfK, uniting two industry leaders with extensi...