Data Engineer
Responsibilities
Qualifications & Requirements
Experience Level: Mid Level
Full Job Description
C1X Inc. is seeking a talented Data Engineer to join our dynamic team in Chennai, India. As a key member of our data engineering group, you will be instrumental in building and maintaining large-scale data products and infrastructure. This role offers the opportunity to shape the future of our data analytics tech stack by leveraging modern big data technologies. You will collaborate closely with business stakeholders, product managers, and engineering teams to address diverse data requirements, driving crucial business insights and product innovation.
Objectives
- Design, construct, and maintain scalable data infrastructure for efficient collection, storage, and processing.
- Ensure straightforward access to reliable data for data scientists, analysts, and business users.
- Facilitate data-driven decision-making and enhance organizational efficiency through the delivery of high-quality data products.
Responsibilities
- Develop and implement large-scale batch and real-time data pipelines utilizing frameworks such as Apache Spark on AWS or GCP.
- Design, manage, and automate data flows across various data sources.
- Implement and uphold best practices for continuous integration, thorough testing, and robust data quality assurance.
- Maintain comprehensive data documentation, definitions, and adherence to governance practices.
- Optimize data systems for performance, scalability, and cost-effectiveness.
- Partner with stakeholders to translate complex business needs into effective data-driven solutions.
Qualifications
- A Bachelor's degree in Computer Science, Engineering, or a related field is preferred; exceptional coding performance on platforms like LeetCode/HackerRank can be considered.
- A minimum of 2 years of experience working on full lifecycle Big Data projects.
- A solid understanding of data structures, algorithms, and software design principles.
- Proficiency in at least two programming languages, with a preference for Python or Scala.
- Experience with key AWS services including EMR, Lambda, S3, DynamoDB (GCP equivalents are also valuable).
- Hands-on experience with Databricks Notebooks and the Jobs API.
- Strong expertise in major big data frameworks such as Spark, MapReduce, Hadoop, Sqoop, Hive, HDFS, Airflow, and Zookeeper.
- Familiarity with containerization technologies like Docker and workflow management tools like Apache Airflow.
- Intermediate to advanced proficiency in SQL, including experience with relational and NoSQL databases (e.g., Postgres, MySQL, Redshift, Redis).
- Demonstrated experience with SQL tuning, schema design, and analytical programming.
- Proficiency in Git for version control and collaborative workflows.
- Adaptability and comfort working with a variety of technologies in a fast-paced, results-oriented environment.
Company
C1X Inc.
C1X Inc. is a rapidly expanding AdTech company specializing in the development of next-generation advertising and marketing technology platforms. Our core mission is to equip enterprise clients with i...