Data Engineer

Job Title: Data Engineer

Location: Hyderabad / Secunderabad, Telangana, Chennai, Pune

As a Data Engineer at Comviva Technology, you will play a crucial role in building and maintaining robust data systems. Your responsibilities will include analyzing and organizing raw data, developing and managing data pipelines using tools like Apache Nifi, and evaluating business needs to align data strategies. You will interpret complex trends and patterns, conduct in-depth data analysis, and report on findings. A key part of this role involves preparing data for advanced analytical models, including prescriptive and predictive modeling, and building algorithms and prototypes. You will be responsible for integrating information from diverse sources, enhancing data quality and reliability using technologies such as Spark, and orchestrating data workflows with Airflow for py-spark jobs. Furthermore, you will identify opportunities for data acquisition, develop analytical tools and programs, and collaborate closely with data scientists and architects on various projects.

Key Responsibilities:

• Analyze and organize raw data from various sources.
• Build and maintain scalable data systems and pipelines, utilizing Apache Nifi.
• Assess and translate business needs and objectives into data solutions.
• Interpret complex data trends and patterns to derive actionable insights.
• Conduct advanced data analysis and clearly report on results.
• Prepare and refine data for prescriptive and predictive modeling applications.
• Develop and implement algorithms and data prototypes.
• Combine and synthesize information from disparate data sources.
• Continuously explore and implement methods to enhance data quality and reliability, leveraging Spark.
• Utilize Airflow for scheduling and executing py-spark jobs.
• Identify strategic opportunities for data acquisition to enrich datasets.
• Develop and maintain analytical tools and programs to support data initiatives.
• Collaborate effectively with data scientists and architects on diverse projects.

Required Skills and Experience:

• 2-3 years of hands-on experience as a Data Engineer.
• Demonstrated technical expertise in data models, data mining, and segmentation techniques.
• Proficiency in programming languages such as Java and Python.
• Solid experience with SQL database design and implementation.
• Strong numerical and analytical problem-solving skills.
• Bachelor's Degree in Computer Science, Information Technology, or a related field. A Master's degree is considered a plus.
• A Data Engineering certification (e.g., IBM Certified Data Engineer) is advantageous but not required.