Big Data Engineer - Bengaluru

Now100 Solutions Private Limited is looking for a skilled Big Data Engineer in Bengaluru to join their team. This role focuses on the design, development, and maintenance of robust, scalable big data processing pipelines. The ideal candidate will possess strong proficiency in Python and PySpark, alongside practical experience with major cloud platforms such as AWS, Azure, or GCP. A solid understanding of CI/CD processes is essential for automating deployments and workflows.

Key Responsibilities:

• Craft and manage sophisticated data pipelines utilizing Python and PySpark.
• Architect and implement high-performance, scalable big data solutions on leading cloud infrastructures (AWS, Azure, or GCP).
• Develop and oversee CI/CD pipelines to ensure automated deployment and rigorous testing.
• Engage collaboratively with data scientists, analysts, and fellow engineers to construct resilient data infrastructure.
• Continuously optimize data processing workflows to enhance performance and ensure unwavering reliability.

Required Skills:

• Expertise in Python
• Proficiency in PySpark
• Hands-on experience with one or more cloud platforms: AWS, Azure, or GCP
• Familiarity with CI/CD pipelines and associated tools

Job Type: Permanent

Experience Level: 0 - 5 Years