Acuity Analytics•1h ago
Foundit
Associate
Bengaluru / Bangalore, India
Permanent job
Mid Level
Full Job Description
About the Role:
We are seeking an Associate Data Engineer to join our multi-year strategic transformation program. You will be instrumental in modernizing global middle- and back-office technology platforms by building scalable, cloud-native, event-driven architectures using AWS, Azure, Databricks, Kafka-based streaming, and API-led integration.
Key Responsibilities:
Develop and optimize data pipelines on the cloud; design batch and real-time ingestion frameworks; build streaming solutions with Structured Streaming; integrate diverse data sources via platforms like AWS MSK or Confluent Kafka; develop ETL/ELT processes for enterprise data platforms; participate in data modeling, quality validation, and metadata management; collaborate across business, technology, and architecture teams to translate requirements into scalable cloud solutions;
Support migration initiatives across AWS and Azure environments; monitor, troubleshoot, and optimize pipelines for performance and reliability; contribute to code reviews, testing, deployment, and production support; help establish engineering best practices.
Key Competencies:
Bachelor's degree in CS, Engineering, or related field; 2–5 years of experience in data engineering or technical roles; hands-on Databricks development (PySpark/SQL); cloud platform expertise (AWS/Azure); experience with data lakes and distributed computing; understanding of real-time streaming concepts including Structured Streaming;
Knowledge of Apache Kafka, AWS MSK, or Confluent Kafka; proficiency with Git and SQL; strong analytical and communication skills.
Preferred Qualifications:
Experience with Delta Lake/Lakehouse architectures; knowledge of event-driven architecture patterns; familiarity with CI/CD pipelines, DevOps practices, Terraform, CloudFormation (Infrastructure as Code); domain experience in financial services or capital markets is highly desirable.
We are seeking an Associate Data Engineer to join our multi-year strategic transformation program. You will be instrumental in modernizing global middle- and back-office technology platforms by building scalable, cloud-native, event-driven architectures using AWS, Azure, Databricks, Kafka-based streaming, and API-led integration.
Key Responsibilities:
Develop and optimize data pipelines on the cloud; design batch and real-time ingestion frameworks; build streaming solutions with Structured Streaming; integrate diverse data sources via platforms like AWS MSK or Confluent Kafka; develop ETL/ELT processes for enterprise data platforms; participate in data modeling, quality validation, and metadata management; collaborate across business, technology, and architecture teams to translate requirements into scalable cloud solutions;
Support migration initiatives across AWS and Azure environments; monitor, troubleshoot, and optimize pipelines for performance and reliability; contribute to code reviews, testing, deployment, and production support; help establish engineering best practices.
Key Competencies:
Bachelor's degree in CS, Engineering, or related field; 2–5 years of experience in data engineering or technical roles; hands-on Databricks development (PySpark/SQL); cloud platform expertise (AWS/Azure); experience with data lakes and distributed computing; understanding of real-time streaming concepts including Structured Streaming;
Knowledge of Apache Kafka, AWS MSK, or Confluent Kafka; proficiency with Git and SQL; strong analytical and communication skills.
Preferred Qualifications:
Experience with Delta Lake/Lakehouse architectures; knowledge of event-driven architecture patterns; familiarity with CI/CD pipelines, DevOps practices, Terraform, CloudFormation (Infrastructure as Code); domain experience in financial services or capital markets is highly desirable.
Company
Acuity Analytics
Acuity Analytics is a leading technology partner specializing in data analytics and engineering solutions, primarily serving financial services institutions like SMBC.
Bengaluru / Bangalore, India
Posted on Foundit