Lead Software Engineer

Role: Lead Software Engineer

About the Role

We are seeking an experienced Python Developer specializing in data engineering to design and develop robust file processing solutions on AWS. This role focuses on building scalable ETL pipelines, automation frameworks, and event-driven architectures.

Key Responsibilities

• Design and build inbound/outbound file processing workflows using AWS Glue, Python, and PySpark.
• Create reusable modules for data ingestion, validation, transformation, and error handling across Amazon S3 environments.
• Configure event-driven architectures using Amazon EventBridge rules for scheduling and automation.
• Develop lightweight serverless functions with AWS Lambda (Python) to integrate diverse systems.
• Implement messaging solutions leveraging Amazon SNS and SQS.
• Set up operational monitoring, logging, alerting, and troubleshooting using Amazon CloudWatch.
• Orchestrate complex workflows with AWS Glue Workflows or Step Functions.

Required Skills & Experience (3-8 Years)

This mid-to-senior level role requires strong hands-on experience in:

• AWS Data Services: Deep expertise with AWS Glue, PySpark, S3 lifecycle policies/partitioning, and CloudWatch.
• Languages & Tools: Advanced Python (OOP, exception handling), SQL, Git, JSON/XML/CSV processing.
• Databases: Experience with Oracle, PostgreSQL, MySQL, or Amazon RDS via REST/SOAP APIs and direct connectivity.
• Solutions Design: Building ETL pipelines, error-handling frameworks, and asynchronous notification systems.

If you are passionate about building scalable data platforms on AWS in Hyderabad and beyond, we want to hear from you!