Oracle Advanced Engineering
Responsibilities
Qualifications & Requirements
Experience Level: Senior Level
Full Job Description
EY is seeking a skilled AWS Data Engineer to join our dynamic consulting team in Noida, India. This senior-level role focuses on designing, developing, and managing scalable data pipelines using modern AWS technologies. You will be instrumental in ingesting, processing, and storing data from diverse sources, supporting critical analytics, BI, and data science initiatives. As a key problem-solver, you will diagnose client needs, bridge existing gaps, and architect blueprints for achieving their desired end goals.
Key Responsibilities:
- Design and implement data ingestion pipelines from sources including on-premise Oracle databases, batch files, and Confluent Kafka.
- Develop Python producers and AWS Glue jobs for efficient batch data processing.
- Build and manage robust Spark streaming applications on Amazon EMR.
- Architect and maintain Medallion Architecture-compliant data lakes on Amazon S3.
- Develop and manage data sinks within Redshift and Oracle databases.
- Automate and orchestrate complex workflows using Apache Airflow.
- Monitor, debug, and optimize data pipelines to ensure high performance and reliability.
- Collaborate effectively with cross-functional teams, including data analysts, data scientists, and DevOps engineers.
Required Skills and Experience:
- Proficiency in Python and Spark (Pyspark) programming.
- Hands-on experience with Amazon S3, AWS Glue, and Amazon EMR.
- Strong SQL skills, particularly with Amazon Redshift and Oracle.
- Proven experience in handling streaming data with Kafka and building real-time data pipelines.
- Solid understanding of data modeling, ETL frameworks, and performance tuning best practices.
- Experience with workflow orchestration tools such as Apache Airflow.
Nice-to-Have Skills:
- Experience with Infrastructure as Code using Terraform.
- Familiarity with additional AWS services including SNS, SQS, DynamoDB, DMS, Athena, and Lake Formation.
- Knowledge of DataSync for file movement and Medallion Architecture for data lakes.
- Experience with monitoring and alerting tools like CloudWatch, Datadog, or Splunk.
Qualifications:
- Bachelor's or Master's degree in Engineering (B.Tech/M.Tech), Computer Applications (MCA), or Business Administration (MBA).
Join EY in Noida, UP, India (201301) and contribute to building a better working world.
Company
EY
EY is a global leader in professional services, dedicated to building a better working world. We empower our clients to achieve sustainable growth, transformation, and operational excellence through a...