Senior Data Engineer

Sr. Data Engineer Position

We are seeking a skilled and experienced Senior Data Engineer with strong expertise in the Azure Data Engineering ecosystem to join our team in Noida.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on Microsoft Azure.
• Develop and manage ETL/ELT pipelines using Azure Data Factory (ADF).
• Work extensively with Azure Data Lake Storage (ADLS) and Azure Databricks.
• Implement batch and real-time data ingestion and processing solutions.
• Develop data transformation solutions using PySpark / Spark / Scala.
• Work with Delta Lake, Spark Structured Streaming, Azure Synapse Analytics, SQL Data Warehouse, Cosmos DB, and other cloud services.
• Create optimized data models and automation workflows using Python and Shell scripting.
• Leverage Azure DevOps for CI/CD pipelines and source control management.
• Troubleshoot performance issues to ensure high-quality data solutions.

Required Skills

Candidate must have at least 2.5 years of experience in Data Engineering with hands-on proficiency in:

• Azure Data Factory, Azure Databricks, and Synapse Analytics.
• Ecosystem knowledge: ADLS, Delta Lake, Spark Streaming/Structured Streaming.
• Programming languages: Python (primary), Scala, SQL.
• Data processing concepts: Batch vs Real-time ingestion on Cloud Infrastructure.

Nice-to-Have Skills

Candidates with Azure certifications or direct client interaction experience are preferred. Knowledge of scalable cloud-based data architecture is a plus.