Databricks Data Engineer

Databricks Data Engineer

---

Join Infosys' Data & Analytics practice to modernize enterprise data landscapes through cloud-native architectures and AI-ready ecosystems. We are seeking a Databricks expert with 2-4 years of experience to lead Lakehouse implementations, ETL/ELT pipeline development, and real-time analytics projects.

Key Responsibilities:

• Design and maintain scalable data pipelines using Databricks and Apache Spark for large-scale structured/unstructured datasets.
• Build robust ingestion frameworks integrating databases, APIs, applications, and cloud storage (Azure/AWS/GCP).
• Implement Lakehouse architecture leveraging Delta Lake to ensure ACID transactions on open table formats.
• Optimize Spark jobs and Databricks workloads for peak performance, scalability, and cost efficiency using PySpark/SQL.
• Collaborate with BI teams to deliver reusable data assets and implement governance controls.
• Troubleshoot production issues and participate in architecture/code reviews to promote best practices.

Requirements:

Strong hands-on experience with Databricks, Delta Lake, DataFrames, and Spark architecture. Proficiency in SQL query optimization. Familiarity with Git-based source code management is essential.