Zorba Consulting India Pvt. Ltd.
Zorba Consulting India Pvt. Ltd.1h ago
Naukri

Senior Data Engineer

Kolkata
Full Time
Senior Level

Auto Apply to 50+ AI Matched Senior Data Engineer Jobs

Use Auto Apply Agents to Bulk Apply jobs with ATS Optimised Resumes, find verified Insider Connections for jobs at Zorba Consulting India Pvt. Ltd.

Full Job Description

About the Role

We are seeking a Senior Data Engineer to spearhead the development of our cutting-edge Data Lakehouse architecture. This pivotal role involves designing, building, and optimizing massive-scale, low-latency data pipelines essential for powering real-time analytics and advanced Machine Learning applications. Join our dynamic team in Kolkata and shape the future of data at Zorba Consulting India Pvt. Ltd.

Key Responsibilities

  • Design and implement highly optimized, production-grade ETL/ELT pipelines utilizing Apache Spark (PySpark/Scala) for processing petabytes of data.
  • Architect and manage a robust Data Lakehouse environment leveraging open-source technologies such as Delta Lake or Apache Hudi to ensure ACID transactions and maintain high data quality standards.
  • Integrate and process real-time data streams efficiently using technologies like Apache Kafka or Kinesis.
  • Develop and deploy automated data quality checks, comprehensive monitoring systems, and lineage tracking mechanisms across all data products.
  • Collaborate closely with the infrastructure team to automate the deployment and scaling of our data platform on the cloud (AWS EMR/Glue or Databricks) using Infrastructure as Code principles with Terraform.
  • Optimize data warehousing and querying performance within leading platforms such as Snowflake or Google BigQuery.

Technical Skills Required

  • Expert proficiency and performance tuning experience with Apache Spark (PySpark or Scala).
  • Mandatory hands-on experience with Data Lakehouse technologies (Delta Lake, Iceberg, or Hudi).
  • Strong experience with at least one major public cloud data platform (AWS, GCP, or Azure).
  • Solid understanding of data modeling techniques (Dimensional, Data Vault) and advanced SQL capabilities.
  • Demonstrated experience with workflow orchestration tools like Apache Airflow or Prefect.

Company

Zorba Consulting India Pvt. Ltd.

Zorba Consulting India Pvt. Ltd.

Kolkata
Posted on Naukri
Senior Data Engineer at Zorba Consulting India Pvt. Ltd. | Kolkata | Apply Now | MindMyJob | MindMyJob - AI Job Search Platform