Data Engineer

About This Role

As a Data Engineer I at GoKwik, you will build and maintain data pipelines that power analytics, machine learning models, and business-critical applications. You will work with modern technologies including Apache Iceberg, Trino, Spark, and Kubernetes.

What You'll Own

• Data Pipeline Development: Design end-to-end pipelines from ingestion to production-ready silver/gold data models using SQL, Python, and PySpark. Implement incremental processing and MERGE operations.
• Data Quality & Reliability: Build validation rules, reconcile transformed data against sources, and create monitoring mechanisms for pipeline resilience.
• Performance Optimisation: Analyse queries to improve efficiency in partitioning, layout, and resource utilisation across large-scale datasets.
• Production Support: Investigate issues, conduct root-cause analysis, and collaborate with Engineering and Data Science teams on code reviews and testing.

Data Platform Engineering

Contribute to orchestration, infrastructure (Kubernetes/GitOps), real-time serving using Apache Pinot/ClickHouse, and data governance. Use AI tools effectively while validating outputs for correctness.