Data Engineer

Role Overview

We are seeking a highly skilled Data Engineer (8+ years experience) to design, build, and optimize next-generation data architecture and pipelines. This remote position requires expert-level proficiency in PySpark and SQL for scalable ETL/ELT processes.

Key Responsibilities:

• Pipeline Development: Architect fault-tolerant ETL/ELT pipelines from diverse sources.
• PySpark Expertise: Optimize large-scale data transformation jobs using Spark DataFrame API.
• SQL Mastery: Advanced querying, manipulation, and performance tuning across Snowflake, Redshift, PostgreSQL.
• Architecture & Automation: Design dimensional models (Snowflake Schema) and implement orchestration tools like Airflow or ADF.
• Quality & Collaboration: Implement validation routines and work with Data Scientists/Analysts.

Required Skills:

• PySpark: Expert-level development of distributed systems applications.
• SQL: Mastery of window functions, complex joins, and tuning on major platforms.
• Programming & Big Data: Strong Python/Pandas proficiency with understanding of big data architecture.

Preferred Qualifications:

• Azure cloud platform expertise (Data Factory, Synapse Analytics).
• Experience with Apache Airflow and streaming technologies like Kafka/Spark Structured Streaming.