Data Engineer

We are seeking a skilled Data Engineer to join our growing team in Chennai, India. In this role, you will be instrumental in designing, building, and maintaining robust data platforms that drive commercial value for our clients.

Responsibilities

• Design and implement scalable data pipelines using ETL/ELT tools and methodologies.
• Configure and manage diverse data storage solutions, including relational databases, data warehouses, and data lakes.
• Develop and implement comprehensive data quality checks and monitoring processes to ensure data integrity.
• Automate data platform deployments and operational tasks through scripting and DevOps best practices, utilizing tools like Git and CI/CD pipelines.
• Ensure adherence to data governance and security standards throughout the data platform development lifecycle.
• Effectively troubleshoot and resolve data platform issues.
• Collaborate closely with Data Architects to understand requirements and translate them into technical design specifications.
• Contribute to data modeling and optimization efforts.
• Work with business stakeholders to understand their needs and translate them into effective technical solutions.
• Document data platform architecture, processes, and best practices.
• Continuously stay abreast of the latest advancements in full-stack development, data engineering, and DevOps.
• Proactively identify and suggest improvements and innovations for our data platforms.

Technical Specifications

• Minimum of 2 years of experience in building data platforms.
• Proficiency in multi-cloud data warehousing solutions such as Snowflake, Redshift, Databricks, Fabric, AWS Glue, Azure Data Factory, Synapse, Matillion, or DBT.
• Strong programming skills in SQL, Apache Spark, and Python.
• Experience with containerization technologies like Docker and Kubernetes is essential.
• Familiarity with CI/CD pipelines and DevOps methodologies is required.
• Experience in implementing or working with data governance frameworks and practices to ensure data integrity and regulatory compliance.
• Knowledge of data quality tools and practices.
• Desirable skills include data visualization using Power BI, Tableau, or Looker, and familiarity with full-stack technologies.
• The ability to work independently, adapt to changing priorities, and learn new technologies quickly is crucial.

Required Skillset:

• ETL or ELT: AWS Glue / Azure Data Factory / Synapse / Matillion / dbt.
• Data Warehousing: Azure SQL Server / Redshift / Big Query / Databricks / Snowflake / Fabric (at least one is mandatory).
• Data Visualization: Looker, Power BI, Tableau.
• SQL and Apache Spark / Python programming languages.
• Containerization technologies (e.g., Docker, Kubernetes).
• Cloud Experience: AWS / Azure / GCP.
• Scripting and DevOps tools (e.g., Git, CI/CD pipeline).