Azure Databricks with Fabric Engine...

We are seeking a highly skilled Data Engineer with 6 to 9 years of experience, specializing in Databricks, Python, Microsoft Fabric, Azure Data Stack, and SQL. The ideal candidate will possess extensive experience in automating data pipelines and managing complex data integration scenarios using Fabric/Databricks. This role requires a proactive individual committed to delivering top-quality work and comprehensive documentation. Key responsibilities include developing and maintaining scalable data pipelines and integrations to accommodate increasing data volumes and complexity. You will leverage Databricks, Python, and SQL for ETL processes in intricate scenarios. Collaboration with stakeholders and agile teams to understand and fulfill complex data requirements is essential. The role demands a strong attention to detail for accurate and clear documentation, and the implementation of robust measures for data accuracy, quality, and accessibility. Monitoring data performance, identifying bottlenecks, and optimizing infrastructure are crucial. This position operates within a fast-paced agile team environment, encouraging initiative to solve problems and enhance systems. Essential technical skills include Microsoft Fabric, Azure Data Factory, Azure Databricks, Azure Functions, Azure DevOps (including CI/CD pipelines), a good understanding of Infrastructure as Code, data modeling, and SQL.