Data Engineer

We are seeking a Senior Data Engineer to design, implement, and maintain robust data pipelines. Your responsibilities will include ensuring data quality, accessibility, and security. You will create data models to support business analytics, manage complex ETL processes, and optimize database performance for both SQL and NoSQL systems. This role involves collaborating with cross-functional teams, including data scientists and analysts, to define data needs and schema designs. You will leverage big data technologies such as Hadoop and Spark for processing large datasets and implement real-time data processing solutions. Ensuring adherence to data governance policies and security standards, including implementing access controls and encryption, is critical. This position also requires thorough documentation of data architecture and processes, and delivering data-driven insights through reporting.

• Design and implement efficient data pipelines for collection, processing, and storage.
• Maintain high standards of data quality, integrity, and accessibility.
• Develop data models to facilitate business analytics and reporting.
• Collaborate with stakeholders to understand data requirements and design appropriate schemas.
• Build and manage ETL processes, automating data ingestion and transformation.
• Administer and optimize SQL and NoSQL databases, monitoring performance and resolving issues.
• Integrate data systems with applications in partnership with software engineers.
• Utilize big data tools like Hadoop and Spark for large-scale data processing.
• Implement distributed systems for real-time data analytics.
• Enforce data governance and security by complying with policies and standards.
• Implement data access controls and encryption measures.
• Document data architecture, pipelines, and operational processes.
• Generate reports and provide insights derived from data analysis.