Data Engineer Lead/Data Warehouse A...

Data Engineer Lead / Data Warehouse Architect - Lansing, MI

We are seeking an experienced Level 3 Data Engineering Lead to join our team as the primary State of Michigan technology lead. This role involves engaging with University of Michigan (UoM) proof-of-concept teams to support best practice adoption within state hosting environments and providing training to DAV data engineers.

Key Responsibilities:

• Oversee the design, development, and maintenance of complex data pipelines for analytics and operational systems using Cloud Service Providers (AWS/Azure/Google).
• Architect and optimize data infrastructure including storage solutions, orchestration frameworks, and ETL/ELT processes within a Lakehouse/Medallion Architecture.
• Ensure high standards of data quality, reliability, and security by implementing validation tools and governance practices to enforce Data Sharing Policies at the point of subscription.
• Lead and mentor junior engineers, providing technical guidance on code reviews, design patterns, and professional growth in distributed architectures (Spark, Kafka, Kubernetes).
• Collaborate with cross-functional partners including data scientists, analysts, and business stakeholders to deliver scalable Data/ML products using Agile DevOps frameworks.

Required Qualifications:

• Education: BS in Computer Science / Data Science / Information Systems required.
• Data Pipeline Experience: Minimum 4 years of hands-on experience hydrating enterprise Data Lakes on CSPs, building Extract/Load pipelines from relational and unstructured sources to Parquet destinations.
• Data Transformation & Governance: Minimum 4 years transforming datasets in a Lakehouse with traceability features for Critical Data Elements. Experience enforcing data sharing policies based on user subscription entitlements is essential.
• Programming Proficiency: Deep expertise in Python, PySpark, SQL (with tuning capabilities), R, and Terraform scripting. Mastery of DAGs, YAML, Docker, and Git/Jenkins workflows.
• Distributed Systems: Proven track record with Big Data frameworks including Apache Spark, Kafka, Kubernetes, Databricks, SageMaker, DBT, NoSQL/Graph databases, and visualization tools like Power BI/Tableau.