Data Engineer
Full Job Description
Join ShipTech Analytics (STA) at Amazon Robotics as a Data Engineer and play a pivotal role in constructing the data and analytics infrastructure that underpins Amazon's expansive global transportation network. You will contribute to our ambitious vision of developing an intelligent, autonomous, and highly scalable performance management and decision support system throughout Amazon Transportation. As a Data Engineer I, you will be instrumental in building standardized data solutions that empower operations teams across the globe. This involves developing both near-real-time (NRT) streaming and batch processing pipelines, delivering essential insights for strategic decision-making. Furthermore, you will create data tools that facilitate self-service analytics and automate operational tasks, while also contributing to innovative AI-driven solutions that provide automated insights across the entire transportation network.
Key Responsibilities
- Design and implement scalable near-real-time (NRT) streaming and batch data pipelines crucial for Amazon's global transportation network.
- Develop data products and tools to optimize the complete data lifecycle, encompassing data ingestion, processing, backfill, storage, vending, and operational support modules.
- Collaborate closely with stakeholders in operations and analytics teams to understand their needs, architect solutions, and deliver datasets and data tools that enable faster, more informed decision-making.
- Ensure data quality and implement improvements for datasets that drive operational excellence and enhance customer experience.
- Partner with cross-functional teams to standardize analytics capabilities and build AI-powered automation.
- Utilize AWS cloud technologies to transform raw data into actionable metrics.
- Champion continuous improvement through rigorous code reviews, insightful design discussions, adherence to operational best practices, and compliance with security standards.
A Day in the Life
You will work collaboratively, starting with stakeholder needs, clarifying ambiguities, and building robust data products and tools. Throughout your day, you will be involved in writing code for data pipelines and developing tools that manage ingestion, processing, storage, and vending, working alongside software engineers. Some days will involve engaging in design discussions for data infrastructure, pipelines, and tools. Beyond coding, you will be responsible for documenting datasets, tools, and runbooks, and actively participating in code reviews. During on-call rotations, you will monitor pipeline health. The dynamic nature of this role is a key attraction, offering opportunities to partner with stakeholders, TPMs, and SDEs in building scalable data pipelines and tools that directly influence the operational efficiency of Amazon's transportation network.
About The Team
ShipTech Analytics serves as the foundational data and analytics backbone for Amazon's worldwide transportation network. This team is dedicated to constructing standardized data solutions that streamline analytics and empower operations teams globally with faster decision-making capabilities. Team members engage in collaborative efforts with cross-functional partners to deliver impactful data products and tools. The team is actively modernizing systems, building advanced near-real-time architectures, and developing AI-powered automation. Opportunities exist for team members to mentor colleagues, prototype innovative ideas, and shape the future of data analytics within Amazon. The team embraces a culture of cross-geographical collaboration, with members in multiple locations working synergistically to support global operations.
Basic Qualifications
- A minimum of 1 year of experience in data engineering.
- Proficiency in SQL.
- Experience in data modeling, data warehousing, and building ETL pipelines.
- Familiarity with at least one query language, such as SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, or Scala.
- Experience with at least one scripting language, including Python or KornShell.
Preferred Qualifications
- Experience with big data technologies like Hadoop, Hive, Spark, or EMR.
- Experience with ETL tools such as Informatica, ODI, SSIS, BODI, or Datastage.
Company
Amazon Robotics
Amazon Robotics is a leading innovator in the field of robotics and automation, driving advancements that power critical operations for Amazon. We are dedicated to creating intelligent, autonomous, an...