AIML - Data Scientist, Evaluation
Full Job Description
Role Overview
We are seeking an AIML - Data Scientist, Evaluation based in Seattle, Washington, United States, to drive critical product impact by measuring and evaluating features used by millions of users globally. This pivotal role partners closely with engineering teams across the organization to enhance search quality for products like Siri and Apple Intelligence.
Key Responsibilities
- Pioneering research and development of robust evaluation methods to continuously improve user-facing product quality, specifically targeting challenges in Large Language Model (LLM) assessment such as building LLM judges and prompt engineering strategies.
- Collaborating with experimentation teams to translate methodological advancements into scalable technologies for daily engineering use.
- Navigating large-scale complex data sets to solve difficult, non-routine analysis problems using advanced statistical methods.
- Executing comprehensive analyses covering data collection, quality control, requirements specification, processing workflows, and high-impact presentations.
- Iteratively building and prototyping sophisticated analysis pipelines that deliver insights at scale for product development teams.
Mandatory Qualifications
This position requires a PhD in Statistics, Mathematics, Econometrics, Economics, Operations Research, Computer Science, or related field with 1+ years of data science experience; OR a Master's Degree in the same fields with 3+ years of relevant experience. Candidates must possess strong proficiency in statistical data analysis and machine learning.
Technical & Soft Skills
- Demonstrable expertise in crafting, conducting, analyzing experiments focused on data quality, evaluation metrics, and risk assessment (especially regarding human annotation variability).
- Strong programming capabilities including SQL/Spark for querying and Python/R/Scala for scripting development.
- The ability to articulate complex business questions into statistical solutions that drive actionable answers.
Company
Apple
Apple Inc., headquartered in Cupertino, California with major operations globally including a significant presence in Seattle, Washington, is a world-leading innovator and designer of consumer electro...