Remote Python Engineer

About the Project

You will build high-quality LLM evaluation and training datasets focused on realistic software engineering problems. Key initiatives include creating verifiable Software Engineering (SWE) tasks based on public repository histories using a synthetic approach enhanced by human-in-the-loop feedback.

Role Responsibilities:

• Analyze and triage GitHub issues across trending open-source libraries to identify challenging scenarios for LLMs.
• Set up, configure, and Dockerize code repositories to ensure consistent development environments.
• Evaluate unit test coverage quality and run local modifications of complex codebases to assess bug-fixing capabilities.
• Collaborate with researchers to design datasets that cover various programming languages and difficulty levels.

Why Join Us?

This unique opportunity allows you to blend practical software engineering with cutting-edge AI research. You will be at the forefront of evaluating LLM interactions with real code, directly influencing the future of AI-assisted development tools.

Day-to-Day Look:

• Navigate complex open-source repositories and modify them locally for testing.
• Led opportunities to manage junior engineers on specific projects.

Requirements

• Experience: Minimum 3+ years of overall software engineering experience (Tech Lead level preferred).
• Tech Stack: Strong proficiency in Python, Git, Docker, and setting up basic software pipelines.
• Skill Set: Ability to understand complex codebases and run/testing real-world projects locally.

Nice-to-Haves

• Previous participation in LLM research or evaluation projects.
• Experience building developer tools, automation agents, or testing them for reliability.

Perks of Freelancing with Turing

• Work from anywhere (Fully Remote).
• Access to cutting-edge AI projects alongside leading LLM companies.

Offer Details & Commitments

Type: Contractor assignment (Note: Does not include medical or paid leave benefits typical of full-time roles).

Duration: 3-month contract with an expected start date next week.

Schedule Options:

• Commitment Required: At least 4 hours/day overlap with PST time zone.
• Average minimum of 20 hours per week. Available options include 20 hrs/week, 30 hrs/week, or a full-time equivalent of 40 hrs/week based on candidate preference and project needs.