AI/ML Reviewer

Join Crossing Hurdles, a leading referral partner for top AI research labs, as an AI/ML Reviewer. This hourly contract role is fully remote and based in India, offering 40 hours per week with flexible scheduling. You will play a crucial role in ensuring the quality and rigor of cutting-edge AI models.

Responsibilities:

• Verify the correctness and consistency of reinforcement learning (RL) environments, including terminal conditions, to align with task objectives.
• Evaluate AI benchmarking pipelines for accuracy, fairness, reproducibility, and adherence to strict experimental standards.
• Provide detailed technical feedback on the design of RL environments, evaluation protocols, and implementation specifics.
• Analyze Python codebases to assess environment behavior, termination logic, and metric calculations.
• Collaborate with AI research and engineering teams to refine evaluation methodologies and benchmarking criteria.
• Validate the reproducibility of AI models by testing performance across various runs, seeds, and hardware setups.
• Document findings comprehensively and suggest improvements to enhance benchmarking reliability and system robustness.

Requirements:

• A solid background in reinforcement learning, computer science, or applied AI research.
• Proven experience with RL environments, understanding their implementation, especially regarding terminal conditions and dynamics.
• Strong knowledge of benchmarking methodologies, evaluation metrics, and experimental protocols in RL.
• Proficiency in Python, with the ability to review and understand code (experience with PyTorch/TensorFlow is a bonus).
• Exceptional critical thinking and analytical skills to identify inconsistencies and implementation flaws.
• A meticulous, detail-oriented approach with a commitment to fairness, accuracy, and reproducibility in AI research.

Key Areas of Focus:

• Reinforcement Learning: Expertise in environment design, termination logic, and reward structures.
• Benchmarking & Evaluation: Experience in reproducibility testing, fairness assessments, and metric development.
• Agentic AI Systems: Familiarity with evaluation protocols for agentic reasoning and behavior.
• Python Code Review: Skills in reviewing environment scripts, evaluation pipelines, and simulation frameworks.
• Experimental Rigor: Understanding of seed management, cross-hardware validation, and experimental stability in AI research.

Application Instructions:

Submit your application for this AI/ML Reviewer position. Our recruitment team will typically send an official message or email within 1-2 days.