SDET - I

Mindtickle is seeking a Software Development Engineer in Test (SDET) I to join our team in Pune, Maharashtra. This role is pivotal in ensuring the quality and reliability of our market-leading revenue productivity platform.

As an SDET, you will be responsible for designing, building, and maintaining scalable automation frameworks to qualify projects before and after release. Your work will directly contribute to consistent product quality across our platform.

Key responsibilities include developing comprehensive automated test suites encompassing unit, integration, regression, and system-level tests. You will also focus on improving CI/CD pipelines to ensure reliable and repeatable releases.

A significant aspect of this role involves working with advanced AI-driven features. You will build scalable testing systems, generate test data, and implement automated regression and synthetic monitoring to ensure ongoing quality. Specifically, you will test automation frameworks for AI features powered by LLMs, including prompt pipelines, retrieval systems, and AI-assisted workflows.

Furthermore, you will develop and maintain evaluation frameworks to measure the quality of AI outputs, focusing on factual accuracy, relevance, bias, and hallucination rates. Robustness, safety, and adversarial testing for AI systems will be integral, involving prompt regression testing, embedding validation, and model performance tracking.

You will also contribute to improving quality engineering practices, such as testability and shift-left approaches, and aim to enhance Developer Experience (DevEx). Collaboration is key, and you will work closely with developers and product managers, participating in design discussions and code reviews to ensure exceptional reliability and performance of all features and workflows.

We are looking for candidates with:

• A minimum of 2 years of experience as a Software Engineer or SDET, with a solid background in automation systems.
• Proficiency in Go, Java, or Python.
• Hands-on experience with LLM APIs and pipelines (e.g., OpenAI, Anthropic, Hugging Face).
• Experience integrating automation into CI/CD pipelines (e.g., GitHub Actions, Jenkins, GitLab CI).
• Strong skills in API testing, version control with Git, and containerization using Docker.
• Familiarity with cloud platforms such as AWS, GCP, or Azure AI.
• Excellent communication and collaboration skills across engineering, ML, and product teams.
• Experience in prompt engineering, embeddings, RAG, and LLM evaluation metrics is a plus.

Preferred qualifications include:

• A Bachelor’s degree in Computer Science, Engineering, or a related field.
• Familiarity with LLMs and GenAI concepts.
• Experience with LLM testing tools like MaximAI, OpenAI Evals, TruLens, LangSmith, or Promptfoo.
• Experience evaluating AI systems for bias, fairness, and AI safety principles.
• A strong analytical mindset and a structured approach to problem-solving.

Join Mindtickle and contribute to shaping the future of revenue productivity. We offer a rewarding work environment with ample opportunities for growth, reflecting our commitment to our customers, each other, and a passion for excellence. We foster a diverse and inclusive culture based on our values: Delight your customers, Act as a Founder, and Better Together.