AI/ML Engineer Data

Role Overview:

We are seeking an experienced AI/ML Engineer focused on Data Engineering, Model Evaluation, and Continuous Improvement at SecNinjaz Technologies LLP. This role is central to building our next-generation VAPT product by ensuring data quality, optimizing model performance through fine-tuning (LoRA/QLoRA), and rigorously evaluating agent interactions.

Key Responsibilities:

• Data Pipeline Development: Engineer robust systems that transform raw agent traces, tool outputs, and expert feedback into structured, versioned datasets suitable for training high-performance models.
• Evaluation & Quality Control: Design comprehensive evaluation frameworks to measure task completion rates, evidence quality, false positives, latency, and compute costs. Implement rigorous data cleaning, deduplication workflows, and annotation processes while preventing data leakage across dataset splits.
• Fine-Tuning & Optimization: Execute reproducible supervised fine-tuning experiments using modern techniques like PEFT (Parameter-Efficient Fine Tuning). Collaborate with security specialists to align model outputs with domain-specific cybersecurity requirements.
• RAG & Retrieval Enhancement: Optimize embedding models, rerankers, and retrieval pipelines to ensure agents access the most relevant information for accurate vulnerability assessment.
• Failure Analysis: Conduct root cause analysis on system failures—whether stemming from prompts, data issues, or model limitations—and propose evidence-based improvements that boost performance on unseen cases.

Technical Requirements:

• Proficiency in Python for building complex ML pipelines and data processing workflows.
• Demonstrable experience with PyTorch and the Hugging Face ecosystem.
• Solid understanding of LLM fundamentals, including training vs. inference dynamics, overfitting prevention, generalization strategies, and loss function optimization.

Why Join Us:

• Gain hands-on experience with enterprise-grade NVIDIA H200 GPU infrastructure hosted on-premises in Delhi for large-scale model training and inference tasks.
• Collaborate directly with cybersecurity experts to translate domain knowledge into high-quality, secure datasets.
• Take ownership of the full improvement lifecycle—from identifying critical failures to deploying validated solutions that drive product success.