LLMOps Engineer / AI Gateway Specia...

About the Role: At Networth Corp, we build the AI Gateway and LLMOps layer enabling enterprises to deploy GenAI and agents at scale — one gateway, one policy, one bill, with real observability. You will collaborate closely with our AI Red Team & Assurance leads to ensure runtime enforcement aligns with client policies. The Challenge: Every team currently ships their own LLM prototypes without a shared gateway, cost visibility, or guardrails, leading security teams to default to 'no'. The enterprise's context leaks every third call. Your Impact: Make GenAI production a routine event rather than a launch initiative. What You'll Do: - Stand up AI Gateway layers (Databricks AI Gateway, Portkey, LiteLLM, Kong AI Gateway, Azure API Management) with unified authentication, rate limiting, and cost attribution per team. - Deploy and operate LLM serving infrastructure (vLLM, TGI, Ray Serve, Databricks Model Serving, Bedrock) featuring canary rollouts, autoscaling, and traffic mirroring. - Configure gateway guardrails including PII redaction, prompt injection defense, jailbreak detection, output validation, and policy enforcement using tools like Guardrails.ai, NVIDIA NeMo Guardrails, Lakera, Protect AI, and OWASP LLM Top 10. - Enable agent platforms with governed tool-use, memory management, sandboxed execution, and identity propagation (LangGraph, CrewAI, Bedrock Agents, Microsoft Copilot Studio). - Wire evaluations into CI/CD so model swaps and prompt changes are gated by regression harnesses (LangSmith, Braintrust, Ragas, DeepEval). - Instrument LLM traffic for latency, token cost, cache hits, quality drift, and safety flags. - Own RAG operations including chunking pipelines, embedding refresh, vector DB deployment, and hybrid search tuning. - Build model gateway routing strategies optimizing for cost/latency/quality trade-offs. Tech Stack: Gateways & Serving: Databricks AI Gateway · Portkey · LiteLLM · Kong AI Gateway · Azure APIM · vLLM · TGI · Ray ServeModel Frameworks: LangChain, LlamaIndex, Bedrock AgentsRuntime: Python (asyncio), FastAPI, Docker, KubernetesAI Governance & Guardrails: OWASP LLM Top 10, NIST AI RMF Who We're Looking For: - 4+ years in ML engineering or backend systems with recent production LLM experience. - Hands-on expertise with at least one AI Gateway or LLM proxy layer. - Strong Python engineering skills (streaming, backpressure). - Familiarity with LLM observability and guardrails frameworks. - Experience with agent frameworks for governed tool-use. Bonus Points: - Open Source contributions to LLMOps. - Speaking at major AI/MLOps conferences. - Experience in regulated industries.