Sr. Data Engineer

Assent is seeking a Senior Data Engineer specializing in AI/ML. This role requires deep expertise in knowledge base construction, retrieval-augmented reasoning (RAQ/RAG), and Generative AI data pipelines to support Assent's research and development into Agentic AI systems.

In this position, you will be responsible for designing, building, and maintaining intelligent data infrastructures that provide context, memory, and reasoning capabilities for autonomous AI agents. Your work will involve integrating structured and unstructured enterprise data into continuously updated knowledge graphs and vectorized stores, enabling dynamic retrieval, planning, and decision-making.

You will collaborate with AI/ML engineers, data scientists, and product teams to develop scalable, auditable, and high-fidelity data pipelines that support both assistive and autonomous AI functions. This role is ideal for individuals passionate about the intersection of data engineering, AI architecture, and knowledge representation.

Key Responsibilities:

• Design, build, and optimize data pipelines for Agentic and Generative AI systems, facilitating context retrieval, multi-step reasoning, and adaptive knowledge updates.
• Develop and manage knowledge bases, vector stores, and graph databases for organizing and retrieving information across regulatory, product, and supplier domains.
• Engineer retrieval-augmented reasoning (RAQ/RAG) pipelines, incorporating embedding generation, contextual chunking, and retrieval orchestration for LLM-driven agents.
• Collaborate with AI/ML, MLOps, Data, and Product teams to define data ingestion, transformation, and retrieval strategies aligned with evolving AI agent capabilities.
• Implement and automate workflows for ingesting structured and unstructured content (documents, emails, APIs, metadata) into searchable, continuously enriched data stores.
• Design feedback and reinforcement loops allowing AI agents to validate, correct, and refine their knowledge sources.
• Ensure data quality, consistency, and traceability through schema validation, metadata tagging, and lineage tracking within knowledge and vector systems.
• Integrate monitoring and observability to assess retrieval performance, coverage, and model-data alignment for deployed agents.
• Collaborate with data governance and security teams to enforce compliance, access control, and Responsible AI data handling standards.
• Document schemas, pipelines, and data models to ensure reproducibility, knowledge sharing, and long-term maintainability.
• Stay abreast of AI data innovations, evaluating new technologies in graph reasoning, embedding architectures, autonomous data agents, and memory frameworks.
• Adhere to corporate security policies and follow Assent's established processes and procedures.

Qualifications:

• 8+ years of experience in data engineering or applied AI infrastructure, with practical expertise in knowledge-centric or agentic AI systems.
• Proven experience in building retrieval-augmented generation (RAG) and retrieval-augmented reasoning/querying (RAQ) data pipelines.
• Strong proficiency in Python and SQL, with experience in large-scale data processing and orchestration workflows (e.g., Airflow, Prefect, Step Functions).
• In-depth familiarity with vector databases (e.g., Weaviate, Pinecone, FAISS, Elastic Vector Search, Milvus) and graph databases (e.g., Neo4j, AWS Neptune, ArangoDB).
• Hands-on experience with embedding generation, semantic indexing, and context chunking for LLM retrieval and reasoning.
• Experience with agentic AI protocols and orchestration frameworks such as Model Context Protocol (MCP), LangChain Agents, Semantic Kernel, DSPy, LlamaIndex Agents, or custom orchestration layers.
• Knowledge of cloud data platforms (AWS preferred: S3, Glue, Lambda, ECS, Athena, Redshift) and infrastructure-as-code tools.
• Knowledge of data modeling, schema design, and indexing strategies for both relational and NoSQL systems.
• Understanding of LLM data workflows, including prompt evaluation, retrieval contexts, and fine-tuning data preparation.