AI Engineer III

Description

AI Engineer III

This role will be posted through 9:00 am Tuesday 10/27.

Location

Remote (United States), with core U.S. business hours required.

How you’ll contribute to our mission:

We’re looking for a hands-on AI Engineer to design, build, and ship customer-facing, production-grade features powered by modern LLMs. You’ll partner with product, data, platform, and Customer Experience/Support to turn messy real-world problems into reliable, safe, and measurable AI solutions. You’ll close the loop from voice-of-customer insight model/design choices launch telemetry and iteration, with success measured by outcomes like task completion, CSAT, time-to-resolution, and deflection rate—not just model scores.

You will help develop the next-generation agentic platform that powers customer-facing assistants across the entire journey—from discovery and onboarding to in-product guidance and support. These agents will reason, plan, call internal tools/APIs, retrieve knowledge, and escalate to humans gracefully. You’ll collaborate with CX/Support, Product, and Platform to integrate with CRM and knowledge bases, implement memory and personalization, enforce safety/quality guardrails, and run evaluations and A/B tests. Success is measured in real CX outcomes: shorter time-to-resolution, higher FCR/CSAT, lower effort, and reliable containment.

How you’ll drive success:

• Own end-to-end development of LLM features: problem framing, data prep, prototyping, offline/online evaluation, deployment, and monitoring.
• Build retrieval-augmented generation (RAG) pipelines with vector search (e.g., FAISS, Pinecone, OpenSearch/KNN) and document orchestration.
• Implement prompt strategies, tool use/function calling, and guardrails for safety, bias, and privacy.
• Integrate models in production services (REST/GraphQL/gRPC), including auth, rate limiting, and observability.
• Stand up evals and experiment frameworks (A/B tests, golden sets, regression suites) with clear success metrics.
• Optimize for latency, cost, and quality: prompt compression, caching, model selection, fine-tuning/LoRA, distillation where appropriate.
• Collaborate with DevOps/MLOps/Platform to automate CI/CD, data/version management, and feature flags.
• Embed with CX/Support to mine tickets, chats, and call transcripts; convert VOC into training/eval datasets and backlog priorities.
• Instrument user journeys and define online/offline evals (win rate, hallucination rate, TTR, CSAT/NPS); run A/B tests and ship iterative improvements.
• Build feedback loops (thumbs-up/down, rationale capture, escalation) and human-in-the-loop fallbacks that protect quality.
• Own reliability and UX details that matter for customers: latency budgets, safe fallbacks, clear handoff to human agents, accessibility.
• Partner with Trust/Legal/Security to ensure privacy-by-design and compliant data handling; implement guardrails and red-team mitigations.

Success looks like (first 6 months):

• Document designs and teach best practices to engineering partners.
• Ship 1–2 LLM features to production with SLAs, monitoring, and rollback plans.
• Establish an eval harness (offline + online) and quality gates for prompts/RAG. Reduce average latency/cost per request by- 20% without quality regression.
• Create internal runbooks and dashboards for reproducibility and troubleshooting.

What you bring to help us grow:

• Model customization (fine-tuning/LoRA) and synthetic data generation.
• Streaming and toolcalling/agents, structured outputs (JSON, function schemas).
• Cloud & MLOps: AWS (SageMaker/Bedrock/Lambda), Docker, Terraform, Kubernetes.
• Frontend integration patterns for AI UX (streaming UIs, fallbacks, user feedback loops).
• Domain experience in compliance-heavy environments (e.g., education, finance, healthcare).

What you’ll need to thrive:

• 4–6 years in applied ML/AI or backend engineering with measurable production impact.
• Strong Python and software engineering fundamentals (testing, types, CI/CD).
• Practical LLM experience: OpenAI/Anthropic, or cloud providers (AWS Bedrock, Azure OpenAI, GCP Vertex).
• Experience with at least one deep learning or LLM framework (PyTorch, Transformers, vLLM) and one orchestration library (LangChain, LlamaIndex, Guidance, or custom).
• RAG and data pipelines: chunking/embedding strategies, vector DBs, metadata filtering, and document QA.
• Monitoring/telemetry for AI systems (e.g., MLflow, Weights & Biases, P…