Senior Backend Engineer, AI Platform

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Backend Engineer, AI Platform based in the United States.

This is an opportunity to build the core AI infrastructure powering a digital platform used by millions of people. As a Senior Backend Engineer, you will work closely with platform architects and engineering teams to design, develop, and scale production-ready AI capabilities from the ground up. You will bring deep expertise in large language models (LLMs), retrieval-augmented generation, model serving, and reliable backend architecture to deliver practical AI features at scale. The role offers significant technical ownership, from system design and API development to deployment, monitoring, and ongoing production support. You will help establish engineering standards, improve AI quality and reliability, and shape the platform’s evolution as the team grows. This is a full-time, remote opportunity for a US-based engineer who enjoys solving complex technical challenges in a fast-moving environment.

Accountabilities

• AI Platform Development: Design, build, and scale the backend infrastructure required to deliver production-grade AI features across a platform serving millions of users.
• LLM Integration and Serving: Develop and maintain services that integrate multiple LLM providers, supporting model routing, failover, token streaming, and cost and usage tracking.
• Retrieval-Augmented Generation: Build and optimize retrieval systems, including vector search, embedding pipelines, context assembly, and citation-supported responses.
• Backend Architecture and APIs: Design robust APIs and backend services, taking end-to-end ownership of service architecture, schemas, implementation, deployment, and operational support.
• Reliability and Observability: Anticipate and address production risks such as hallucinations, retrieval failures, provider outages, latency issues, and unexpected costs through appropriate monitoring and instrumentation.
• AI Evaluation and Quality: Develop practical, repeatable evaluation methods to assess response relevance, safety, source quality, and overall AI performance.
• Production Operations: Monitor live services, troubleshoot incidents, improve system resilience, and support ongoing performance and reliability improvements.
• Cross-Functional Collaboration: Work closely with platform architects and fellow developers to shape technical decisions, establish engineering best practices, and integrate AI capabilities with existing application services.

Requirements

• Backend Engineering Experience: At least five years of backend engineering experience using a statically typed language such as Go, Java, Kotlin, or C#.
• Production LLM Expertise: Demonstrated experience shipping and maintaining LLM-powered features in production, including retrieval-augmented generation, streaming responses, and tool use.
• LLM Serving Infrastructure: Hands-on experience with model-serving systems, including multi-provider routing, failover strategies, token streaming, and cost and usage metering.
• Retrieval Systems: Experience developing vector search solutions, embedding pipelines, context assembly workflows, and systems that provide source-backed answers.
• Reliability and Observability: Strong understanding of AI failure modes and the ability to implement instrumentation, monitoring, and safeguards to detect and address production issues.
• AI Evaluation: Ability to assess AI output quality using straightforward, repeatable tests covering relevance, safety, and source quality rather than relying exclusively on academic benchmarks.
• API Design and Ownership: Strong API design skills and the ability to own backend services throughout their lifecycle, from data schema and implementation to deployment and on-call support.
• Location and Work Authorization: Must be based in the United States and authorized to work in the country.
• Preferred Programming Skills: Go experience is strongly preferred. Familiarity with Python is also beneficial.
• Preferred AI Infrastructure Experience: Experience with LLM gateways or serving frameworks such as LiteLLM, vLLM, TGI, or comparable technologies.
• Preferred Data and Model Experience: Familiarity with vector databases such as Qdrant or pgvector, embedding pipelines, and fine-tuning open-weight models using LoRA or full fine-tuning methods.
• Additional Advantages…