Senior Software Engineer (AI Inference & Runtime Platform)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer (AI Inference & Runtime Platform) based in the United States.

Join a mission-driven engineering team building advanced AI infrastructure to address complex challenges across critical industries, including energy, sustainability, logistics, and climate technology. In this role, you will own the runtime foundations that power AI workloads, from high-performance model inference to secure execution environments for AI-generated code. You will design and operate scalable systems across Kubernetes clusters, GPU infrastructure, and cloud platforms. Working across Rust, Python, distributed systems, and infrastructure automation, you will help deliver reliable, cost-efficient, and secure AI services. This opportunity is ideal for an engineer who has built and operated production-grade inference or serverless GPU platforms and enjoys solving challenging systems problems. You will also contribute to open-source software and help establish engineering standards as the platform evolves.

Accountabilities:

You will own the core infrastructure that enables AI models and agent-generated workloads to run securely, efficiently, and reliably in production. Your responsibilities will include:

• Manage the model-serving infrastructure for open-weight large language models, including deployment, configuration, cold-start optimization, model-specific service-level objectives (SLOs), upgrades, and canary releases.
• Operate and improve the Kubernetes infrastructure supporting inference and sandbox workloads, including custom operators, custom resource definitions (CRDs), autoscaling, GPU scheduling and sharing, and node lifecycle management.
• Own the stateful data plane end to end, ensuring reliable operation, documented recovery procedures, and regularly tested restoration processes.
• Develop and maintain the secure agent-sandboxing runtime and host-side control plane, covering lifecycle management, command execution, snapshots, forking, teardown, usage metering, and isolation-boundary security.
• Build and maintain FastAPI control-plane services, infrastructure-as-code configurations using Terraform/OpenTofu and Bicep, and dashboards that support effective operations.
• Provide the infrastructure foundation used by backend engineering teams, collaborating across services and investigating issues that span the platform and application layers.
• Establish and maintain observability, performance, reliability, and operational practices that help teams diagnose problems and make informed decisions.
• Contribute to an open-source-oriented development process through reviewed pull requests, comprehensive documentation, reproducible builds, and releases as components mature.

Requirements:

You are an experienced systems engineer with a strong production background, sound security judgment, and the ability to operate complex infrastructure under real-world conditions. You bring most of the following qualifications:

• At least 5 years of experience delivering production systems using a systems programming language. Strong Rust expertise is valuable, while deep experience in Go, C/C++, or Zig combined with a genuine willingness to work in Rust is also relevant.
• Solid understanding of asynchronous runtimes, memory safety, and low-level debugging, including troubleshooting at the system-call boundary.
• Hands-on experience operating Kubernetes workloads that other teams depend on, including controllers or operators, scheduling, autoscaling, and node lifecycle management.
• Proven ability to threat-model isolation boundaries involving technologies such as namespaces, cgroups, seccomp, and hypervisors, including assessing what an untrusted guest could observe, manipulate, or exhaust.
• Practical knowledge of secure AI-agent execution, including least-privilege access, preventing credentials from entering sandbox environments, maintaining audit trails, and requiring human approval for actions that modify systems of record.
• Experience deploying or operating open-weight LLM inference infrastructure, such as vLLM, SGLang, or comparable serving systems, and packaging models into reliable, metered production endpoints.
• Strong distributed-systems performance skills, including measurement-driven optimization, latency analysis, and the ability to demonstrate improvements with concre…