Software Engineer Architect – AI Agents (Healthcare)

About 100ms

100ms is building AI agents that automate complex patient access workflows in U.S. healthcare — starting with benefits verification, prior authorisation, and referral intake in speciality pharmacy.
Our platform combines LLM-based agents, deep healthcare domain expertise, and operational infrastructure to help care teams move faster and get patients on treatment sooner.

Role Summary

We are looking for a Software Engineer Architect to own the technical architecture of our AI agents platform as it scales. In healthcare, accuracy and trust are everything — our systems make phone calls to payers, handle Protected Health Information, and operate inside real clinical workflows, so the architecture underneath them has to be reliable, observable, secure, and fast to evolve.

This is a hands-on, deeply technical role, not a pure design-review position.

You will define how our services, data, and agent pipelines fit together; make the build-vs-buy and technology choices that shape the next three years of the platform; write code on the hardest problems; and raise the engineering bar across the team.

You will work directly with the founders, product, platform/infrastructure, and security teams, and your decisions will be informed by real customer and clinical workflows.

Key Responsibilities

• Own the end-to-end architecture of the AI agents platform — services, APIs, data models, async task pipelines, LLM/voice agent orchestration, and integrations with payer systems, EMRs, and telephony.

• Design for reliability, latency, and safety: define patterns for fault tolerance, idempotency, retries, observability, and graceful degradation across agent workflows that run at scale in production.

• Set the architecture for multi-tenancy, data isolation, and PHI handling in partnership with our Security and Compliance Lead, keeping HIPAA and SOC 2 requirements designed in rather than bolted on.

• Make and document high-leverage technical decisions — build vs. buy, service boundaries, storage and queueing choices, LLM provider and inference strategy — through lightweight design reviews and ADRs.

• Stay hands-on: prototype critical paths, write production code on the most ambiguous and highest-risk problems, and unblock teams when they hit hard technical walls.

• Partner with the platform team on how workloads run on our Kubernetes (GKE) infrastructure — autoscaling, deployment topology, cost, and SLOs for agent and voice workloads.

• Raise the engineering bar: mentor senior engineers, drive code and design review culture, and define standards for testing, evaluation, and release safety across teams.

• Translate product and customer needs into a technical roadmap; work with founders and product to sequence architectural investments against business priorities.

• De-risk scale: anticipate where the system breaks at 10x call volume, 10x customers, and 10x team size, and lay the groundwork before it does.

Requirements

• 10+ years of professional software engineering experience, including several years designing and operating large-scale, production distributed systems.

• A track record of owning architecture for a product or platform end to end — you have made foundational technical decisions, lived with their consequences, and evolved them.

• Deep expertise in backend systems: service design, RESTful/async APIs, event-driven architectures, task queues, and data modelling across relational and non-relational stores.

• Strong grasp of reliability engineering — observability, fault tolerance, capacity planning, and incident-informed design — for systems with strict correctness and latency requirements.

• Hands-on proficiency in at least one of our core languages (Python or Go); comfort reading and reviewing code across the stack, including React-based frontends.

• Working knowledge of cloud-native infrastructure (GCP/AWS/Azure), Kubernetes, and infrastructure-as-code, and how architectural choices play out operationally.

• Experience building or integrating AI/LLM-backed systems — or clear evidence you can go deep fast on agent orchestration, evaluation, guardrails, and inference trade-offs.

• Strong security instincts: you design with data isolation, encryption, access control, and auditability in mind from day one.

• Excellent written and verbal communication — you can carry a design from whiteboard to ADR to aligned teams, and explain trade-offs to founders…