AI/LLM Safety Engineer

Job Type

Full-time

Description

AI/LLM Safety Engineer

Own Your Impact. At Propio, we don’t believe careers happen to people. We believe people create them.

Here, you’re trusted to make decisions, challenge assumptions, drive innovation, and shape outcomes. Your success is not limited by hierarchy or tenure. It’s fueled by your ambition, your curiosity, and your willingness to own your impact. If you’re looking for a role where you can simply maintain the status quo, this probably isn’t it, but if you’re looking for a place where your ideas matter, your growth is accelerated, and your work creates meaningful impact across the world, we’d love to talk.

Why Propio?

Every day, communication changes lives. A patient receives care they otherwise couldn’t access. A family gains critical information. A business connects with a customer. A community becomes more inclusive. These moments happen because barriers are removed. And behind those moments are Propio team members who show up every day to solve problems, innovate, and build the future. This isn’t just work. This is world impact.

As a AI/LLM Safety Engineer, you’ll have the opportunity to make a meaningful contribution to the continued growth and transformation of Propio.

You’ll be empowered to:

• Take ownership of important initiatives and outcomes.
• Drive meaningful business results.
• Influence decisions and contribute new ideas.
• Partner with talented, high-performing team members.
• Challenge yourself through continuous learning and growth.
• Help shape the future of a rapidly growing organization.

What You’ll Own

The AI/LLM Safety Engineer will join our AI team and take ownership of how safely our models and agents behave in production; with a focus on AI Safety, Trust & Safety, and Responsible AI. You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do. Agent safety is the primary focus of this role: you will help ensure that as our systems gain the ability to call tools and take actions, they do so within well-defined, well-tested boundaries.

LLM Safety Evaluation & Red Teaming

• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships.
• Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure.

Guardrails & Runtime Controls

• Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting.
• Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users.

Agent Safety (primary focus of this role)

• Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls.
• Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves.

Monitoring & Observability

• Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production.

Governance & Collaboration

• Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards.
• Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.

What Makes Someone Successful Here

The most successful people at Propio aren’t necessarily the ones with the longest resumes. They’re the people who:

• Take ownership instead of waiting for direction.
• Embrace challenges as opportunities to grow.
• Continuously seek better ways of working.
• Turn ideas into action.
• Hold themselves and others accountable to high standards.
• Are driven by making a measurable impact.

Requirements

What You’ll Bring

Required Qualifications

• Ba…