Applied AI Safety Engineer [SK-15331]

This a remote position but you must work EST hours. The Personalization mission makes deciding what to play next easier and more enjoyable for every listener. From curated blends to personalized weekly playlists, we’re behind some of our most-loved features. We built them by understanding the world of music and podcasts better than anyone else. Join us and you’ll help keep millions of users listening by making great recommendations for each and every one of them. We ask that our team members be physically located in Central European or Eastern US time zones for the purposes of our collaboration hours.

We are looking for a hands-on applied researcher to identify, measure, and reduce safety risks in our conversational and agentic AI products. You will own ambiguous safety problems from initial threat modelling through evaluation design, data creation, analysis, mitigation, and ongoing monitoring.

What You’ll Do

• Develop product-specific threat models and harm taxonomies for conversational, recommender, and tool-using AI systems.
• Design and run single- and multi-turn adversarial evaluations, combining expert red teaming, automated attack generation, synthetic data, and sampled production data.
• Build reusable Python evaluation pipelines, LLM-as-a-judge workflows, regression tests, dashboards, and curated golden datasets.
• Validate evaluators against human labels and quantify coverage, judge reliability, false positives, false negatives, and safety–utility trade-offs.
• Turn findings into practical mitigations, including policy and prompt changes, context engineering, classifiers, data improvements, preference tuning, and system-level controls.
• Work directly with Engineering and Trust & Safety to embed evaluations into product-development and monitoring loops.
• Communicate results clearly to technical and non-technical stakeholders.
• Full-time availability is preferred, although part-time arrangements may be considered.

Who You Are

• You have personally delivered safety evaluations or mitigations for a real AI or machine-learning product, using automation to scale processes.
• You have strong coding agent and practical data-analysis skills, with sufficient programming and SQL to independently judge code and queries.
• You have experience designing adversarial tests, evaluation datasets, taxonomies, rubrics, and metrics.
• You can work autonomously when the risk, success criteria, and methodology are initially unclear.
• You have experience working across research, engineering, product, policy, or Trust & Safety.
• You communicate clearly in writing and have a record of turning research findings into action.

It’s a Plus If You Have

• Experience evaluating multi-turn or tool-using agents.
• Experience calibrating LLM judges or building human-in-the-loop evaluations.
• Experience with multilingual or multimodal evaluation.
• Experience with preference tuning or other model-alignment techniques.
• An MSc or PhD in an AI/ML-related field.

The target hiring compensation range for this role is $72.00/hr to $78.00/hr. Compensation is based on several factors including, but not limited to education, relevant work experience, relevant certifications, and location.

• About Skill:**

Skill connects the best professional, IT, engineering, financial and administrative talent with the world’s biggest brands. Our eligible talent get access to benefits such as health benefit contributions, retirement plans with match and flexible spending accounts.

Skill is an equal-opportunity employer. We evaluate qualified applicants without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We’re about creating an inclusive environment-one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.