Applied AI Safety Engineer [SK-16736]

Aquent is partnering with a leading global company that is revolutionizing how millions discover and enjoy content. This organization is at the forefront of understanding user preferences, leveraging advanced technology to make every interaction seamless and delightful. Their mission is to simplify and enhance the discovery experience, ensuring users always find something they love. Join a team dedicated to innovation, where your contributions will directly shape the future of content recommendation and engagement for a vast global audience, empowering users worldwide to explore and connect with content they adore.

Are you passionate about the responsible development of artificial intelligence? Do you thrive on tackling complex challenges to ensure AI systems are safe, fair, and trustworthy? We are seeking a visionary applied researcher to join a pioneering team, where you will be instrumental in safeguarding cutting-edge AI products. Your work will directly impact millions of users, ensuring that the next generation of intelligent systems operates with integrity and reliability. This is a unique opportunity to own ambiguous safety problems from concept to deployment, making a tangible difference in the ethical evolution of AI and shaping the future of responsible technology.

What You’ll Do

• Develop comprehensive threat models and harm taxonomies specifically tailored for conversational, recommender, and advanced tool-using AI systems.
• Design and execute sophisticated adversarial evaluations, incorporating expert red teaming, automated attack generation, synthetic data, and real-world production data.
• Construct robust, reusable Python evaluation pipelines, implement LLM-as-a-judge workflows, create regression tests, build intuitive dashboards, and curate essential golden datasets.
• Rigorously validate evaluators against human labels, meticulously quantifying coverage, judge reliability, false positives, false negatives, and the critical safety–utility trade-offs.
• Translate complex findings into actionable mitigations, including policy adjustments, prompt engineering, context improvements, classifier development, data enhancements, preference tuning, and robust system-level controls.
• Collaborate closely with engineering teams and trust & safety experts to seamlessly integrate evaluations into the product development lifecycle and ongoing monitoring processes.
• Articulate complex technical results with clarity and precision to both technical and non-technical stakeholders, fostering understanding and driving informed decisions.

Must-Have Qualifications

• Demonstrated experience in delivering safety evaluations or implementing mitigations for real-world AI or machine learning products, with a proven ability to scale processes through automation.
• Strong coding proficiency and practical data-analysis skills, including sufficient programming expertise and SQL knowledge to independently assess code and queries.
• Proven experience in designing adversarial tests, developing comprehensive evaluation datasets, creating detailed taxonomies, establishing clear rubrics, and defining relevant metrics.
• Ability to work autonomously and drive projects forward, even when initial risk assessments, success criteria, and methodologies are ambiguous.
• Experience collaborating effectively across diverse teams, including research, engineering, product development, policy, or trust & safety functions.
• Exceptional written communication skills, with a track record of translating research findings into impactful actions and recommendations.

Nice-to-Have Qualifications

• Experience in evaluating multi-turn or tool-using agents, understanding their unique safety challenges.
• Proficiency in calibrating LLM judges or building sophisticated human-in-the-loop evaluation systems.
• Experience with multilingual or multimodal evaluation, addressing the complexities of diverse data types and languages.
• Familiarity with preference tuning or other advanced model-alignment techniques.
• An MSc or PhD in an AI/ML-related field, providing a strong theoretical foundation.

About Skill Skill connects the best professional, IT, engineering, financial and administrative talent with the world’s biggest brands. Our eligible talent get access to benefits such as health benefit contributions, retirement plans with match and flexible spending accounts.

Skill is an equal-opportunity employer. We evaluate qu…