Senior AI Data Operations Analyst [SK-11207]

Our media client, partnered with Aquent, is a global leader in the entertainment industry, dedicated to connecting millions of users with engaging audio experiences. They are at the forefront of innovation, continuously evolving how listeners discover and interact with music, podcasts, and audiobooks. This is an incredible opportunity to join a team that shapes the daily experience of a vast global audience, directly influencing how users discover and engage with content through cutting-edge AI and machine learning.

We are seeking a **Senior AI Data Operations Analyst** to play a pivotal role in ensuring the highest quality and relevance of their recommendation engines and AI/LLM-powered features. In this hands-on, highly collaborative position, you will be a key driver of data quality and annotation strategy, bridging critical gaps between product managers, data scientists, and engineers. Your work will directly contribute to building the essential datasets needed to train and evaluate next-generation features, making a tangible impact on the core user experience and the future of content discovery.

What You’ll Do

• Perform hands-on data annotations and lead larger, cross-functional annotation sessions to generate high-quality training datasets for advanced recommendation models.
• Establish clear criteria, metrics, and qualitative success measures for core user experience features and the LLM judges evaluating them.
• Design and execute structured qualitative testing using internal tools, ensuring human judgment remains central while leveraging AI/LLM tools to scale evaluation efforts efficiently.
• Partner on major, publicly announced initiatives, driving the evolution of personalized and agentic user experiences.
• Improve and maintain robust evaluation processes, guidelines, and documentation that are adopted across various product groups.
• Present qualitative findings, data trends, and quality risks clearly and concisely to product, design, research, and engineering leads, influencing strategic decisions.

Who You Are

• Data Quality / Annotation Background: Proven experience in data annotation, data quality, or product/content quality analysis with direct ownership over evaluation workflows.
• Strong Qualitative & Analytical Skills: Highly comfortable running structured qualitative evaluations, handling large data sets, and transforming subjective feedback into clear, actionable quality metrics.
• AI/ML Familiarity: Strong functional understanding of machine learning product development and how ML/LLM/agentic features are evaluated and trained. You understand how data quality directly impacts AI outputs.
• Process & Framework Builder: Demonstrated ability to create or refine robust evaluation frameworks and guidelines that empower team members to consistently maintain high quality standards.
• Strong Communicator: Excellent written and verbal communication skills, with the ability to confidently present findings and guide cross-functional teams through complex evaluation initiatives.
• Tool Proficiency: Comfortable working with standard data tools (e.g., Excel, Google Sheets) and adept at quickly learning internal proprietary annotation and evaluation platform tools.

Nice-to-Haves

• Experience in digital media or streaming entertainment platforms (though data annotation experience in other tech industries is fully welcome).

The target hiring compensation range for this role is $47.00/hr to $51.00. Compensation is based on several factors including, but not limited to education, relevant work experience, relevant certifications, and location.

About Skill Skill connects the best professional, IT, engineering, financial and administrative talent with the world’s biggest brands. Our eligible talent get access to benefits such as health benefit contributions, retirement plans with match and flexible spending accounts.

Skill is an equal-opportunity employer. We evaluate qualified applicants without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, and other legally protected characteristics. We’re about creating an inclusive environment-one where different backgrounds, experiences, and perspectives are valued, and everyone can contribute, grow their careers, and thrive.