Data Scientist - AI Evaluation (West Coast Preferred)

Data Scientist — AI Evaluation Analytics

Role Overview

Provide data science and analytical expertise to improve the measurement, reliability, and interpretation of AI system performance through data-driven evaluation approaches.

Key Responsibilities

· Support development and refinement of evaluation metrics, scoring approaches, and benchmark methodologies.

· Validate datasets, data quality criteria, and evaluation inputs to improve reliability of results.

· Analyze evaluation data, performance trends, and measurement signals to generate actionable insights.

· Apply statistical analysis and experimentation techniques to assess AI system effectiveness.

· Develop analytical approaches to identify trends, patterns, and opportunities for improvement.

Required Skills & Experience

· Strong background in data science, analytics, statistics, or related fields.

· Experience designing metrics, analytical frameworks, or measurement approaches.

· Proficiency in Python, SQL, and data analysis techniques.

· Experience working with large datasets and extracting meaningful insights.

· Strong communication skills with the ability to explain analytical results clearly.

Preferred Experience

· Experience evaluating AI/ML systems and agentic AI applications.

· Experience with experimentation, benchmarking, or performance analysis.

· Experience working with software engineering, productivity, or quality metrics.

Pay: $55.00 - $74.00 per hour

Expected hours: 40.0 per week

Work Location: Remote