Data Scientist AI Evaluation
- Source: Indeed
- Twinstar Holdings LLC
- $114,400 to $156,000 /yr
- Remote
- United States
- Mid level
Job Description
Data Scientist — AI Evaluation Analytics
Role Overview
Provide data science and analytical expertise to improve the measurement, reliability, and interpretation of AI system performance through data-driven evaluation approaches.
Key Responsibilities
- Support development and refinement of evaluation metrics, scoring approaches, and benchmark methodologies.
- Validate datasets, data quality criteria, and evaluation inputs to improve reliability of results.
- Analyze evaluation data, performance trends, and measurement signals to generate actionable insights.
- Apply statistical analysis and experimentation techniques to assess AI system effectiveness.
- Develop analytical approaches to identify trends, patterns, and opportunities for improvement.
Required Skills & Experience
- Strong background in data science, analytics, statistics, or related fields.
- Experience designing metrics, analytical frameworks, or measurement approaches.
- Proficiency in Python, SQL, and data analysis techniques.
- Experience working with large datasets and extracting meaningful insights.
- Strong communication skills with the ability to explain analytical results clearly.
Preferred Experience
- Experience evaluating AI/ML systems and agentic AI applications.
- Experience with experimentation, benchmarking, or performance analysis.
- Experience working with software engineering, productivity, or quality metrics.
Pay: $55.00 - $75.00 per hour
Expected hours: 40.0 per week
Work Location: Remote
Skills
- benchmark
- Python
