QA AI Tester

Job Title: QA AI Tester

Location: Hyderabad, India

Experience: 8+ Years

Shift : 2:00 PM - 11:00 PM

Role Overview

We are looking for an experienced QA AI Tester to drive quality assurance strategies for AI/ML-driven systems. This role goes beyond traditional UI/API testing and focuses on validating machine learning models, large language models (LLMs), and data pipelines. You will play a critical role in ensuring reliability, fairness, and performance of AI-powered solutions.

Key Responsibilities

• Lead end-to-end QA strategy for AI/ML and data-driven systems
• Design and implement test frameworks for ML model validation and evaluation
• Perform LLM testing , including prompt evaluation, response validation, and bias detection
• Validate data pipelines (ETL) for data quality, consistency, and integrity
• Build and maintain automation frameworks using Python
• Define test datasets, benchmarks, and evaluation metrics for AI models
• Collaborate with data scientists, ML engineers, and product teams
• Identify risks related to model drift, bias, and performance degradation
• Implement continuous testing practices in CI/CD pipelines
• Ensure compliance with quality, ethical AI, and governance standards

Required Skills & Qualifications

• 8+ years of experience in QA, with strong exposure to AI/ML systems
• Hands-on experience in ML model testing and validation techniques
• Strong understanding of LLMs and prompt engineering/testing methodologies
• Experience with data validation and pipeline testing
• Proficiency in Python for test automation
• Knowledge of statistical analysis and model evaluation metrics
• Experience working in Agile/Scrum environments

Preferred Qualifications

• Experience with tools like Evidently AI, DeepChecks , or similar ML monitoring/testing frameworks
• Familiarity with MLOps practices and tools
• Exposure to cloud platforms (AWS, Azure, or GCP)
• Understanding of AI ethics, bias detection, and explainability

Leadership Expectations

• Mentor QA teams on AI testing practices
• Drive innovation in AI quality engineering
• Establish best practices and standards for AI testing