AI Tester

Job Title: AI Tester

Location: Pan India (Remote Work Mode)

Experience: 512 Years

Role Overview
Responsible for QA and testing of AI/ML and LLM-based web applications, ensuring performance, reliability, accuracy, security, and regulatory compliance.

Key Responsibilities

• Design and execute functional tests covering edge cases, prompt variations, bias validation, and LLM behavior.
• Create comprehensive test plans, test strategies, and test cases.
• Use bug-tracking and test case management tools to manage defects and testing activities.
• Collaborate with product, development, business, and platform teams.
• Test AI chat features, along with standard website and UI components.
• Validate model quality, evaluation metrics, model drift, and performance.
• Test RAG pipelines, retrieval accuracy, response grounding, hallucinations, and agent/tool-use scenarios.
• Conduct non-functional testing, including performance, reliability, and security testing.
• Support test documentation, validation evidence, and audit/compliance activities.
• Perform AI observability testing, including application logs, monitoring, and model behavior analysis.

Requirements

• 37 years of experience in QA/testing or validation, preferably involving AI/ML systems.
• Good understanding of AI concepts, LLMs, evaluation metrics, and model drift.
• Experience in API testing, SQL/data validation, website/UI testing, and test design.
• Hands-on experience with GenAI/LLM testing, prompt engineering, RAG validation, and hallucination testing.
• Knowledge of Responsible AI practices, including bias, fairness, and output reliability.
• Knowledge of Python or TypeScript for test automation.
• Experience analyzing application logs using log aggregation and AI observability tools.
• Exposure to GxP-regulated environments, preferably within pharmaceutical, biotechnology, or life sciences organizations.
• Strong analytical, problem-solving, and communication skills.
• Hands-on experience in LLM/RAG validation, hallucination testing, model drift detection, Responsible AI practices, and AI observability.
• Experience in GxP/pharma validation, model drift testing, Python-based test automation, and AI observability.

Interested candidates can share their CV on [email protected]