AI Test Engineer

Dear Job Seekers,

Greetings from TATA Consultancy Services!!!

Hope you and your family are safe and well.

We are conducting face to face Interview for AI Testing/GenAI Testing/LLM Testing/Generative AI Testing

If interested Kindly revert with your updated CV for further process with TCS.

Walk In Drive Date: 1st of Aug 2026 (Saturday), Hyderabad.

Registration Time: 9.00 AM 1.00 PM

Venue- Tata Consultancy Services Limited , Deccan Park

Plot No.1, · Software Units Layout, · Madhapur, Hyderabad 500 081.

Skills- AI Testing/GenAI Testing/LLM Testing/Generative AI Testing

Exp- 6+

Location- Hyderabad

AI Testing/GenAI Testing/LLM Testing/Generative AI Testing Testing Engineer

We are seeking a highly skilled AI/GenAI Testing Engineer with 6–10 years of experience in Quality Engineering and a strong understanding of AI/LLM-based applications. The ideal candidate will be responsible for validating AI model responses, evaluating RAG systems, ensuring Responsible AI compliance, designing test datasets, and implementing AI quality assurance practices across the software development lifecycle.

Key Responsibilities

AI Response Evaluation & Validation

• Evaluate AI/LLM-generated responses using:
• Human Evaluation
• Automated Evaluation
• LLM-as-a-Judge methodologies
• Assess AI outputs for:
• Accuracy
• Relevance
• Groundedness
• Consistency
• Completeness
• Design and execute test scenarios for Generative AI and RAG-based applications.
• Identify and analyze hallucinations, factual inaccuracies, inconsistencies, and response quality issues.
• Validate response faithfulness against source knowledge and business requirements.

AI Evaluation Frameworks & Tooling

• Utilize and support AI evaluation frameworks such as:
• RAGAS
• LangSmith (preferred)
• Similar GenAI testing and observability tools
• Define, measure, and analyze AI evaluation metrics including:
• Precision
• Recall
• Faithfulness
• Context Relevance
• Toxicity
• Bias
• Safety
• Collaborate with development teams to integrate AI evaluation processes into CI/CD pipelines.
• Support automation of AI testing and reporting through scripting and evaluation workflows.

Responsible AI & Governance

• Ensure AI systems adhere to Responsible AI principles, including:
• Fairness
• Safety
• Transparency
• Explainability
• Bias Detection and Mitigation
• Validate system prompts, instructions, prompt templates, and guardrails.
• Perform prompt testing, adversarial testing, and jailbreak validation.
• Identify security and governance risks associated with AI-generated content.
• Support governance, compliance, and audit requirements for AI solutions.

Test Data & Dataset Management

• Collaborate with business stakeholders and SMEs to understand use cases and requirements.
• Design, curate, and maintain high-quality datasets, including:
• Golden Datasets
• Adversarial Datasets
• Edge Case Datasets
• Regression Test Datasets
• Prepare and manage datasets for model validation, evaluation, and continuous testing.
• Maintain dataset versioning, traceability, and quality standards.

Collaboration & Quality Engineering

• Work closely with AI Engineers, Product Owners, Data Scientists, and Domain Experts.
• Provide actionable insights and recommendations to improve AI quality and user experience.
• Drive best practices, standards, and reusable frameworks for AI testing.
• Support continuous improvement initiatives across AI/GenAI quality assurance processes.

Required Skills & Qualifications

Mandatory Skills

• Strong understanding of:
• AI Testing
• GenAI Testing
• LLM Testing Concepts
• AI Quality Assurance
• Experience in AI response evaluation techniques:
• Human Evaluation
• Automated Evaluation
• LLM-as-a-Judge
• Hands-on experience identifying:
• Hallucinations
• Bias
• Toxicity
• Inconsistencies
• Grounding Issues
• Experience in test data design, dataset creation, and data validation.
• Understanding of Responsible AI frameworks, governance principles, and AI risk management.
• Exposure to RAG-based applications and AI evaluation methodologies.

Preferred Skills

• Experience with RAG evaluation tools such as:
• RAGAS
• DeepEval
• Promptfoo
• Exposure to LangSmith or similar AI observability platforms.
• Understanding of Prompt Engineering and System Prompt Design.
• Knowledge of AI Security Testing, Prompt Injection, and Jailbreak Testing.
• Experience with Python scripting, test automation, and evaluation pipeli…