Agentic AI & GenAI Test Automation Engineer

Position: Agentic AI & GenAI Test Automation Engineer
Experience: 7+ Years
Location: Pan India
Joining: Immediate or within 15 days

Key Responsibilities

Test Automation

• Design, develop, and maintain robust automation frameworks for UI, API, and integration testing.
• Create and execute automated regression, smoke, functional, and end-to-end test suites.
• Develop reusable test components to improve execution efficiency and maintainability.
• Integrate automated tests into CI/CD pipelines.

Agentic AI & GenAI Validation

• Validate agent-to-agent interactions, workflow orchestration, and AI-driven decision-making processes.
• Test LLM-powered workflows, retrieval mechanisms, prompt execution, and tool invocation scenarios.
• Design test strategies to evaluate AI output consistency, accuracy, reliability, and guardrails.
• Validate AI observability, traceability, and audit requirements.

Healthcare Product Quality Assurance

• Ensure compliance with healthcare quality standards and data privacy requirements.
• Validate patient-facing and operational workflows across healthcare applications.
• Perform test data management while maintaining confidentiality and regulatory compliance.

API & Integration Testing

• Design and automate API test suites using REST/GraphQL services.
• Perform contract, integration, and end-to-end testing.

Quality Engineering

• Participate in requirement analysis and provide feedback from a quality perspective.
• Perform risk-based testing and quality assessments.
• Analyze defects, perform root cause analysis, and collaborate with developers for resolutions.
• Generate quality metrics, reports, and release readiness assessments.

Required Skills

Technical Skills

• Strong experience in test automation using JavaScript/TypeScript, Python, or Java.

• Hands-on experience with automation frameworks such as:

• Selenium

• Playwright (preferred)

• Experience in API testing using Postman, REST Assured, or similar tools.
• Knowledge of CI/CD pipelines such as Azure DevOps, GitHub Actions, and Jenkins.
• Experience with Git and version control practices.
• Strong understanding of Agile/Scrum methodologies.
• Experience using JIRA to create bugs, test plans, and test suites.
• Experience using Confluence for documentation.

AI/GenAI Testing Knowledge

• Understanding of:

• Generative AI
• Agentic AI systems
• Large Language Models (LLMs)
• Retrieval-Augmented Generation (RAG)

• Multi-Agent Workflows

• Exposure to testing AI-driven applications, conversational systems, or intelligent assistants.
• Knowledge of prompt testing and evaluation methodologies.

Healthcare Domain

• Experience in healthcare, payer, provider, clinical, or digital health platforms.
• Understanding of healthcare regulations, data privacy, and compliance considerations is an added advantage.

Preferred Qualifications

• Hands-on experience with Playwright for modern web automation.
• Experience validating AI agents built using LangChain, LangGraph, AutoGen, CrewAI, Semantic Kernel, or similar frameworks.
• Knowledge of performance testing tools such as JMeter or LoadRunner.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Familiarity with observability and monitoring solutions.
• Exposure to accessibility and security testing.

Desired Competencies

• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management.
• Ability to work in cross-functional Agile teams.
• Continuous improvement mindset and passion for automation.
• Ability to adapt quickly in a rapidly evolving AI landscape.

Education

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or related discipline.