Fullstack AI Quality Engineer – Applied & Agentic AI Systems
- Source: LinkedIn
- Top Gen AI Jobs
- On-site
- Hyderabad
- Full-Time
- Mid level
Job Description
Home/Jobs/Fullstack AI Quality Engineer – Applied & Agentic AI Systems
Fullstack AI Quality Engineer – Applied & Agentic AI Systems
Roche
Hyderabad
7+ years
2 days ago
$56.6K–71.1K/yr
Full-time
Onsite
Skills Required LLM
RAG
Agentic AI
Gen AI
Agentic workflow validation
LangChain
AI evaluation frameworks
Python
Pytest
allure
TypeScript
Playwright
Jest
Cypress
React
Description Roche is seeking an AI Quality Engineer specialized in testing Generative and non-Generative AI applications and agentic systems. The role involves full-stack quality assurance including manual and automated testing across Python and TypeScript stacks, focusing on AI and LLM-based systems.
Company: Roche
Role: AI Quality Engineer
Location: Hyderabad, India
Experience
- 7+ years of experience in software testing
- At least 1 year focused on genAI and/or agentic applications
- Experience working in regulated environments
- Experience working with Agile development teams
Qualification
- B.Sc. in Computer Science, Software Engineering, or equivalent
- B.Eng. in Computer Science, Software Engineering, or equivalent
- M.Sc. in Computer Science, Software Engineering, or equivalent
- M.Eng. in Computer Science, Software Engineering, or equivalent
- English communication skills at C1+ level
Responsibilities
- Design and implement testing strategies for LLM-based and agentic AI systems
- Conduct manual exploratory testing and automated UI/API/MCP testing
- Validate classical machine learning and generative AI behaviors
- Develop and maintain quality assurance frameworks and compliance documentation
- Identify and test edge cases, biases, and fairness in AI model outputs
- Collaborate on security and performance testing of AI applications
- Integrate automated testing suites into CI/CD pipelines
- Curate and manage evaluation datasets for benchmarking
- Leverage observability tools to monitor AI system performance and feedback
Additional Responsibilities
- Co-create and maintain AI-specific testing frameworks
- Document testing strategies, test cases, and guidelines specific to AI
- Work in a regulated environment with qualified infrastructure and validated applications
- Understand GxP vs non-GxP products and CSV and ITSM processes
- Stay current with latest AI and GenAI testing methodologies and best practices
- Communicate findings and recommendations clearly
- Work hours structured to overlap with Central European Time golden hours
Nice To Have
- Practical experience testing applications on AWS cloud platform
- Experience with monitoring, log analysis, and performance metrics tracking
- Experience with Roche ALM tools such as Jira and GitHub
- Use of AI coding assistants and autonomous agents to accelerate development and testing
- Conduct code reviews for human-written and AI-generated code
- Understanding of AI quality assurance standards in regulated industries
More Skills GitHub Actions, CI/CD pipelines, RAGAS, DeepEval, Promptfoo, LangSmith, Arize Phoenix, AWS, Grafana, Jira, GitHub, AI evaluation tools, AI/LLM testing, Automated UI/API testing, Machine learning validation, Generative AI evaluation, Agentic AI systems, Security testing methodologies
Prepare for this role
Recommended resources to build the skills for this position. Sponsored.
Top 10 RAG Quick-Prep Questions
Zenaique
A shorter RAG interview prep set for quick review.
Top 20 LLM System Design Questions (Quick)
Zenaique
A focused LLM system design interview question set.
Top 50 LLM System Design Interview Questions
Zenaique
Curated LLM system design questions from Zenaique.
More LLM jobs
AI/LLM Safety Engineer
Propio Language Services
Leawood
Today
AI/ML Engineer 2
Day And Zimmermann
United States
Today
Software/AI developer
Rackner
United States
Today
Conversational AI Engineer
Bright Vision Technologies
United States
Today
Sr Lead Software Engineer - Agentic AI Solutions
JPMorgan Chase
Mumbai
Today
Senior Technical Consultant — AI Engineering & Agentic Salesforce Delivery
Salesforce
Bengaluru
Today
Skills
- LangChain
- Python
- Pytest
- TypeScript
- microsoft-typescript
- Playwright
- Jest
- Cypress
- React
- amazon-web-services
- JIRA
- Github
- github-actions
- Grafana
- curated
- propio
- Salesforce
You missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
