Fullstack AI Quality Engineer – Applied & Agentic AI Systems
- Source: LinkedIn
- Top Gen AI Jobs
- On-site
- Hyderabad
- Full-Time
- Mid level
Job Description
Home/Jobs/Fullstack AI Quality Engineer – Applied & Agentic AI Systems
Fullstack AI Quality Engineer – Applied & Agentic AI Systems
Roche
Hyderabad
7+ years
2 days ago
$56.6K–71.1K/yr
Full-time
Onsite
Skills Required LLM
RAG
Agentic AI
Gen AI
LangChain
RAGAS
Agentic Workflow Validation
Promptfoo
DeepEval
Golden Datasets
LangSmith
Arize Phoenix
Python
Pytest
allure
Description Roche is seeking an AI Quality Engineer to test Generative and non-Generative AI applications and agentic systems, ensuring reliability, fairness, and effectiveness of AI-powered solutions in healthcare.
Company: Roche
Role: AI Quality Engineer
Location: Hyderabad, India
Experience
- 7+ years of experience in software testing
- At least 1 year focused on genAI and/or agentic applications
- Experience working in regulated environments with qualified infrastructure and validated applications
- Experience working with Agile development teams
Qualification
- B.Sc., B.Eng., M.Sc., M.Eng. in Computer Science, Software Engineering, or equivalent degree
- Strong background in testing methodologies
- English communication skills at C1+ level
Responsibilities
- Design and implement testing strategies for LLM-based and agentic AI systems
- Perform manual exploratory testing and automated UI/API testing
- Validate classical machine learning and generative AI behaviors
- Develop and maintain quality assurance frameworks and compliance documentation
- Identify and test edge cases, biases, and fairness in AI model outputs
- Collaborate on security and performance testing of AI applications
- Create and maintain comprehensive test documentation and guidelines
- Integrate automated testing suites within CI/CD pipelines
- Curate and manage evaluation datasets for benchmarking
- Leverage observability and feedback loops to monitor AI system performance
Additional Responsibilities
- Work in a highly regulated environment ensuring safety, compliance, and robustness of AI solutions
- Use critical thinking and problem-solving for AI prompt testing and data distribution analysis
- Proactively identify potential issues and suggest improvements
- Communicate findings and recommendations clearly
- Align working hours to overlap with Central European Time for collaboration
Nice To Have
- Practical experience testing applications on cloud platforms such as AWS
- Experience with monitoring, observability tools, and log analysis
- Familiarity with Roche mandatory ALM tools like Jira and GitHub
- Use of AI coding assistants and autonomous agents to accelerate development and testing
- Conduct code reviews for human-written and AI-generated code
- Understanding of security testing methodologies for AI applications
- Knowledge of regulatory compliance and AI quality assurance standards in regulated industries
- Stay current with latest AI and GenAI testing methodologies and best practices
More Skills TypeScript, Playwright, Jest, Cypress, GitHub Actions, CI/CD pipelines, AWS, Grafana, Jira, GitHub, AI evaluation tools, AI/LLM testing frameworks, Automated UI/API/MCP testing, Machine learning validation, Generative AI evaluation, Agentic AI systems testing
Prepare for this role
Recommended resources to build the skills for this position. Sponsored.
Top 10 RAG Quick-Prep Questions
Zenaique
A shorter RAG interview prep set for quick review.
Top 20 LLM System Design Questions (Quick)
Zenaique
A focused LLM system design interview question set.
Top 50 LLM System Design Interview Questions
Zenaique
Curated LLM system design questions from Zenaique.
More LLM jobs
AI/LLM Safety Engineer
Propio Language Services
Leawood
Today
AI/ML Engineer 2
Day And Zimmermann
United States
Today
Software/AI developer
Rackner
United States
Today
Conversational AI Engineer
Bright Vision Technologies
United States
Today
Sr Lead Software Engineer - Agentic AI Solutions
JPMorgan Chase
Mumbai
Today
Senior Technical Consultant — AI Engineering & Agentic Salesforce Delivery
Salesforce
Bengaluru
Today
Skills
- LangChain
- Python
- Pytest
- amazon-web-services
- JIRA
- Github
- TypeScript
- microsoft-typescript
- Playwright
- Jest
- Cypress
- github-actions
- Grafana
- mc-lms
- curated
- propio
- Salesforce
You missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
