Senior QA (Gen AI - Must have)

;

• Design, automate, and execute API test suites (functional, contract, performance, resilience)
• Define LLM evaluation strategies (performance testing, golden sets, guardrails, hallucination/consistency checks, relevance latency KPIs); build automated eval pipelines tied to releases.
• Create PDLM deliverables: verification plans/protocols, test reports, risk-based coverage, VV trace matrices to requirements and use cases.
• Integrate test infrastructure with Azure DevOps (CI/CD, test orchestration, quality gates, dashboards); enforce build-time gates for API and LLM quality.
• Partner with Product, Clinical, and RD to refine requirements and acceptance criteria; ensure summaries render correctly for different user roles and workflows.

What is required from candidate

• 5+ years in backend/API test automation (REST, JSON, auth), performance testing, and CI/CD.
• Hands-on with Python/TypeScript, Postman/Newman or REST-assured, PyTest/Jest, k6/JMeter, Docker.
• Experience validating ML/LLM systems (eval datasets, metrics, prompt/response testing, safety/guardrails).
• Strong documentation and traceability discipline for regulated product lifecycles (PDLM/QMS).
• Bonus: Azure DevOps pipelines, FHIR/HL7 familiarity, healthcare data privacy/security.
• Bonus: Frontend testing capabilities are preferred, but not part of the PDLM requirements.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.