Senior QA AI Engineer

Responsibilities

• Lead quality engineering and artificial intelligence testing initiatives to ensure reliable, scalable, and secure software solutions across cloud-based platforms.
• Design, develop, and maintain automated test suites for Java-based backend services and RESTful application programming interfaces to ensure high software quality across releases.
• Own the test strategy for Anthropic Claude integrated features, including large language model output validation, regression baselines, tool-call accuracy verification, context management validation, and end-to-end integration coverage.
• Build and maintain continuous integration integrated evaluation frameworks for non-deterministic artificial intelligence outputs, including rubric-based scoring, automated prompt regression suites, and behavioral drift detection.
• Collaborate with product owners, software engineers, and machine learning engineers to define testable requirements, review acceptance criteria, and ensure high-quality delivery across agile releases.
• Perform database validation testing against PostgreSQL and Microsoft SQL Server environments to verify data integrity, transformation logic, and backend workflow accuracy.
• Conduct artificial intelligence specific security testing, including prompt injection, adversarial input validation, jailbreak testing, and sensitive data leakage assessments aligned with Moody s governance standards.
• Define and enforce synthetic or masked test data standards for artificial intelligence feature testing to ensure appropriate handling of confidential and regulated information.
• Monitor and report quality, performance, and artificial intelligence observability metrics including token consumption, streaming latency, tool-call success rates, model error rates, and reliability indicators.
• Integrate automated testing into continuous integration and continuous delivery pipelines, ensuring quality gates and deployment controls are effective and actionable.
• Conduct test code reviews, mentor quality engineering team members, perform exploratory testing, and drive continuous improvement of testing practices and processes.
• Support performance and load testing of cloud-native software-as-a-service platforms on Amazon Web Services and Microsoft Azure to validate scalability, availability, and reliability requirements.
• Maintain clear test documentation, traceability matrices, defect analysis reports, and quality metrics to ensure transparency and alignment across stakeholders.

Education

• Bachelor or Master of Science in Computer Science, Mathematics, Physics, or a related field

Skills and Competencies

• 5-7 years of experience in software quality engineering with hands-on expertise testing Java-based backend services and RESTful application programming interfaces
• Proficiency with test automation frameworks such as Selenium, TestNG, JUnit, RestAssured, or Cucumber for user interface and application programming interface testing
• Strong understanding of relational databases including PostgreSQL and Microsoft SQL Server to perform data validation and verify data integrity across test scenarios
• Hands-on experience testing Anthropic Claude integrated features, including large language model output validation, semantic evaluation techniques, hallucination detection, and streaming application programming interface behavior
• Familiarity with Model Context Protocol and agentic artificial intelligence testing patterns, including tool-use accuracy, context management, and deterministic versus non-deterministic output validation
• Experience with artificial intelligence security testing, including prompt injection testing, jailbreak validation, sensitive data leakage checks, and application reliability and performance assessments
• Strong understanding of software design principles and microservices architectures, with the ability to develop effective test strategies and identify system-level edge cases
• Deep expertise in artificial intelligence, with a track record of implementing advanced AI solutions to drive strategic transformation and operational efficiency. Strong experience using AI tools to lead innovation initiatives. Demonstrated leadership in managing AI-related risks, ensuring ethical governance, and fostering a culture of responsible AI adoption across the organization.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability…