Senior Automation QA Engineer – AI & Agentic Development

Senior / Staff Automation QA Engineer – AI & Agentic Development

Hands-on senior+ IC role ~70% engineering Automation, AI evaluation & agentic quality

Company Description

SANNOS is an intelligence platform for global compliance, providing automated assessment and risk generation for complex regulatory environments. Trusted by Big 4 firms and major institutions, SANNOS streamlines audit and compliance reviews from days of manual work to minutes. The company was founded by experienced Risk & Compliance professionals who built the solution they needed on the frontlines of highly regulated industries. SANNOS helps organizations treat compliance as a strategic advantage, bridging executive judgment with advanced computing. The platform supports frameworks such as ISO27001, DORA, NIS2, NIST, CMMC, Secure Control Framework (SCF), and CSRD.

The Role

We’re looking for an experienced Senior / Staff Automation QA Engineer (SDET) who combines strong software engineering and test automation skills with practical experience using AI and coding agents in real engineering work .

This is not a traditional manual QA role.

You’ll spend most of your time building automation, improving quality engineering tools, testing AI-powered features, creating evaluation frameworks, debugging complex failures, and using AI agents as part of your engineering workflow .

We care less about whether you’ve experimented with the latest AI tool and more about what you’ve actually built, how you used it, and what improved as a result.

The position is posted as a Senior / Staff band . Final level will be determined during the interview process based on your technical depth and the scope you have previously owned.

What You’ll Do

• Spend approximately 70% of your time hands-on engineering automation code, AI evaluations, debugging, tooling and technical reviews.
• Build and maintain automated testing for APIs, web applications, integrations and end-to-end business flows .
• Integrate quality gates into CI/CD pipelines while keeping feedback fast and useful for developers.
• Use LLMs and coding agents to write and review test code, analyse logs, create test data and investigate failures.
• Build repeatable AI-assisted engineering workflows , rather than relying on one-off prompts.
• Test AI-powered features including prompts, tool calls, multi-step agent workflows and failure recovery .
• Create evaluation datasets, scoring approaches and automated checks for AI quality, safety and reliability .
• Test for risks including hallucinations, prompt injection, sensitive-data exposure, incorrect tool use and unsafe agent actions .
• Track and improve engineering quality through metrics such as automation coverage, execution time, flaky-test rate, escaped defects and AI evaluation pass rates.
• Mentor other engineers through reviews, pairing, technical demonstrations and shared engineering standards.

What We’re Looking For

You have:

• 8+ years of experience in software QA or quality engineering.
• 6+ years designing and maintaining production-grade test automation frameworks that you built or substantially owned.
• Strong software development skills in Python, Java, JavaScript/TypeScript, C# or a comparable language .
• Deep practical experience with at least one modern automation framework such as Playwright, Cypress, Selenium, Appium, pytest or RestAssured .
• Experience testing APIs, databases, integrations and end-to-end workflows .
• Experience embedding automated quality checks into CI/CD .
• Regular, hands-on experience working with LLMs or coding agents in engineering .
• At least one delivered AI or agentic testing initiative, such as an LLM evaluation pipeline, AI-assisted automation workflow or QA ownership of an AI-powered product .
• A practical understanding of LLM risks including inconsistent output, hallucinations, prompt injection, data exposure and unsafe tool use.

For the AI part of the role, evidence matters more than years.

We’ll be interested in examples of things you have personally built or substantially improved, such as:

• A significant automation framework and the scale it supported.
• Measurable improvements in test runtime, reliability, release speed or escaped defects.
• An AI-assisted engineering workflow you designed and actually used.
• How you evaluated the quality o…