AI QA Engineer

AI QA Engineer - Fully Remote (EU or UK) - Professional Level English required

About the Company

Our client is an innovative AI native technology company transforming the construction industry through an AI-powered cloud platform. As they continue to build an AI-native organisation , they are looking for an AI QA Engineer to help shape quality across their product.

The Role

This is a hands-on role combining QA automation, AI/LLM testing and non-functional testing. You’ll help define testing strategy, build automation and establish the frameworks needed to test increasingly sophisticated AI and agentic features.

What you’ll do

• Own quality strategy across functional, non-functional and AI testing.
• Build automated performance, load and stress testing using tools such as JMeter, k6 or Gatling.
• Develop and contribute to LLM/AI evaluation frameworks, assessing accuracy, hallucination, relevance, safety and other quality metrics.
• Test AI and agentic behaviours, including non-deterministic outputs, prompts, model regressions and guardrails.
• Champion shift-left and continuous testing across engineering and product.
• Establish quality metrics, observability and production quality signals.
• Use AI-assisted tools to improve testing efficiency and coverage.

What you bring

• Strong QA foundations across test planning, functional, regression and exploratory testing.
• Practical experience or strong working knowledge of AI/LLM testing, including hallucination, consistency, safety and latency.
• Strong automation experience with Playwright, Cypress or similar, using TypeScript or Python.
• Hands-on experience with performance/load testing and CI/CD integration.
• Understanding of LLM evaluation, LLM-as-a-judge and frameworks such as DeepEval or RAGAS.
• Experience with API testing, microservices and cloud-native environments.
• A strong quality ownership mindset and ability to influence engineering and product teams.
• Professional level of English speaking

Why Join?

• Help shape quality within an AI-native organisation.
• Have genuine influence over quality strategy and engineering practices.
• Work on AI technology transforming the construction industry.
• Build new frameworks and processes rather than simply inheriting them.
• Join a collaborative, high-growth environment with competitive compensation and flexible working.

If you’re passionate about quality, excited by AI and want to help define the future of testing, we’d love to hear from you.