AI Automation Test Engineer

Job Description:

AI Automation Engineer
Location: Pune, Mumbai, Bengaluru

Experience - 5-8 Years

Role Overview

The QA Automation Engineer sits within the Digital Product team responsible for building Dentsu’s global advertising QA product used across markets to ensure media accuracy and operational excellence.

This role will focus on improving, expanding, and maintaining automated test coverage across the Pipes platform. The QA Automation Engineer will work across three engineering squads, partnering with developers and product managers to ensure we deliver stable, high‑quality functionality at speed.

This is a technical automation role, not a manual QA / test analyst position. The engineer will be responsible for writing automated tests, maintaining our testing frameworks, and contributing to a robust CI pipeline.

Given the level, the role is expected to work with guidance from senior engineers, while gradually taking ownership of the test automation suite. They will be working alongside and as part of the development team, and have a great deal of autonomy to influence the development team’s testing processes.

Key Responsibilities

Test Automation & Technical Delivery

• Google Vertex AI / Gemini integration testing - can hit live endpoints, validate responses, handle auth/rate limits, parse structured outputs.
• LLM output validation - design test cases that detect regression in model behaviour (consistency, edge-case handling); script adversarial/prompt-injection tests against known threat patterns.
• Responsible for setting up any AI testing frameworks and automation needed.
• Develop, improve, and maintain automated test suites across frontend and backend components.
• Write high‑quality, maintainable automated tests that support rapid delivery and reliability.
• Identify and implement a test coverage tool to monitor coverage.
• Own the execution and reporting of automated test pipelines within CI/CD workflows.
• Monitor test flakiness and stability; proactively diagnose root causes and implement fixes.
• Contribute to discussions on testing strategy, tooling, and testability improvements across the product.
• Ensure all automated testing follows best practice, is version‑controlled, and integrates cleanly into engineering workflows.
• Design and implement E2E testing to simulate real user actions and validate the full workflow.

Collaboration & Ways of Working

• Work day‑to‑day with developers across three squads to embed quality into feature development.
• Participate in squad ceremonies (stand‑ups, refinement, planning), providing input on test impact and automation needs.
• Collaborate with product managers and engineering leads to understand feature requirements and identify test scenarios.
• Work within an agile, cloud‑first engineering environment and contribute to continuous improvement of testing processes.
• Document and track bugs encountered, providing detailed reproduction steps, environment details and visual documentation to facilitate efficient resolution.

Learning & Growth

• Learn from developers to deepen technical skills.
• Stay current with modern test automation frameworks, patterns, and tools, recommending improvements where valuable.
• Contribute to documentation of test suites, frameworks, and best practices.

Required Experience & Skills

• 5+ years’ experience in QA automation or software engineering roles with a testing focus.
• Google Vertex AI / Gemini integration testing .
• LLM output validation
• Experience developing automated tests for web applications (frontend or backend).
• Familiarity with at least one E2E / Browser Automation framework (e.g., Cypress, Playwright, Selenium) and one Unit Framework (xUnit, Jest, NUnit).
• Ability to read and write code in at least one programming language (JavaScript/TypeScript, C#, Python, or similar).
• Understanding of CI/CD pipelines and automated test execution.
• Ability to work within an agile engineering environment.

Desirable / Nice-to-Have Skills

Experience with some of the following is beneficial but not required:

Backend Testing

• xUnit or similar unit testing frameworks for .NET environments.
• API / integration testing methods and tools.

Frontend Testing

• Cypress, Playwright, or similar browser automation tools.
  (We currently use Cypress but are open to modernising our approach.)

**Gene…