Lead AI Quality Engineer

About Phase2

Our remote first employees join and stay at Phase2 to be a force. In pursuit of digital experience that advances the human experience, you will do your best work, live fully, and succeed as a team.

Phase2 a digital agency passionate about customer experience, grounded in data & insights, and rooted in more than two decades of successful technology delivery. We guide industry-leading companies to what’s next in their digital landscape.

And it all begins with finding the right people who align with our values and passion for impact.

About the role

The Lead AI Quality Engineer owns the evolution of quality assurance at Phase2. Working closely with leaders within the Engineering Department, including other Quality Specialists, you will lead setting the vision for how Phase2 maintains a high level of quality in the software we produce, in an era when our engineers’ use of agentic AI has led to unprecedented rates of code production. You will also own setting and executing a roadmap that implements this vision: establishing consistent guidelines, methodologies, and tools that serve as the standard for quality assurance across our projects, and building a quality system capable of keeping pace with AI-accelerated delivery.

Your ownership of quality assurance spans two fronts: testing our AI-powered products and features, and applying AI-driven tools and techniques to test our legacy software systems. This includes establishing standards and guidelines for creating test plans for AI-powered features (LLM-based applications, agentic workflows, ML-driven components), and for using AI-powered testing tools responsibly with appropriate human oversight. Your ownership of quality strategy runs end-to-end, from requirements and design through production monitoring, across complex digital and AI-powered delivery engagements.

This is a hybrid role: allocated half-time to work on client projects, and half-time to overseeing and growing the Quality practice. Your direct work with clients involves leading quality strategy design and execution across one or more client delivery teams, presenting quality outcomes directly to client stakeholders, and coaching Phase2 project team members and client teams toward a quality ownership mindset. Your Quality practice oversight responsibilities include setting the vision and roadmap, executing on that plan, and managing, mentoring, and coaching other members of the quality team.

In collaboration with the VPs of Engineering, you will help socialize our approach to quality across other departments, including project management, strategy, sales, and marketing, ensuring that everyone has a shared understanding of the vision and how it is being executed.

This position reports to: VP, Engineering

Who You Are

• Quality strategist and technical leader: You’ve built your career hands-on in QA, and you’re ready to own quality strategy rather than just execute test plans. You can translate coverage decisions into a business narrative and advise clients directly.
• Systems thinker for quality: You design coverage models anchored to business-critical outcomes, and you’re hands-on, building automation frameworks and evaluation pipelines yourself.
• AI-native testing mindset: You bring a modern, AI-first testing approach to every project, including directing AI-powered testing tools, using prompt engineering to generate adversarial test scenarios, and evaluating LLM and ML outputs for accuracy, hallucination, and bias.
• Technical range and adaptability: You are resourceful across unfamiliar client stacks and delivery models, ramping up quickly on new tools and AI systems through hands-on exploration and collaboration with subject-matter experts.
• Client-facing consultant: You act as a quality consultant to client stakeholders, coaching engineering teams to adopt a quality ownership mindset and reporting outcomes in business terms.
• Mentor and practice builder: You manage and mentor the QA team, growing their skills, and bringing emerging AI testing methodologies back to the team proactively.

What you’ll do

• Set the vision and roadmap for software quality. Define the standards for how Phase2 ensures quality in the AI era, from how we write tickets to how we test our work. Evaluate our current suite of AI-enabled and traditional testing tools, and our current QA practices and methods, and maintain a roadmap that outlines planned improveme…