Director, Enterprise Quality Engineering and Agentic Evaluation

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Director, Enterprise Quality Engineering and Agentic Evaluation based in United States.

This is a strategic technology leadership role focused on building and leading a modern enterprise quality engineering capability.
You will shape how quality, reliability, and release assurance are embedded across the software delivery lifecycle and critical business platforms.
The role combines automation-first quality engineering with risk-based governance, helping improve delivery speed while reducing production risk.
You will also define how emerging agentic AI solutions are evaluated, validated, monitored, and governed for enterprise-scale adoption.
Working across Engineering, Product, Architecture, Data & AI, Operations, and business teams, you will establish measurable standards for engineering excellence.
The position offers significant influence over quality strategy, technology platforms, testing practices, and organizational operating models.
You will lead teams and partners while providing independent quality intelligence and risk assessments to senior technology and business stakeholders.

Accountabilities:

• Define and lead the enterprise quality engineering strategy, operating model, roadmap, service catalog, standards, and governance framework across the technology organization.
• Own the enterprise quality engineering platform, including test automation and management capabilities, automation assets, integrations, test execution infrastructure, quality reporting, and AI evaluation capabilities.
• Establish risk-based quality models, release-readiness criteria, quality gates, certification standards, and evidence requirements for critical platforms, business processes, integrations, data products, and AI agents.
• Lead a centralized quality engineering service responsible for planning, automating, executing, and governing testing across enterprise products, platforms, transformation programs, and high-risk releases.
• Develop reusable automation frameworks, model-based testing patterns, API and integration testing capabilities, regression suites, and continuous testing practices aligned with business risk and production impact.
• Establish enterprise capabilities for evaluating AI agents, including scenario design, evaluation datasets, regression testing, tool-use validation, policy and permission checks, human escalation criteria, and production quality monitoring.
• Own the enterprise approach to test data, test environments, service virtualization, production validation, and quality analytics to reduce release risk and minimize delivery delays.
• Provide independent quality intelligence, release-risk assessments, and actionable recommendations to engineering, product, program, and executive stakeholders.
• Manage vendor-outsourced quality engineering resources, technology partners, systems integrators, and specialist testing providers to deliver consistent enterprise quality outcomes.
• Build a quality engineering culture centered on automation, objective assurance, reuse, engineering discipline, measurable outcomes, and continuous improvement.

Requirements

• Bachelor’s degree in Computer Science, Engineering, Information Systems, Mathematics, or a related discipline; equivalent relevant experience may be considered.
• 12+ years of experience in quality engineering, software testing, test automation, platform engineering, enterprise technology delivery, or related technology disciplines, including 5+ years leading quality engineering, test automation, enterprise testing, or platform-oriented teams.
• Demonstrated experience building or leading modern quality engineering capabilities beyond manual testing, including automation strategy, risk-based testing, continuous testing, quality gates, and release assurance.
• Strong experience with enterprise test automation and test management platforms, preferably Tricentis Tosca and qTest.
• Proven experience leading testing or quality engineering across multiple products, platforms, business processes, programs, or business units.
• Strong understanding of API, integration, data, workflow, cloud, event-driven, distributed-system, performance, resilience, and production validation testing.
• Practical knowledge of generative AI, large language models, retrieval, AI agents, tool calling, evaluation methodologies, …