Senior Quality Engineer - Playwright/Typescript/AI

Our client is seeking a Staff Quality Engineer (Playwright/Typescript/AI - LLMs) to support the quality and validation of AI-powered products and platforms. This role focuses on testing LLM-based applications, agentic AI systems, and Retrieval-Augmented Generation (RAG) workflows in production environments.

Job Title: Staff Quality Engineer (AI & Automation)

Salary: $130,000 - $150,000 Base + 15% Bonus

Location: Hoboken, NJ (Hybrid | 3 Days Per Week Onsite)

Required Skills & Experience

• 8+ years of Quality Engineering or Software Quality Assurance experience.
• Hands-on experience testing AI/LLM-powered applications in production environments.
• Experience testing agentic AI systems, multi-agent workflows, and RAG pipelines.
• Strong understanding of AI evaluation methodologies including RAGAS, LLM-as-Judge, regression baselines, and non-deterministic testing.
• Strong automation experience using Playwright or similar testing frameworks.
• Proficiency with TypeScript and/or Java.
• Experience with API testing, backend validation, and test data management.
• Experience building and maintaining automated test frameworks across UI, API, and backend layers.
• Experience designing and executing functional, regression, integration, and end-to-end testing strategies.
• Experience integrating automated testing into CI/CD pipelines.
• Experience leading quality efforts for complex software projects within Agile environments.

Preferred Qualifications

• Experience with vector databases, embeddings, or prompt engineering.
• AWS or cloud platform experience.
• Experience with AI-assisted development tools such as Cursor or GitHub Copilot.
• Experience with AI observability, monitoring, or evaluation platforms.
• Bachelor’s degree in Computer Science, Engineering, or a related technical field.

If interested in learning more, please apply directly or reach out to Ryan Dwyer at [email protected]