QA Engineer - AI Agents

Required Skills & Experience

• Familiarity with LLMs and Agentic frameworks (OpenAI, Anthropic, LangChain/LangGraph)
• Strong hands-on QA/SDET experience in UI test automation and API test automation
• Hands on ability to build AI driven testing and verification workflows
• Experience using Playwright for AIdriven UI exploration
• Proficiency with Python and SQL (Postgres or SQL Server)
• Knowledge of DSPy or other promptoptimization / evaluation frameworks
• Background in red teaming, adversarial testing, or stress testing
• Exposure to vector databases (pgvector, Qdrant) and RAG systems
• Familiarity with data pipelines and orchestration tools (dbt, Airflow, Apache Hop)
• Experience with Docker, observability, or ML tooling (OpenTelemetry, Weights & Biases)

Nice to Have Skills & Experience

• Experience building AI Testing Agents i.e. Exploration Agents, Adversarial Agents

Responsibilities Include:

• Build and evolve an autonomous QA Swarm that tests the ERP UI, APIs, and financial workflows
• Design AI agents that explore the product, find broken flows, and validate reconciliations
• Replace brittle, manual tests with selfhealing, AImaintained test suites
• Create Judge / Eval Agents to assess agent accuracy, hallucinations, and data safety
• Reverseengineer legacy business logic and wrap it in modern, automated tests
• Stresstest systems using synthetic ERP data and adversarial scenarios
• Act as the final gatekeeper of production reliability for agentdriven features