QA AI Engineer

Role & responsibilities

• Strong experience in Automation, functional, integration, regression, and API testing.
• Experience with API testing using Postman, REST Assured, SoapUI, Swagger, or similar tools.
• Experience testing REST APIs, WebSockets, event streams, and asynchronous workflows.
• Basic SQL knowledge for data validation.
• Understanding of Databricks, Delta tables, data pipelines, or cloud-based data platforms.
• Familiarity with RAG, LLMs, embeddings, semantic search, knowledge graphs, and AI application testing.
• Ability to validate AI responses against source grounding, expected business rules, and trace evidence.
• Experience testing rule-based workflows, exception workflows, state machines, or case management systems.
• Understanding of authentication, authorization, role-based access, data privacy, and audit validation.
• Experience with test management and defect tracking tools such as Jira, Azure DevOps, or similar.
• Good analytical, communication, and documentation skills.

Preferred candidate profile

• Experience testing applications on Azure Databricks, Unity Catalog, Mosaic AI, or MLflow.
• Exposure to EDI/X12, healthcare transactions, pharmacy, payer, provider, claims, pricing, or drug data workflows.
• Exposure to vector search, graph databases, context graphs, triple stores, or decision trace stores.
• Experience with automated API testing and Python-based test automation frameworks such as PyTest.
• Experience validating model outputs, prompt changes, agent workflows, and retrieval quality.
• Familiarity with FDA drug data, DEA registry validation, NADAC/GPO pricing, EPCIS, VRS, or serialization workflows.
• Familiarity with CI/CD validation, smoke testing, release certification, and production sanity testing.
• Knowledge of observability tools such as Grafana, Prometheus, OpenTelemetry, Azure Monitor, or Log Analytics.
• Understanding of healthcare, pharma, or regulated enterprise environments is a plus.