Required Skills:
- AI/ML Testing
- PyTest
- Test Automation
- Python
- LangSmith
- CI/CD
- API Testing
- Cloud Platforms
- SQL & Databases
Nice to Have:
Qualification
B.E or B.Tech Experience (in years)
8 Job Description
We are seeking a Lead Software Engineer – Automation Testing with 6–8 years of experience in designing, developing, and executing automation frameworks for AI/ML-driven applications. The ideal candidate brings hands-on experience with GenAI Testing, LangSmith, PyTest, Agentic frameworks, and strong AI/ML evaluation expertise. Key Responsibilities AI / GenAI Testing • Design and execute V&V strategies for classical ML and GenAI/ agentic systems. • Define test plans, tests cases, and execute them • Evaluate AI models for accuracy, performance, safety, robustness, hallucinations, bias, and correctness. • For GenAI, rely on LangSmith for LLM observability, tracing, monitoring, and evaluation, for offline and oline evaluation • For classical ML UC, ensure V&V tests are done during the whole life cycle of the model (ML monitoring) Test Automation • Build scalable automation frameworks using PyTest integrated with CI/CD pipelines. • Develop automated test suites for APIs, microservices, and AI pipelines. • Perform sanity, integration, functional, regression, and UAT testing across QA, UAT, and Prod environments. • Implement automation strategies: Keyword-driven, Data-driven, BDD, Hybrid frameworks. Quality Assurance & Reporting • Analyze test results and provide insights to engineering teams. • Log and track defects using standard bug-reporting tools. • Maintain high-quality documentation for all test processes and reports. Skill Set Required
• 6–8 years of experience in software testing & automation. • Strong programming expertise in Python. • Good understanding of Data Science related metrics (for both classical ML and GenAI UC)Experience with AI/Chatbot products and testing AI applications. • Experience with Git, GitHub Copilot, and CI/CD pipelines. • Knowledge of Azure/AWS cloud environments. • Familiarity with MySQL, PostgreSQL, Oracle, SQL Server, NoSQL databases. • Strong verbal & written communication skills. • Experience with multi-stakeholder delivery models including onshore–offshore. • Qualification: B.E./B.Tech Soft Skills • Analytical & Critical Thinking • Curiosity & Continuous Learning • Strategic Communication(Active Listening, Technical Translations to business impacts)
