QA Engineer – AI Evaluation & Software Testing

Company: Tasknera

Location: Remote (India)

Position Type: Full-Time Contract (Bench Resources Only)

Pay Rate: ₹900/hour (Up to ₹1.27 LPA / Month ~₹15.24 LPA)

Experience Required: 7+ Years

Contact: [email protected]

Job Summary

We are seeking experienced QA Engineers to support a high-volume software quality evaluation project focused on AI training and human-in-the-loop evaluation.

This role requires strong hands-on experience in manual and automated software testing, combined with mandatory prior paid experience in AI response evaluation, RLHF, model output evaluation, or human data annotation. Traditional software QA experience alone will not qualify for this position.

Key Responsibilities

• AI Output Evaluation: Evaluate and rate AI-generated technical outputs against defined quality criteria, guidelines, and complex evaluation rubrics.
• Accuracy Assessment: Identify responses that appear technically correct but contain subtle inaccuracies, omissions, logical flaws, or incomplete information.
• Software Testing Execution: Design, execute, and assess comprehensive functional, regression, negative, boundary, and edge-case test scenarios.
• Defect & Bug Management: Identify, isolate, and document software defects with clear, concise, and reproducible steps in structured bug-tracking systems.
• Technical Feedback: Provide detailed, actionable written feedback and annotations for developers without requiring additional clarification.
• Process Improvement: Collaborate with project teams to refine evaluation rubrics, testing guidelines, and QA methodologies.

Required Qualifications & Experience

• 7+ years of professional experience in QA Engineering, Software Testing, SDET, or Test Engineering.
• Mandatory Experience: Demonstrated paid professional experience in at least one of the following:
• AI / LLM Response Evaluation
• RLHF (Reinforcement Learning from Human Feedback)
• Rubric-Based Grading for AI Training
• Human Data Annotation / Data Labeling
• AI Training Data Quality Evaluation
• Testing Expertise: Solid foundation in test case design, functional, regression, negative, and boundary testing.
• Automation: Hands-on experience with test automation frameworks (Selenium, Playwright, Cypress, or Appium).
• Communication: Excellent written English (B2+ level) to write clear technical feedback and bug documentation.
• Profile Verification: A valid LinkedIn profile is mandatory.

Preferred Skills & Tools

• API & Defect Management Tools: Postman, Jira, TestRail, Zephyr, BrowserStack.
• Strong analytical, critical-thinking, and problem-solving skills with high attention to detail.

Verification & Onboarding Process

• Interview: 1 Round of Technical Evaluation.
• ID Verification: Shortlisted candidates must scan a provided QR code to upload a selfie and a valid government-issued ID proof prior to onboarding.
• Background Verification (BGV): Not required.

How to Apply

Interested candidates or vendors with qualified bench resources should send an updated resume along with their LinkedIn profile link to [email protected]

Pay: Up to ₹1,500,000.00 per year

Application Question(s):

• Do you have 7+ years of total professional experience in Software QA, SDET, or Test Engineering?
• Do you have prior paid professional experience specifically in AI evaluation, LLM response rating, RLHF, or human data annotation?
• Are you currently a bench resource available to join immediately as a contractor?
• How do you identify subtle technical inaccuracies, logical flaws, or omissions in an AI-generated output that otherwise looks correct?
• Do you have hands-on experience using test automation frameworks like Selenium, Playwright, Cypress, or Appium?
• List the automation frameworks (e.g., Selenium, Playwright) and bug-tracking tools (e.g., Jira, TestRail) you are most proficient with, along with your notice period and expected hourly rate.

Work Location: Remote