AI Specialist/Data annotation - Non voice
- Source: Naukri
- Tech Mahindra
- On-site
- Hyderabad
- Full-Time
- Senior
Job Description
Role Summary
The GenAI Specialist Rating evaluates AI-generated responses against defined quality rubrics
to determine whether responses meet gold-standard expectations. Specialists assess factual
accuracy, grounding, completeness, helpfulness, safety, and user satisfaction while providing
clear rationale for every decision.
Their work directly contributes to high-quality evaluation datasets used for model training,
benchmarking, and calibration.
Key Responsibilities
- Evaluate AI-generated responses using standardized evaluation rubrics
- Assess factual accuracy of geographic and local information
- Verify that responses are properly grounded in available map and place data
- Evaluate:
Factuality
Grounding
Helpfulness
Completeness
Relevance
Clarity
Safety
Instruction following
- Identify hallucinations and unsupported claims
- Distinguish between critical and minor defects
- Provide detailed justification for ratings
- Escalate ambiguous or policy-sensitive cases
- Maintain consistency across evaluation tasks
- Participate in calibration exercises
- Contribute feedback to improve evaluation guidelines
Core Skills
Generative AI Evaluation
- Understanding of LLM behavior
- Prompt-response evaluation
- Hallucination detection
- Response quality assessment
Analytical Thinking
- Critical reasoning
- Evidence-based decision making
- Attention to detail
- Pattern recognition
Maps Knowledge
- Geographic reasoning
- Navigation concepts
- POIs (Points of Interest)
- Local search behavior
- Route interpretation
Language Skills
- Excellent written English
- Reading comprehension
- Ability to interpret nuanced prompts
Secondary Skills
- Search verification techniques
- Knowledge of local businesses
- Travel and navigation terminology
- Familiarity with Maps products
- Basic data annotation experience
- Spreadsheet proficiency
Ancillary Skills [Good to know]
- SQL (basic)
- Annotation platforms
- Jira
- Documentation tools
- AI evaluation platforms
Qualifications
- Work Experience: 03 years in data annotation, content evaluation, technical writing,
or a related field. Experience with AI/LLM evaluation or editing is preferred.
- Educational Qualifications: Bachelor’s degree in Linguistics, English, Computer
Science, Humanities, or a related field, or equivalent practical experience.
Skills
- JIRA
- d3
