Retail SME - AI Evaluation Expert
- Source: LinkedIn
- Mercor
- Remote
- New York City
- Senior
Job Description
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D’Angelo , Larry Summers , and Jack Dorsey .
Position: Retail Specialist
Type: Contract Compensation: $60–$80/hour Location: Remote Role Responsibilities
- Guide research and engineering teams to close knowledge gaps in retail merchandising, category management, and operations reasoning.
- Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
- Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
- Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
- Collaborate with other subject matter experts to ensure consistency and accuracy in training data.
Qualifications Must-Have
- 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
- Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria — mandatory; please describe this experience in your application.
- Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising).
- Ability to engage reliably for at least 35 hours/week during weekdays.
- Verbal and written communication skills, problem-solving skills, and interpersonal skills.
Compensation & Legal
- W-2 employment with Cincinnatus LLC
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: [email protected]
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Skills
- benchmark
- catalyst
