Humanities Expert - AI Evaluation

Role Overview

We are looking for Humanities Experts to join our team and contribute their subject-matter expertise to the development and evaluation of high-quality AI systems.

In this role, you will review questions and answers across a range of humanities topics, applying your academic knowledge, critical thinking, and attention to detail to assess the quality of AI-generated content. You will play an important role in helping ensure that AI systems produce responses that are accurate, relevant, coherent, well-reasoned, and aligned with high-quality standards.

Requirements

• +5 years of experience
• Bachelor’s degree or equivalent professional or academic expertise in a Humanities-related field, such as:
• -History
• -Philosophy
• -Literature
• -Linguistics
• -Languages
• -Cultural Studies
• -Sociology
• -Anthropology
• -Political Science
• -Religious Studies
• -Arts & Humanities
• -Or a related discipline
• Strong subject-matter knowledge and the ability to engage with a broad range of humanities topics.
• Excellent critical thinking, analytical reasoning, and written communication skills.
• Strong attention to detail and ability to identify subtle factual or logical issues.
• Ability to objectively assess the quality of written content and distinguish between strong and weak answers.
• Familiarity with AI training, data annotation, evaluation, or related AI data workflows.
• Ability to follow detailed guidelines and apply evaluation criteria consistently.
• Strong English proficiency, both written and reading.
• Ability to work independently and manage responsibilities in a remote environment.

Responsibilities

• Review and evaluate questions and answers related to humanities disciplines.
• Assess content for factual accuracy, relevance, clarity, completeness, and logical consistency.
• Identify inaccuracies, misleading statements, gaps in reasoning, ambiguities, and other quality issues.
• Apply subject-matter expertise to evaluate nuanced or complex topics.
• Judge whether responses appropriately address the question and fulfill the expected requirements.
• Provide clear and consistent evaluations based on established quality guidelines.
• Navigate topics across different areas of the humanities while maintaining strong analytical standards.
• Apply best practices in AI training data generation, annotation, and evaluation.
• Contribute to improving the quality and reliability of datasets used to train and evaluate AI models.

Education & Experience

• Bachelor’s degree or equivalent practical experience in any field.
• Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role is preferred but not required.

Benefits:

• Work from home

Application Question(s):

• Are you fine with this short term contract(1 Month)?
• Can you join immediately for this role?
• Please mention your LinkedIn URL
• Do you have experience(5+ YOE) with History/Philosophy/Literature/Sociology/Political Science or similar?

Experience:

• AI Evaluation: 5 years (Required)

Work Location: Remote