Senior AI Data Annotation Jobs in Seattle

Role Summary Rex.zone is hiring for a remote, full-time Senior AI Data Annotation role supporting training-data quality for modern AI/ML systems. You will deliver high-precision human feedback across RLHF, LLM evaluation, prompt evaluation, and QA evaluation to drive measurable model performance improvement.

Key Responsibilities

• Deliver senior-level AI data annotation across text, image, and mixed-modality tasks
• Execute RLHF workflows including preference ranking, pairwise comparisons, and (when required) rationale capture
• Perform rubric-based LLM evaluation and QA evaluation for helpfulness, honesty, harmlessness, and policy compliance
• Run prompt evaluation to identify failure modes, jailbreak risk, and instruction-following gaps
• Apply strict annotation guidelines compliance; report ambiguities and propose guideline improvements
• Conduct error analysis, label auditing, calibration, and consensus reviews to stabilize annotation quality
• Support content safety labeling (toxicity, self-harm, violence, hate, regulated content) with structured QA review
• Collaborate with engineers/data ops on task design, schema updates, and quality monitoring signals
• Document edge cases clearly and escalate issues that impact training data quality

Required Qualifications

• Mid-Senior experience in AI data annotation, data labeling, and/or production LLM evaluation
• Understanding of RLHF concepts and how human feedback improves model alignment
• Proven ability to follow and improve annotation guidelines with high attention to detail
• Experience with QA evaluation methods (audits, gold sets, calibration, inter-annotator agreement)
• Familiarity with NLP labeling (classification, sentiment, NER) and/or computer vision annotation
• Excellent written communication for documenting decisions, rationales, and edge cases
• Ability to work remotely within the US and handle sensitive data securely

Compensation

• Hourly pay range: $30–$50 per hour (USD)

How To Apply Apply via Rex.zone and highlight relevant AI data annotation, RLHF, LLM evaluation, prompt evaluation, and guideline-driven QA evaluation experience.