Senior AI Data Annotation Jobs in Atlanta

Senior AI Data Annotation Specialist (Atlanta, Remote) Rex.zone is hiring a Senior AI Data Annotation Specialist to create, review, and audit high-quality training data used for modern AI systems, including LLMs and multimodal models. You will execute complex annotation tasks, run QA evaluation workflows, and support RLHF through ratings, rankings, and critiques that improve model performance.

What You’ll Work On

• Large language model evaluation and prompt evaluation
• RLHF preference ranking, instruction-following scoring, and rubric-based critiques
• NLP annotation (e.g., named entity recognition, entity linking, text classification)
• Computer vision annotation (bounding boxes, polygons, keypoints) and multimodal checks
• Content safety labeling and policy-based evaluation
• Dataset error analysis and training data quality audits

Responsibilities

• Produce accurate labels while maintaining annotation guidelines compliance
• Perform QA evaluation, second-pass reviews, and sampling-based audits
• Calibrate with reviewers to maintain consistency and inter-annotator agreement
• Document edge cases, resolve disagreements, and propose guideline updates
• Partner with data operations and engineering to improve tools and workflows and report systematic failure modes observed during evaluations

Qualifications

• Experience delivering high-volume, high-precision AI data annotation
• Strong English comprehension and writing for prompt evaluation and critique
• Familiarity with RLHF concepts (pairwise ranking, preference data)
• Experience with QA evaluation processes and reviewer calibration
• Comfort with NLP and computer vision annotation tools and concepts

Work Model Remote, full-time role aligned to Atlanta talent, collaborating with distributed teams across the US.

Compensation Hourly base pay range: $30–$50 per hour (USD), depending on scope and demonstrated performance.