AI Data Annotator, Machine Learning

About ProtocolClear

ProtocolClear is an independent, principal-led consultancy in security, privacy, AI governance, and compliance. We operate an in-house, Lagos-based annotation team that produces high-quality human judgment for AI training. This role fills a specific gap: we need annotators who actually understand machine learning and can read and reason about Python so that we can label and evaluate ML-domain data correctly. Security made clear is how we work, and on this stream, the same standard of precision applies to data science.

About the role

This is specialized, domain-expert annotation for machine learning and data science content used to train and evaluate AI systems. You will judge whether a model’s output, code, or reasoning is correct, perform RLHF-style preference and quality judgments, evaluate Python and data-handling tasks, label model behavior against a rubric, and write short rationales for each decision. The founder, an AI security architect, supplies the rubric. Your job is to apply real machine learning and Python knowledge to that rubric consistently and defensibly. Note for candidates: this stream is launching on a rubric-based model (higher per-task pay and longer estimated time per task) and will be restructured once the client’s exact terms are confirmed.

The standard your work is held to

Your work is held to a five-star client standard, and on machine learning content, the bar for technical accuracy is unforgiving. A response that sounds right but is technically wrong, a buggy code judgment, a confused metric, a mislabeled preference, is an error, and errors are rejected, not paid, traced to the annotator, redone on your own time, and penalized. You are expected to tell a genuinely correct, well-reasoned ML answer from a confident, plausible, wrong one. You must read Python accurately, reason about model behavior and evaluation metrics correctly, and never let fluent wording substitute for correctness. When the rubric is genuinely ambiguous on an edge case, you flag it rather than guessing.

What you will do

• Evaluate model responses to machine learning, data science, and Python questions for technical accuracy and completeness.
• Read and assess Python code for correctness, bugs, and expected output.
• Perform pairwise and rubric-based preference judgments (RLHF) on model outputs.
• Apply quality and harm labels against the supplied taxonomy.
• Judge whether a model’s reasoning chain is valid, not only whether the final answer is right.
• Write short rationales tying each judgment to the rubric.
• Maintain agreement with other annotators.

What we are looking for (required)

• A real machine learning or data science background, through hands-on or studied knowledge of core ML concepts, model evaluation, and data handling.
• Working ability to read and reason about Python, including common libraries such as NumPy and pandas.
• The ability to tell technically correct ML content from a plausible-sounding error.
• Familiarity with RLHF or model-evaluation concepts at a practitioner level, or the ability to learn the taxonomy quickly and apply it precisely.
• Strong reasoning and English comprehension.
• Discipline to apply a rubric consistently.

Nice to have (preferred)

• Hands-on Python development or data science project experience.
• Familiarity with ML frameworks such as scikit-learn, PyTorch, or TensorFlow.
• Prior RLHF, red-teaming, or model-evaluation experience.
• Experience with inter-annotator agreement, gold tasks, and adjudication.
• A statistics or quantitative background.
• Experience writing technical rationales or findings.

Compensation

The package combines a monthly base, per-task output pay for accepted work, and performance kickers for sustained quality. Because this is specialized, domain-expert work, the range sits above general annotation. Provisional realistic monthly earnings range from about ₦600,000 for light-accepted output to about ₦1,500,000 for sustained full output at five-star quality. This range is provisional and will be revised once the client’s exact per-task terms are received. The first three months are a paid trial at per-task rates plus kickers.