AI LLM Dataset Engineer Senior

Role & responsibilities

The AI/LLM Dataset Engineer (Senior) is the technical lead for the golden dataset and QA design workstreams on the Cleris Pulse NLQ Evaluation Framework. This role owns the seeded synthetic data generation pipeline, statistical distribution engineering, Jinja2 QA template library, and all schema documentation. The engineer coordinates two Domain SMEs and the API Integration Engineer on a day-to-day basis, reporting to the Engagement Lead.

Preferred candidate profile

Synthetic Data Pipeline

• Design and implement the seeded data generation pipeline: numpy.random.default_rng(seed=42) + Faker 24.x + pandas 2.x
• Engineer realistic statistical distributions: Pareto =1.16 for revenue, Poisson =3.2 for event frequency, Gaussian mixture for date clustering
• Implement controlled imperfection injection: ~1% duplicate rows, 23% NULLs (Bernoulli p=0.025), 0.5% outlier records, boundary date values
• Run FK integrity gate: DuckDB PRAGMA foreign_key_check must pass before any CSV file is exported
• Produce SHA-256 manifest.json per dataset and manage Git LFS versioning (tag: dataset-v1.0.0)

QA Design & Verification

• Build the Jinja2 SQL template library for all 5 question tiers (T1–T5) per domain
• Verify all expected_answers by executing reference SQL against the golden dataset in DuckDB before inclusion
• Collaborate with Domain SMEs on natural-language question phrasing and business context accuracy
• Author judge_reference answers for each QA pair to support LLM-as-Judge calibration

Schema & Documentation

• Produce ER diagrams for all 5 domains using dbdiagram.io; present for TechM/Cleris approval
• Draft DDL SQL per domain; run SQLFluff linting before presenting for stakeholder review
• Produce the CSV Header Specification document for joint sign-off with Cleris
• Produce Data Dictionaries (field name, type, description, sample values, constraints) for all domain tables