Applied AI Engineer (Contract)

Company Intro

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains - from doctors and lawyers to physicists and engineers - and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages . We are a well-funded startup with an enviable portfolio of clients including Anthropic , Amazon , Microsoft , Poolside , Recraft , and Shopify .

Recently, we secured strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin , CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA , UK , the Netherlands , Serbia , and more.

About position

Companies increasingly want their own models instead of paying for frontier APIs. A typical case: an enterprise client running a customer-support agent on a proprietary model wants a fine-tuned open-weight model that matches its quality at a fraction onef the cost. You’ll own these engagements technically, end to end.

You’ll sit down with the client’s team to understand the use case. You’ll recommend the right model, technique and data. Then you’ll build it: fine-tuning, evaluation and handover. Behind you is Toloka’s expert workforce and data platform, which produce the training and evaluation data.

This is a hands-on engineering role with real client exposure: roughly 60% building and experimenting , 40% working with clients . Sales owns the commercial relationship. You own the technical answer.

What you’ll do

Client discovery and solution design

• Lead technical discovery with client engineering and ML teams. Understand the use case, the current setup (e.g. a prompted frontier-model agent), constraints (cost, latency, privacy, deployment) and what “good” means to them.
• Ask the questions that shape the project: Is fine-tuning the right answer, or would better prompting or retrieval-augmented generation (RAG) do? Which base model and size? SFT, preference tuning, RL or distillation? What data is needed, how much of it, and from whom?
• Turn this into a clear proposal: approach, data plan, evaluation plan, success criteria, risks and timeline.
• Present results and recommendations to client stakeholders, from engineers to engineering leadership.

Hands-on fine-tuning and evaluation

• Fine-tune open-weight models (Qwen, Llama, Mistral, Gemma, GLM, DeepSeek) with SFT, LoRA/QLoRA, preference optimization (DPO and variants) and RL (PPO, GRPO, RL with verifiable rewards) where justified.
• Distill frontier-model behavior into smaller models: generate and filter teacher data, then fine-tune and close the quality gap.
• Fine-tune for agentic use: tool calling, multi-turn dialogue, following policies and procedures.
• Build evaluation that proves the result: baseline comparison against the client’s current model, task-specific test sets, LLM-as-judge and human evals, and regression checks.
• Optimize for production: quantization, serving with vLLM or similar, and cost-per-request and latency analysis.
• Define the data the project needs. Work with our solution engineers and expert workforce to produce it: SFT demonstrations, preference pairs, rubrics, edge cases.

What You Bring (Must-Have)

• 4+ years in ML engineering or applied ML research , with at least 2 years hands-on with LLM fine-tuning .
• You have personally fine-tuned open-weight LLMs on GPU infrastructure you managed, beyond hosted fine-tuning APIs. That includes multi-GPU training, LoRA/QLoRA and full fine-tuning.
• Hands-on experience with distillation or with preference or RL methods (DPO, PPO, GRPO or similar).
• Strong evaluation discipline. You can show convincingly whether a fine-tuned model is as good as the baseline, and you know when it isn’t.
• Practical knowledge of inference: serving, quantization, and the cost and latency trade-offs between model sizes and frontier APIs.
• Fluent with the stack: PyTorch and Hugging Face (Transformers, TRL, PEFT), plus some of Axo…