Senior Software Engineer - AI Code Review - Technical Advisor

Our well-know AI client is seeking exceptional Sr Software Engineers - Technical Advisors who want to spend a focused period doing something most engineers never get to do: find out exactly where frontier models break on real engineering work.

You will take on hard problems across production-grade codebases, then work out precisely where and why model-generated solutions fall short. You will also build the hard problems those models are tested on. Your judgment about what separates good engineering from plausible engineering is the entire point of the role.

What you’ll do

• Evaluate Agent Sessions: Analyze how AI coding agents behave over entire coding sessions—tracking what the agent investigates, verifies, assumes, and leaves undone.
• Review Model-Written Pull Requests: Audit model-generated pull requests against real production repositories, documenting every identified issue alongside its severity and detailed technical rationale.
• Build Evaluation Benchmarks: Design and construct hard, container-based problems that serve as rigorous test benchmarks for frontier models.
• Collaborate with AI Researchers: Work directly alongside AI researchers on frontier problems, producing clear written analyses that explain root-cause failures and boundary-condition gaps.
• Maintain High Written Rationale Standards: Author original, clear technical rationale for all evaluations (all written deliverables must be independently authored without AI text generation, though AI tools are welcomed for codebase exploration and running test suites).

This isn’t a typical “write code and ship features” role. The client is focused on evaluating how frontier AI coding models perform on real, production-grade engineering work, and they need experienced engineers who can act as a high-bar technical authority: reviewing model-generated code, figuring out precisely where and why it breaks, and explaining that reasoning clearly in writing. Your judgment about what separates genuinely good engineering from plausible-but-superficial engineering is the entire point of the role.

What you’ll actually do:

• Review model-generated pull requests against real production repositories, documenting every issue you find along with its severity and your technical rationale.

• Evaluate full AI coding-agent sessions, tracking what the model investigated, verified, assumed, or skipped.

• Design and build container-based (Docker) test problems that serve as benchmarks for evaluating frontier models.

• Write clear, original technical rationale explaining why code fails. All written deliverables must be self-authored, no AI text generators for your written analysis, though AI tools are welcome for exploring codebases or running test suites.

• Collaborate asynchronously with Anthropic’s AI researchers to share findings and refine evaluation criteria.

Who this is a strong fit for:

- Must have 8+ years of hands-on production engineering experience (Senior, Staff, Principal, Tech Lead, or Open-Source Maintainer backgrounds all fit well)

• Real experience in codebases with a genuine code review culture, whether at a startup, big tech, or open source

• Comfortable dropping into unfamiliar languages and codebases on short notice

• Genuinely comfortable with Docker, git, and the command line to reproduce and debug issues locally

• Can work across backend, frontend, APIs, data, testing, or dev tooling

• Write clearly enough to explain why something is broken, not just how to fix it fast

Not the right fit if: you’re an Engineering Manager/Director who’s stepped away from hands-on code review, your experience is limited to no-code/low-code or single-framework MVP work, you’re a QA tester or non-technical prompt engineer, or you don’t have real Docker and terminal/CLI experience.

Contract type: W2 Contractor. This role cannot support C2C arrangements or visa sponsorship, so you’ll need to confirm US remote work eligibility.

Who you are

Are comfortable dropping into unfamiliar codebases and languages you don’t use every day?

the repos change weekly!

Are comfortable with containers and reproducing results locally?

Can explain your reasoning as clearly as you can write the solution?

Are more interested in why a solution is right than in shipping it quickly?

Want to spend a focused period on this rather than committing to a permanent role?

Daily tasks

Solve difficult engineering problems across real, production-grade codebases

Ident…