AI Systems & Evaluation Engineer Intern

About WalletGyde

WalletGyde is building a Financial AI platform designed to help people better understand, manage, and improve their financial lives through trustworthy, explainable, and personalized financial intelligence.

As we continue evolving our Financial AI architecture, one of our highest priorities is making sure the system does more than generate convincing responses. We want to understand whether every financial answer is correct, grounded in the right data, produced through the right logic, and communicated responsibly.

Over the past year, our team has built a growing reliability infrastructure around our Financial AI, including deterministic financial calculations, Ground Truth datasets, regression testing, financial personas, Guardrails, Tone & Trust standards, QA workflows, failure classification, and what we refer to internally as the WalletGyde Financial Harness.

We are now looking for an AI Systems & Evaluation Engineer Intern to help us strengthen and evolve these systems.

The Role

This is not a traditional machine learning internship focused primarily on model training.

You will work across the systems surrounding modern AI models to help WalletGyde answer one critical question:

How do we know WalletGyde produced the right financial answer, for the right reason, with the right behavior?

You will help investigate where AI failures originate, improve how we evaluate system performance, strengthen our Ground Truth methodology, build regression coverage, and help convert manual evaluation processes into more repeatable engineering systems.

This role sits at the intersection of AI engineering, evaluation, financial logic, QA, data analysis, and responsible AI.

What You Will Work OnGround Truth & Financial Accuracy

You will help audit, validate, version, and expand WalletGyde’s Ground Truth datasets and expected financial answers.

This includes reviewing how expected answers connect to approved financial definitions, formulas, time periods, and deterministic calculation logic.

AI Evaluation

Help improve how WalletGyde measures AI performance using a combination of:

Deterministic evaluators

Model based graders

Expected versus actual comparisons

Financial accuracy testing

Behavioral evaluation

Human review

You will help us move beyond simply asking whether a chatbot response passed or failed and toward understanding why a response succeeded or failed.

Failure Analysis

Investigate failures across multiple parts of the AI stack, including:

Financial data

Date and period logic

Deterministic calculations

Retrieval and RAG

Orchestration and routing

Guardrails

Model responses

Evaluation methodology

The goal is to determine the actual root cause rather than assuming every incorrect response is an LLM problem.

Regression Testing

Help maintain and expand WalletGyde’s regression suite as new edge cases and failures are discovered.

You will work with existing test cases and help ensure engineering improvements do not unintentionally break previously correct behavior.

QA Engineering

Help convert portions of our existing manual QA methodology into repeatable engineering workflows.

This may include test runners, structured evaluation outputs, regression reports, automated checks, and eventually CI/CD evaluation gates.

Guardrail Evaluation

Help build executable tests around behaviors such as:

Uncertainty

Unsupported financial claims

Clarification

Privacy

Financial boundaries

Hallucination prevention

Responsible redirection

Safe handling of incomplete information

Tone & Trust Evaluation

WalletGyde has developed a Tone & Trust framework designed around principles such as:

Never Shame

Never Pretend Certainty

Answer First

Encourage the User

Connect Insight to Action

You will help explore how qualitative standards like these can become measurable, reproducible evaluation criteria.

Financial Harness Engineering

Support the broader WalletGyde Financial Harness connecting:

Financial Data → Deterministic Logic → Ground Truth → Retrieval → AI Reasoning → Guardrails → Tone → Product Response → Evaluation

You will help us understand how these components interact and how failures can be traced through the system.

Evaluation Automation

Work alongside WalletGyde engineers and external technical partners to help move portions of our evaluation methodology into automated workflows, regression pipelines, observability systems, an…