Multimodal AI / Computer Vision Engineer — Video-to-Training-Data Pipeline

Multimodal AI / Computer Vision Engineer — Trading Education Data Pipeline

We are building an AI system designed to learn discretionary price-action trading from expert educational videos.

We need an engineer to build a pipeline that converts narrated trading-course videos into structured, machine-readable training examples by synchronizing:

• Video frames
• Course transcripts
• Mouse/pointer movement
• Candlestick charts
• Chart annotations and text
• Individual price bars
• Expert explanations and trading decisions

Core Problem

The instructor teaches using chart slides containing candlestick bars, EMA lines, annotations, arrows, and text. During the video, he moves the mouse over specific bars or chart regions while saying things such as:

• “This bar”
• “These bars”
• “This breakout”
• “This pullback”
• “This second entry”
• “This wedge”
• “I would buy here”
• “I would not take this trade because…”

The charts usually do not contain explicit OHLC data or bar IDs.

The goal is to determine exactly which chart bars or regions the instructor is referring to and connect those objects with the corresponding transcript and explanation.

For example:

Video timestamp: 00:43:17
Referenced bars: 57–63
Setup: H2 / second-entry buy
Context: Trading range
Location: Near top of range
Instructor decision: No trade
Reason: Poor location / insufficient probability
Source: Exact video and transcript segment

The resulting dataset will eventually be used for retrieval, evaluation, and potentially fine-tuning an AI trading decision system.

Primary Responsibilities

Build a pipeline that:

• Extracts frames and timestamps from course videos.
• Synchronizes transcripts to exact video timestamps.
• Detects the chart area within each slide/frame.
• Detects and tracks individual candlestick bars.
• Assigns stable bar IDs from left to right.
• Detects the instructor’s mouse/pointer.
• Tracks pointer movement across time.
• Resolves phrases such as “this bar” or “these bars” to the specific bars or chart regions being indicated.
• Detects existing chart annotations, arrows, trend lines, EMA lines, and text where useful.
• Associates the instructor’s explanation with the referenced chart objects.
• Outputs structured training examples in JSON/database format.
• Stores the exact source video, timestamp, transcript, frame/crop, referenced bars, and confidence for every example.
• Abstains when the referenced object cannot be determined confidently rather than creating bad labels.

Important Design Principle

Precision is more important than recall.

The system should not attempt to label every teaching moment.

If the pointer/reference alignment is ambiguous, the example should be marked UNCERTAIN and preserved for later human review or stronger future models.

A smaller dataset of highly reliable examples is substantially more valuable than a large dataset containing incorrect Brooks labels.

Trading Knowledge Structure

The first version will focus on approximately 12 primary Al Brooks setups.

Hundreds of additional Brooks concepts will be treated primarily as context, rather than giving every concept equal weight.

The pipeline therefore needs to support:

Primary setup → referenced bars → surrounding context → expert interpretation → expert decision → expert reasoning.

The engineer does not need to already be an Al Brooks expert. Trading-domain interpretation will be provided and developed separately.

The engineering problem is primarily multimodal grounding and data extraction.

Technical Skills

Strong experience with:

• Python
• Computer vision
• OpenCV
• Video processing
• Object detection/tracking
• Coordinate transformations
• Temporal alignment
• Speech/transcript alignment
• Multimodal LLM/VLM APIs
• Structured data pipelines
• JSON/SQL or similar data storage
• Automated validation/testing

Experience with the following would be especially useful:

• Cursor/pointer tracking
• Candlestick/chart recognition
• OCR
• Vision-language models
• Video understanding
• Temporal grounding
• Weak supervision
• Human-in-the-loop labeling systems
• Fine-tuning or dataset creation for multimodal models

Expected Pipeline

The target architecture is approximately:

Course video
→ frame extraction
→ transcript synchronization
→ chart detection
→ candlestick/bar detection
→ pointer tracking
→ language-reference resolution
→ Brooks concept associa…