Edge AI & Embedded Systems Engineer

Mission

Design and own the on-device AI engine that runs perception and alignment models on AR headsets and ruggedised edge hardware – in environments with unreliable or zero connectivity. Factories have bad WiFi. Construction sites have none. If your system fails offline, the product is dead.

System Ownership

• Primary: On-device inference runtime (model loading, execution, resource management)
• Primary: Model optimisation pipeline (quantization, pruning, graph optimisation → deployment-ready artefacts)
• Primary: Offline-first architecture (local queue, retry sync, conflict resolution)
• Primary: Edge data compression (point cloud + telemetry compression before cloud sync)
• Secondary interface: CV team (you deploy and optimise their perception models on-device)
• Secondary interface: Cloud/Backend team (you sync data to their ingestion APIs when connectivity resumes)
• Does NOT own: Model training/architecture design (CV/AI teams), MR rendering (MR team), cloud infrastructure (Backend team)

What You Will Build

• On-device inference engine – Serve TensorRT/CoreML/ONNX models on heterogeneous edge hardware (ARM SoCs, Apple Neural Engine, Qualcomm Hexagon DSP). Manage model lifecycle: load, warm-up, execute, swap.
• Model quantization & optimisation pipeline – Take FP32 trained models → INT8/FP16 quantized, graph-optimised, target-specific artefacts. Maintain accuracy within 1% of FP32 baseline while hitting latency targets.
• Offline-first data architecture – Local SQLite/LevelDB store for scan data, inspection results, telemetry. Queue-based sync with exponential backoff. Conflict resolution when offline edits collide with cloud state. Edge data compression – Compress point clouds (Draco, custom octree encoding) and telemetry streams before upload. Target: 10× compression ratio with- Latency reduction architecture – Profile end-to-end inference pipeline (pre-processing → inference → post-processing). Identify and eliminate bottlenecks. Hit the 200ms total budget.

• Thermal management – Monitor device temperature during sustained inference. Implement throttling strategies that degrade gracefully (reduce resolution, increase skip frames) rather than crash. Core Technical Responsibilities

• Build the inference serving layer that abstracts across TensorRT (NVIDIA), CoreML (Apple), ONNX Runtime (cross-platform), and potential future runtimes
• Implement INT8 quantization with calibration dataset management – track accuracy regression per quantization profile
• Design the offline-first sync protocol: local writes are always available, cloud sync is opportunistic, conflict resolution is deterministic
• Build the data compression pipeline for point clouds: evaluate Draco, custom octree, and lossy compression with configurable geometric tolerance
• Profile and optimise CUDA kernel performance for inference pre/post-processing on NVIDIA Jetson-class hardware
• Implement cross-compilation pipelines (ARM64 targets) with deterministic builds and automated hardware-in-the-loop testing
• Build the device health monitoring system: battery, temperature, memory, storage, connectivity status – exposed as telemetry to cloud

Required Technical Mastery

• Inference runtimes: TensorRT, CoreML, ONNX Runtime – not just API usage, but understanding of graph optimisation passes, layer fusion, memory planning
• Quantization: Post-training quantization (PTQ), quantization-aware training (QAT), calibration dataset selection, accuracy/latency tradeoff analysis
• ARM optimisation: NEON intrinsics, memory hierarchy awareness, cache-friendly data layout for ARM Cortex-A / Apple M-series / Qualcomm Snapdragon
• Languages: C++ (primary – systems programming, memory management, RAII, no exceptions), Rust (desirable for safety-critical paths), Swift (Apple ecosystem), Python (tooling/scripting)
• Embedded systems: Cross-compilation (CMake, Bazel), hardware-in-the-loop testing, deterministic memory allocation (no runtime malloc in hot paths)
• Offline-first patterns: Local-first databases, CRDTs or operational transform for conflict resolution, queue-based sync with idempotent operations
• Compression: Point cloud compression algorithms, protocol buffer / flatbuffer serialisation, delta encoding for time-series telemetry

Production Challenges You’ll Solve

• Thermal throttling – The AR headset runs sustained inference for 2 hours on a factory floor at 35°C ambient. The SoC hits 85°C and starts throttlin…