Senior Lead-AI Engineer

Position Summary
This role owns agent reliability, observability, and trust controls. Monitors system health, trace logs, response quality, and hallucination signals; establishes detection thresholds and alerts. Works closely with the AI platform team to diagnose issues, improve guardrails, and ensure stable, compliant operation in production

AI Engineer will:

• Monitor data changes in ServiceNow-SNOW (Views in Databricks) for any inconsistencies that affect agents’ reliability.
• Controls and monitors data access points (SNOW/Azure/unstructured data, Documentation etc)
• Consults proposed changes with Tech/Business.
• Design and develops quick changes to data pipelines, agents, interfaces
• Trains end users on prompt usage and best practices; tests and refines existing prompts; identifies new prompts aligned to evolving business needs. Partners with the Business Product Owner to ensure a closed-loop learning system, where user feedback, agent performance metrics, and production outcomes are systematically incorporated to improve prompt effectiveness and accuracy.
• Ensures prompts are strictly scoped to Master Control Framework (MCF) - defined domains, controls, and measures, preventing drift into non-authoritative logic.
• Confirms that agent responses triggered by prompts map directly to defined MCF elements such as control IDs and calculation logic. Validates reliance on MCF-authoritative data sources and enforces refusal or escalation behaviour when queries fall outside MCF scope or require human judgment.
• Translates user feedback insights into prioritized enhancement requests.
• Partners with Tech to improve agent usability, response clarity, and front-end experience without altering approved business logic.
• Translates business needs into deliverable increments, and ensures alignment across architecture, security, and platform standards
• The role combines technical knowledge and expertise (hands-on) with functional knowledge of Risk management.

Responsibilities

• Builds and deploys machine learning and Gen AI solutions for FRMC user requirements.
• Develops small, easy and fast changes, automation of the maintenance tasks, data pipelines, agents, interfaces in cooperation with Business/Tech/DA Team
• Partners with Tech and FRMC business AI governance and users to improve agent usability, response clarity, governance, and front-end experience without altering approved business logic.
• Prioritizes and converts user feedback and business needs into deliverable increments, ensuring alignment with architecture, security, and platform standards. First line tester for any changes and enhancements.
• Through using LLMs, and prompt engineering supports the ongoing maintenance and continuous improvement of AI agents in FRMC
• Confirms that agent responses triggered by prompts map directly to defined master control framework, controls testing, and risk and control process elements such as control IDs and calculation logic according to the AI agent’s role.
• Controls and monitors data access points (SNOW/CO/unstructured data, GSOPs etc) Monitor process (business) and/or technology changes to act promptly without interruption of the solution.
• Ensure documentation, trainings, governance rules (Tech Azure governance) are updated and available Partners with the Business Product Owner to ensure a closed-loop learning system, where user feedback, agent performance metrics, and production outcomes are systematically incorporated to improve prompt effectiveness and accuracy.
• Maintain and share a detailed understanding of the underlying service landscape and the tools/platforms used
• Validates reliance on FRMC AI agents -authoritative data sources and enforces refusal or escalation behaviour when queries fall outside MCF scope or require human judgment.

Basic Qualification
We are seeking professionals with the following required skills and qualifications to help us achieve our goals:

• Bachelor’s degree in computer science, engineering, data science or related field.
• 5+ years of hands-on experience building and/or modifying Azure Web applications (Azure Webapps) and use of Azure AI solutions
• Strong Python skills and experience with AI frameworks (Lang*) + FastAPI, Kong, Pydantic, Spark
• Experience with MS Azure products and concepts (Databricks, ADF, Genie, WebApps, AppServices, KeyVault etc) and containerization (Docker, Kubernetes).
• Solid understanding of data engineering: SQL, view…