AI Observability Engineer
- Source: Naukri
- Tata Consultancy Services
- On-site
- Bengaluru
- Full-Time
- Mid level
Job Description
Role & responsibilities
AI Model Monitoring & Observability
- Design and implement observability frameworks for AI/ML and GenAI applications.
- Monitor model performance, prediction quality, latency, throughput, and availability.
- Track model drift, data drift, concept drift, and performance degradation.
- Establish AI-centric KPIs, SLAs, and SLOs.
Generative AI Observability
-
Monitor LLM applications for:
- Hallucinations
- Response quality
- Prompt effectiveness
- Toxicity detection
- Cost and token consumption
- Retrieval quality in RAG systems
MLOps & Governance
- Collaborate with data scientists and ML engineers to deploy monitoring solutions.
- Ensure compliance with Responsible AI and governance policies.
Incident Management
- Investigate AI service outages and model failures.
- Perform root cause analysis (RCA).
Preferred candidate profile
- Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or related field.
- 5+ years in Data Engineering, AI/ML, MLOps, or Platform Engineering.
- 2+ years working with GenAI/LLM solutions.
- Experience supporting production AI applications.
Skills
- drift
