Senior/Lead GenAI Engineer / AI Evaluation & Agentic AI Engineer
- Source: Naukri
- Anri Solutions Hr Services
- $524.42 to $26,220.76 /yr
- On-site
- Bengaluru
- Full-Time
- Senior
Job Description
- Experience building AI/ML solutions using Google Vertex AI on Google Cloud Platform (GCP)
- Hands-on experience with Google ADK, LangGraph, and Agentic AI frameworks
- Strong expertise in LLM & Agent Evaluation Frameworks, including trajectory/trace-based evaluation, tool-calling & planning metrics, Pass@k multi-run evaluation, and LLM-as-a-Judge / Agent-as-a-Judge methodologies
- Experience with Prompt Engineering, Retrieval-Augmented Generation (RAG), Knowledge Graphs, Semantic Modeling, and Vector Databases
- Experience implementing AI Guardrails, AI Governance, Model Monitoring, Evaluation Gating, and Responsible AI practices
- Strong Python development skills with experience in FastAPI, Pandas, NumPy, PyTest, and REST API development
- Experience with OCR and Intelligent Document Processing (IDP) using Google Document AI, Tesseract OCR, Azure Document Intelligence, or Amazon Textract
- Hands-on experience with Named Entity Recognition (NER), document classification, information extraction, text preprocessing, tokenization, embeddings, semantic search, and NLP pipelines
- Experience with fuzzy string matching and record linkage algorithms including Jaro-Winkler, Levenshtein Distance, Damerau-Levenshtein, Jaccard Similarity, Cosine Similarity, TF-IDF, FuzzyWuzzy/RapidFuzz, and entity resolution techniques
- Strong understanding of Machine Learning, Data Science, Statistical Analysis, Feature Engineering, Classification, Regression, Clustering, Recommendation Systems, and model evaluation metrics
- Experience with AI evaluation and observability tools such as Braintrust, Arize Phoenix, OpenTelemetry, Promptfoo, Galileo, DeepEval, Confident AI, Ragas, Langfuse, and LangSmith
- Hands-on experience with Google Kubernetes Engine (GKE), Cloud Run, Docker, Kubernetes, and cloud-native application deployment on GCP
- Experience implementing security controls including RBAC, Data Redaction, PII Detection & Masking, Secrets Management, and AI security governance
- Strong understanding of cloud-native architectures, CI/CD pipelines, containerized deployments, MLOps, LLMOps, software engineering best practices, and production AI deployment
- Excellent analytical, problem-solving, stakeholder communication, technical documentation, and solution architecture skills
Skills
- vertex
- google-cloud-platform
- LangGraph
- guardrails
- Python
- FastAPI
- Pandas
- NumPy
- Pytest
- processing-js
- document-ai
- tesseract-ocr
- microsoft-azure
- amazon-textract
- opentelemetry
- galileo
- Kubernetes
- google-kubernetes-engine
- google-cloud-run
- Docker
