Software Engineer - (Machine learning Platform)

This role is for one of our clients

Industry: Software Development

Seniority level: Mid-Senior level

Min Experience: 5+ years

Location: Bengaluru

JobType: full-time

We are seeking a Software Engineer – Machine Learning Platform to build and scale an AI-native platform that powers intelligent, data-driven applications across complex enterprise workflows. In this role, you will transform unified operational data into real-time intelligence by building the infrastructure that enables context-aware AI agents, retrieval systems, and production-grade machine learning models.

You will own core platform capabilities including the LLM control plane, model gateway, agent runtime, retrieval infrastructure, and ML platform services. Your work will establish the standards for building, evaluating, deploying, and governing both generative AI and traditional machine learning solutions, enabling product teams to deliver AI-powered features safely, reliably, and at scale.

Requirements

Key Responsibilities

LLM Platform & AI Infrastructure

• Design and operate an enterprise-grade LLM gateway and control plane featuring intelligent routing, rate limiting, failover, token management, and cost optimization.
• Develop unified APIs and SDKs (REST/gRPC) with standardized schemas, structured outputs, caching, and comprehensive observability.
• Implement security and privacy controls including content moderation, prompt validation, response filtering, and PII protection.
• Enable seamless multi-model and multi-provider LLM integration with automated versioning, canary deployments, and rollback mechanisms.

Agent Platform & Orchestration

• Build and maintain the runtime environment for AI agents, including tool registries, permissions, function calling, retrieval, and grounding capabilities.
• Design orchestration patterns for sequential, planner-executor, streaming, and long-running workflows.
• Integrate human-in-the-loop approval mechanisms and safe execution controls before automated actions interact with production systems.
• Monitor agent performance, reliability, and operational health through telemetry and evaluation frameworks.

Machine Learning Platform

• Develop scalable infrastructure supporting training, deployment, and inference for classical machine learning models and deep learning workloads.
• Build standardized pipelines for experiment tracking, model packaging, deployment, and lifecycle management.
• Implement automated monitoring for model performance, drift detection, retraining, and continuous optimization.
• Support online and batch inference workflows for production ML applications.

Knowledge Graph & Data Platform

• Build and evolve knowledge graph infrastructure and entity resolution systems.
• Develop reliable, scalable data ingestion and transformation pipelines.
• Deliver contextual information to AI systems through secure access controls, lineage tracking, and governance.
• Implement hybrid retrieval systems combining graph databases, vector search, and keyword search for high-quality context retrieval.

Model Governance & Reliability

• Design continuous evaluation frameworks measuring model quality, factual accuracy, bias, safety, and performance.
• Define and monitor Service Level Objectives (SLOs) for latency, uptime, throughput, and infrastructure cost.
• Manage model registries, versioning, approvals, audit trails, and reproducible deployment workflows.
• Build autoscaling strategies, monitoring dashboards, and cost optimization mechanisms.

Developer Experience

• Create reusable SDKs, templates, developer tools, CLIs, documentation, and sandbox environments.
• Improve platform usability by establishing best practices for ML engineering, MLOps, AI safety, and deployment workflows.
• Mentor engineering teams and promote scalable engineering standards across the organization.

Required Qualifications

• 5–11+ years of experience building large-scale machine learning platforms, data platforms, or distributed software systems.
• Strong software engineering skills with expertise in distributed systems, concurrency, and scalable API design.
• Production experience with Python and at least one of Java, Go, or Scala.
• Hands-on experience building microservices and REST/gRPC APIs.
• Strong understanding of cloud-native architecture, containers, and Kubernetes.
• Experience with MLOps platforms including ML…