AI Infrastructure Engineer

Job Description

​We are seeking a highly skilled and hands-on AI Infrastructure Engineer to design, build, and maintain our next-generation, self-hosted AI infrastructure. In this role, you will bridge the gap between enterprise infrastructure and advanced AI engineering. You will be responsible for deploying, optimizing, and orchestrating multi-model Large Language Models (LLMs), Vision-Language Models (VLMs), and Agentic AI frameworks on both raw bare-metal hardware and enterprise container platforms.

​The ideal candidate thrives in a high-performance computing environment, understands the nuances of bare-metal GPU clustering, and has a proven track record of moving open-source AI stacks from experimental setups into secure, scalable production environments.

​Key Responsibilities

​1. AI Infrastructure & Deployment Orchestration

• ​Design, provision, and manage high-performance compute clusters on bare-metal (Rocky Linux / Ubuntu) and enterprise platforms (Red Hat OpenShift / Kubernetes).
• ​Deploy, optimize, and maintain scalable LLM/VLM serving engines using vLLM and Ollama, ensuring low-latency and high-throughput inference.
• ​Configure and scale distributed training and inference workloads using Ray framework, specifically optimizing for multi-GPU, multi-modal applications (Text, Image, and Audio).
• ​Set up, secure, and customize enterprise-grade user interfaces like Open WebUI for cross-functional team access.

​2. Advanced RAG & Model Lifecycle Management

• ​Architect and maintain end-to-end Retrieval-Augmented Generation (RAG) pipelines with advanced context management, chunking strategies, and hybrid search capabilities.
• ​Integrate and tune vector databases (e.g., Milvus, Qdrant, PGVector) to support high-context RAG operations.
• ​Establish reproducible pipelines for model fine-tuning (PEFT, LoRA, QLoRA) on private datasets, optimizing memory efficiency and training times.

​3. Agentic AI & Autonomic Systems

• ​Design and deploy scalable Agentic AI frameworks (e.g., CrewAI, AutoGen, LangGraph) capable of executing multi-step complex workflows, tool use, and self-correction.
• ​Develop custom agent “skills”, API integrations, and secure sandboxed execution environments for autonomous agents.

​Required Technical Skills

​Core AI & MLOps Stack

• ​Inference Engines: vLLM (including tensor-parallel/pipeline-parallel setups), Ollama, Hugging Face TGI.
• ​Distributed Computing: Ray (Ray Train, Ray Serve, Ray Core).
• ​Frameworks & UIs: Open WebUI, LangChain, LangGraph, CrewAI, AutoGen.
• ​Data & RAG: Advanced RAG architecture, semantic chunking, metadata filtering, and vector database administration.
• ​Fine-Tuning: DeepSpeed, Axolotl, or standard PyTorch-based fine-tuning workflows utilizing LoRA/QLoRA.

​Infrastructure & System Engineering

• ​Orchestration: Advanced Kubernetes administration, Red Hat OpenShift, Helm charting, and K8s operators (specifically NVIDIA GPU Operator).
• ​Operating Systems: Expert-level administration of Linux distributions (Rocky Linux / RHEL, Ubuntu Server).
• ​Hardware & Networking: Deep understanding of bare-metal GPU nodes (NVIDIA H100, L40S, A100), PCIe topologies, NVLink, and high-speed networking (InfiniBand/RoCE, SR-IOV).
• ​Containers: Docker, Podman, and secure container runtimes (CRI-O, containerd).

Pay: ₹15,000.00 - ₹40,000.00 per month

Benefits:

• Provident Fund

Work Location: In person