Databricks Platform Engineer / Manager – AWS

Databricks Platform Engineer / Manager – AWS

Experience: 5+ Years
Employment Type: C2C
Location: Hybrid – Noida, Bengaluru, Chennai, Pune, Kochi, Mumbai, Jaipur, Gurugram

Job Summary

We are looking for an experienced Databricks Platform Engineer / Manager with strong expertise in Databricks administration and AWS cloud to own the design, security, governance, automation, performance, and cost optimization of enterprise Databricks platforms.

The ideal candidate will enable data engineering, analytics, and AI teams through secure, scalable, reliable, and highly automated Databricks platform operations.

Key Responsibilities

• Manage Databricks workspaces across Dev, Test, UAT, and Production environments.
• Maintain platform availability, reliability, upgrades, and performance.
• Define and manage cluster policies, instance pools, and standardized configurations.
• Manage AWS IAM, SSO, authentication, authorization, PATs, service principals, and secret scopes.
• Administer Unity Catalog, RBAC, governance, compliance, and audit controls.
• Support Git integration, CI/CD pipelines, Terraform-based IaC, and automated deployments.
• Monitor platform health and troubleshoot production incidents.
• Optimize cluster sizing, auto-scaling, Databricks usage, and cloud costs.
• Maintain audit logging, security controls, compliance, and operational documentation.
• Collaborate with Data Engineering, DevOps, Security, and Architecture teams.

Required Technical Skills : Databricks

• Databricks Workspace Administration
• Cluster Management & Cluster Policies
• Instance Pools
• Databricks Workflows
• Databricks Repos
• Unity Catalog
• Databricks Runtime (DBR) Management

AWS

• AWS IAM
• VPC
• EC2
• S3
• CloudWatch
• KMS
• Networking & Security Groups

DevOps / Automation

• Git
• CI/CD
• Terraform
• Infrastructure as Code (IaC)
• Azure DevOps / GitHub Actions / Jenkins

Platform Operations

• Monitoring & Incident Management
• Production Support & Troubleshooting
• Performance Tuning
• Cost Optimization
• Security & Governance
• High Availability & Scalability

Required Experience

• 5+ years of relevant experience in Databricks / Cloud / Platform Engineering.
• Strong experience administering enterprise Databricks environments across multiple workspaces.
• Hands-on experience with Unity Catalog and AWS IAM integration.
• Experience with cluster governance, Databricks runtime upgrades, Git integration, CI/CD, monitoring, audit logging, and secrets management.
• Strong experience with Terraform, DevOps, production support, and cost optimization.
• Strong understanding of cloud architecture, networking, security, scalability, and HA.

Preferred Qualifications

• Experience managing large-scale Databricks environments with 100+ users.
• Strong experience with Terraform-based provisioning.
• Knowledge of Lakehouse Architecture, Delta Lake, FinOps, governance, and security frameworks.
• Experience with enterprise platform modernization and automation.

Nice to Have

• AWS Organizations / Control Tower
• AWS PrivateLink / VPC Endpoints
• CloudWatch / Datadog / Splunk
• Apache Spark internals
• AI/ML workloads on Databricks

Education & Certifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.
• Preferred certifications:
• Databricks Certifications
• AWS Solutions Architect
• AWS SysOps Administrator
• Terraform Associate

Ideal Candidate

A strong Databricks Platform Engineering professional with hands-on AWS expertise who can independently own platform operations, security, governance, automation, performance, and cost optimization, while working closely with Data Engineering, DevOps, Security, and Architecture teams.

Pay: ₹90,000.00 - ₹120,000.00 per month

Experience:

• Databricks: 5 years (Required)

Work Location: Remote