Azure Databricks Developer
- Source: Indeed
- Sparsh Technologies Pvt Ltd
- Remote
- India
- Mid level
Job Description
We are looking for an experienced Azure Databricks Developer with 9–10+ years of experience in Data Engineering/Azure Cloud Data Development. This is a 100% remote contractual opportunity requiring candidates to work in the EST time zone.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
- Build and orchestrate ETL/ELT workflows using Azure Data Factory (ADF).
- Develop data ingestion and transformation pipelines using ADLS Gen2, Delta Lake, and Spark SQL.
- Implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold).
- Implement incremental loads, CDC, MERGE operations, and SCD Type 1/Type 2.
- Optimize Spark workloads through partitioning, caching, file optimization, and performance tuning.
- Configure and manage Databricks Jobs, Workflows, clusters, and environments.
- Implement data quality checks, validation, error handling, monitoring, and reconciliation.
- Work with Unity Catalog, RBAC, governance, and data lineage.
- Integrate Databricks with ADLS, Azure SQL, APIs, JDBC sources, and external systems.
- Develop CI/CD pipelines using Azure DevOps, Git, and YAML.
- Work with Terraform, ARM, or Bicep for Infrastructure-as-Code.
- Troubleshoot production data pipelines and perform root-cause analysis while meeting SLAs.
Required Skills
- 9–10+ years of Data Engineering / Azure Cloud experience
- Azure Databricks
- PySpark
- Spark SQL & SQL
- Delta Lake
- Azure Data Factory (ADF)
- ADLS Gen2
- Lakehouse / Medallion Architecture
- CDC, MERGE, SCD Type 1 & Type 2
- Spark performance optimization
- Azure DevOps, Git & YAML
- Terraform / ARM / Bicep
- Production support and troubleshooting
Preferred Skills
- Unity Catalog
- Databricks administration, Jobs/Workflows and clusters
- Azure SQL and API/JDBC integrations
- Datadog/Grafana
- Azure/Databricks cost optimization
- Enterprise-scale production data platforms
Additional Details
Location: India – 100% Remote
Shift: EST Time Zone
Experience: 10+ Years
Employment Type: Contractual / Temporary
Budget: ₹1.2 LPM – ₹1.5 LPM
Joining: Immediate Joiners Preferred
Candidates must be strongly hands-on with Azure Databricks and PySpark and should have experience supporting production-grade data engineering environments.
Pay: ₹120,000.00 - ₹150,000.00 per month
Benefits:
- Work from home
Work Location: Remote
Skills
- microsoft-azure
- azure-databricks
- databricks
- pyspark
- azure-data-factory
- delta-lake
- apache-spark-sql
- Spark
- unity-3d
- azure-sql
- azure-devops
- yaml
- Terraform
- datadog
- Grafana
You missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
