Azure DataBricks Lead
- Source: Naukri
- Summitworks Technologies
- On-site
- Hyderabad
- Full-Time
- Senior
Job Description
If your experience matches the above requirements, please email your updated resume to [email protected]
Job description:
- Lead the design, development, and delivery of enterprise-scale data solutions on the Databricks Lakehouse Platform.
- Provide technical leadership while remaining hands-on with development, troubleshooting, and performance optimization.
- Architect and implement ETL/ELT pipelines using PySpark, Spark SQL, and Databricks.
- Design and build batch and real-time/streaming data pipelines using Structured Streaming, Kafka, or Event Hubs.
- Implement Delta Lake and Medallion Architecture (Bronze, Silver, Gold) data platforms.
- Drive Spark performance tuning, query optimization, cluster sizing, and cost optimization initiatives.
- Lead code reviews, establish development standards, and mentor Data Engineers.
- Integrate data from APIs, cloud storage, databases, and enterprise applications.
- Implement CI/CD, DevOps, monitoring, and data governance best practices.
Collaborate with Architects, Product Owners, and Business stakeholders to define scalable data solutions
Skills
- microsoft-azure
- azure-databricks
- databricks
- pyspark
- Spark
- apache-spark-sql
- Kafka
- delta-lake
