Azure Databricks
- Source: Indeed
- Uno Gravitas
- On-site
- State of Telangāna, India
- Full-Time
- Mid level
Job Description
Job Title: Azure Databricks Developer
Role & Responsibilities:
•
Design, implement, and manage data pipelines and analytics workflows using Azure Databricks.
•
Develop and optimize large-scale data models and processing frameworks for big data analysis and reporting.
•
Collaborate with data engineers, data scientists, and business teams to understand requirements and translate them into robust data solutions.
•
Write efficient, reusable, and scalable PySpark or Scala code for processing and transforming large datasets in Azure Databricks.
•
Work with Azure Data Lake Storage (ADLS), Azure Blob Storage, and other cloud storage services for data ingestion and processing.
•
Implement real-time and batch data processing workflows in Databricks using Apache Spark.
•
Experience with Change Data Capture (CDC) and Change Data Feed (CDF) methodologies for real-time data integration and synchronization.
•
Worked on Slowly Changing Dimension (SCD) modelling using PySpark to efficiently handle historical data and changes over time.
•
Implemented Data Governance best practices using Unity Catalog to manage data access, lineage, and compliance in Azure Databricks.
•
Optimize performance and troubleshoot issues related to data pipelines, Databricks clusters, and jobs.
•
Collaborate on the design and implementation of ETL processes and workflows.
•
Participate in the deployment and monitoring of Databricks-based data solutions.
•
Write unit tests and maintain quality documentation for the solutions built.
Skills & Qualifications:
•
Strong experience in Azure Databricks and expertise in managing clusters, notebooks, and jobs.
•
Proficiency in Apache Spark, PySpark, Python for distributed data processing and analysis.
•
Experience with Azure cloud services, especially Azure Data Lake, Azure Blob Storage, and Azure SQL Database.
•
Familiarity with data modelling and data warehousing concepts (star schema, snowflake schema).
•
Expertise in writing SQL for data manipulation and querying large datasets.
•
Experience with ETL tools such as Azure Data Factory or Apache Airflow.
•
Knowledge of cloud-native CI/CD pipelines and automation.
•
Experience with Data Lake architectures, data ingestion, transformation, and visualization.
•
Ability to troubleshoot and optimize Spark applications, ensuring high performance.
•
Experience working with version control tools like Git.
Work Location: In person
Skills
- microsoft-azure
- azure-databricks
- databricks
- pyspark
- Scala
- azure-storage
- aws-data-lake-storage
- Spark
- unity-3d
- Python
- azure-sql
- azure-sql-database
- azure-data-factory
- Airflow
- apache-airflow
You missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
