Senior Databricks / Azure Data Engineer
- Source: Naukri
- NTT DATA, Inc.
- On-site
- Bengaluru
- Full-Time
- Senior
Job Description
Roles and Responsibilities :
- Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure Databricks.
- Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
- Develop complex SQL queries to optimize database performance and troubleshoot issues in Azure DevOps environment.
- Ensure high availability of the system by implementing monitoring tools such as Prometheus/Grafana.
Job Requirements :
- 5-8 years of experience in Data Engineering with expertise in ADF, Azure Databricks, PySpark.
- Strong understanding of Git version control system for managing code repositories.
- Experience working on big data technologies like Hadoop/Hive/Pig/Sqoop etc. .
- Proficiency in writing complex SQL queries for querying relational databases.
Skills
- azure-databricks
- databricks
- microsoft-azure
- azure-data-factory
- azure-devops
- Prometheus
- Grafana
- pyspark
- Hadoop
- apache-hadoop
- pig
- sqoop
- apache-sqoop
