Azure Data Engineer
- Source: Naukri
- Alike Thoughts
- Hybrid
- Bengaluru
- Full-Time
- Mid level
Job Description
Job Description:
- Pipeline Development: Build and optimize ETL pipelines using Databricks and PySpark.
- Data Integration: Manage data from Azure Data Lake Storage (ADLS Gen2) and multiple formats (Parquet, Delta, JSON, CSV).
- Performance Tuning: Optimize Spark SQL queries, cluster configurations, and partitioning strategies.
- CI/CD: Use GitHub Actions or Azure DevOps for workflow automation.
-
Qualifications:
- 5+ years IT experience, with 23 years in Databricks.
- Strong skills in Python, PySpark, SQL.
- Familiarity with ADF, ADLS Gen2, Azure SQL.
Skills
- microsoft-azure
- azure-databricks
- databricks
- pyspark
- azure-storage
- aws-data-lake-storage
- JSON
- apache-spark-sql
- Spark
- Github
- github-actions
- azure-devops
- Python
- azure-sql
