Job Closed
Azure Databricks Engineer
- Source: LinkedIn
- System Soft Technologies
- On-site
- New Brunswick
- Mid level
Job Description
Azure Databricks Engineer
Location: New Brunswick, NJ (Hybrid)
Duration: 1 Year
Key responsibilities:
- Design and implement scalable data pipelines using Azure Databricks and Delta Lake.
- Develop and maintain Medallion Architecture data layers
- Design and implement solutions using Delta Lake and Lakehouse architecture.
- Build scalable pipelines using PySpark and Spark SQL, with a focus on performance and cost efficiency.
- Develop ETL/ELT frameworks using PySpark and Spark SQL
- Optimize Databricks workloads for performance, scalability, and cost efficiency.
- Implement data quality, lineage, and governance standards using Unity Catalog.
- Automate deployment processes through Azure DevOps CI/CD pipelines.
- Explore and leverage Databricks product features (e.g., Delta share, Lakebridge, ABAC, RBAC) to enhance solution capabilities.
- Collaborate with cross-functional teams to validate migrated datasets and ensure business alignment.
- Support data migration and modernization initiatives from legacy platforms.
Skills
- microsoft-azure
- azure-databricks
- databricks
- delta-lake
- pyspark
- Spark
- apache-spark-sql
- unity-3d
- azure-devops
This job is no longer accepting applications.
Join the Club — Free ForeverYou missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
