Data Engineer – Azure Databricks / PySpark / dbt / Snowflake

Data Engineer – Azure Databricks / PySpark / dbt / Snowflake

Experience: 6+ Years
Job Type: Contract / C2C
Location: Remote
Work Hours: 11:00 AM – 9:00 PM IST

Job Summary

We are hiring an experienced Data Engineer with strong hands-on experience in Azure Databricks, PySpark, Python, dbt, Snowflake, and SQL.

The candidate will be responsible for developing scalable data pipelines, implementing ETL/ELT solutions, optimizing Spark and Snowflake workloads, and working with modern cloud data platforms.

Mandatory Skills

• Azure Databricks – Mandatory
• PySpark – Mandatory
• Python – Mandatory
• dbt – Mandatory
• Snowflake – Mandatory
• Strong SQL
• Strong Data Warehousing and Data Modeling

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.
• Develop ETL/ELT pipelines using Python and PySpark.
• Build and optimize data processing solutions for large datasets.
• Create and maintain dbt models, tests, documentation, and dependencies.
• Work with Snowflake for data loading, transformation, analytics, and performance optimization.
• Integrate data from multiple source systems into Azure and Snowflake.
• Develop complex SQL queries and scalable data models.
• Implement data quality checks, validation, reconciliation, and error handling.
• Perform Spark, PySpark, Databricks, and Snowflake performance tuning.
• Develop batch and near-real-time data pipelines.
• Troubleshoot production data pipeline issues and resolve failures.
• Follow Git, CI/CD, coding, and deployment best practices.
• Collaborate with architects, developers, business stakeholders, and other engineering teams.

Required Technical Skills : Azure Databricks

• Hands-on production experience with Azure Databricks.
• Experience with Databricks notebooks, jobs, and workflows.
• Strong understanding of Apache Spark architecture and distributed processing.
• Experience with Spark performance optimization.

PySpark & Python

• Strong hands-on experience with PySpark.
• Experience with DataFrames, joins, transformations, partitioning, caching, and optimization.
• Strong Python programming and debugging skills.
• Ability to develop reusable, production-grade Python code.

dbt

• Hands-on experience with dbt.
• Experience creating models, tests, documentation, and dependencies.
• Strong understanding of modern ELT practices.

Snowflake & SQL

• Strong hands-on experience with Snowflake.
• Experience with data loading, transformation, warehouse optimization, and performance tuning.
• Strong SQL and data modeling skills.
• Good understanding of data warehousing concepts.

Additional Requirements

• Experience working with large-scale enterprise data platforms.
• Knowledge of ETL/ELT, data quality, and data governance.
• Experience with Git and CI/CD.
• Experience working in Agile/Scrum environments.
• Strong analytical and troubleshooting skills.
• Good communication and collaboration skills.
• Ability to work independently in a remote environment.

Important Requirement

Azure Databricks is mandatory for this position.

Candidates with only AWS Databricks or GCP Databricks experience may not be considered unless they also have relevant hands-on Azure Databricks experience.

Pay: ₹80,000.00 - ₹100,000.00 per month

Experience:

• Data Engineer : 6 years (Required)

Work Location: Remote