Azure Data Engineer-F2F Interview
- Source: Naukri
- Elabs Infotech
- On-site
- Gurugram
- Full-Time
- Mid level
Job Description
We are seeking an experienced Azure Databricks Engineer to design, develop, and optimize scalable data processing solutions on Microsoft Azure. The ideal candidate will have strong expertise in Azure Databricks, Apache Spark, Python, SQL, and modern data engineering practices.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines using Azure Databricks.
- Build scalable data processing workflows with Apache Spark (PySpark/Scala).
- Integrate data from various sources including Azure Data Lake Storage (ADLS), Azure SQL Database, Azure Synapse Analytics, APIs, and other cloud/on-premises systems.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Develop notebooks, workflows, and jobs in Azure Databricks.
- Implement data quality, validation, and monitoring processes.
- Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
- Work with Azure services such as Data Factory, Key Vault, Blob Storage, Event Hubs, and Azure DevOps.
- Implement CI/CD pipelines for Databricks deployments.
- Ensure data security, governance, and compliance using Azure best practices.
Skills
- microsoft-azure
- azure-databricks
- databricks
- Spark
- pyspark
- Scala
- azure-storage
- aws-data-lake-storage
- azure-sql
- azure-sql-database
- azure-synapse
- microsoft-azure-synapse
- azure-synapse-analytics
- azure-data-factory
- azure-key-vault
- vault
- azure-devops
