Senior Azure Data Engineer

Job Details

• Location: Hyderabad, Telangana
• Working Schedule: Full-Time
• Work Arrangement: Hybrid
• Relocation Assistance Available: Yes

Job Summary

We re Hiring: Senior Azure Data Engineer / Data Engineering Lead

Role: Senior Azure Data Engineer / Data Engineering Lead Experience: 8+ Years Level: Senior Individual Contributor

About the Role

We are looking for a senior Azure Data Engineering professional who can design, build, and lead delivery of enterprise-scale cloud data platforms. The ideal candidate will bring strong hands-on engineering depth across Azure, Databricks, Spark, Data Lakehouse, and modern data pipelines, while also guiding engineers, collaborating with business stakeholders, and driving high-quality delivery in an Agile and DevOps environment.

What You ll Lead and Deliver

• Modern Data Platform Lakehouse Engineering: Architect and deliver scalable data lakehouse solutions using Azure Data Lake Storage, Azure Databricks, Delta Lake, Unity Catalog, and Medallion architecture.
• Design curated data products, data marts, and analytics-ready datasets for business analysts, data scientists, and enterprise reporting teams.
• Apply strong data modeling practices, partitioning strategies, schema evolution, performance tuning, and cost optimization.

Data Pipelines, ETL/ELT Streaming

• Build reliable batch and near-real-time pipelines using Azure Databricks, Apache Spark, PySpark, SQL, Azure Data Factory, Synapse, and related Azure services.
• Implement ingestion frameworks for structured, semi-structured, and unstructured data from multiple enterprise sources.
• Ensure data quality, lineage, validation, observability, error handling, and reusable pipeline patterns.

Engineering Leadership

• Lead and mentor a team of data engineers, set engineering standards, review designs, and guide implementation decisions.
• Collaborate with architects, product owners, business stakeholders, data scientists, security, platform, and operations teams.
• Drive Agile delivery, estimation, sprint planning, technical roadmaps, and continuous improvement across the team.

DevOps, Security Governance

• Implement CI/CD for data workloads using Azure DevOps or GitHub Actions, including automated testing, deployment, and release governance.
• Apply secure engineering practices using Microsoft Entra ID, role-based access control, secrets management, and data governance controls.
• Promote monitoring, alerting, operational readiness, and SRE-aligned practices for production data platforms.

Candidate Profile

Education: Bachelor’s degree in Computer Science, Engineering, Information Technology, Data Engineering, or equivalent practical experience.

Experience: 8+ years of experience in data engineering, data platform engineering, ETL/ELT, BI, analytics, or cloud data application development.

4+ years of hands-on experience designing and delivering cloud-based data platforms on Microsoft Azure.

2+ years of strong experience with Azure Databricks, Apache Spark, PySpark, Spark SQL, Delta Lake, and Lakehouse architecture.

Proven experience leading engineers, reviewing architecture/design, owning delivery outcomes, and driving technical excellence.

Core Azure Skills

• Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, Azure SQL, Dedicated SQL Pool, Cosmos DB, Logic Apps, Azure Monitor, and Application Insights.
• Experience with Microsoft Entra ID, access control, secure connectivity, key management, and enterprise data governance.

Azure Databricks Skills

• Hands-on expertise in Databricks workspaces, notebooks, jobs/workflows, clusters/serverless compute, Delta tables, Auto Loader, Delta Live Tables or Lakeflow-style declarative pipelines.
• Strong understanding of Unity Catalog, catalog/schema/table governance, data lineage, access controls, data quality, and lakehouse security practices.
• Experience optimizing Spark jobs, Delta Lake performance, partitioning, Z-ordering/clustering strategies, workload monitoring, and cost-efficient compute usage.

Skills Competencies

• Technical Skills:

Python, PySpark, SQL, Spark SQL, data modeling, ETL/ELT design, orchestration, data quality, metadata management, and performance tuning.

Modern lakehouse patterns including Bronze/Silver/Gold layers, Delta Lake, schema evolution, slowly changing dimensions, and reusable ingestion frameworks.

Disclaimer: This job posti…