Lead/Senior Azure Data Engineer - Hyderabad

Job Title:                         Lead/Senior Azure Data Engineer

Employment Type:        Full Time, Permanent

Position Location:          Hyderabad

Qualifications:                BE/B.Tech/MCA Degree in Computer Science, Engineering, or similar relevant field

Total Experience:           7-10 years

Working Model:             Work from office

About the Role

We are seeking Data Engineers to design, develop, and support scalable, cloud-native data platforms on Microsoft Azure. The ideal candidate will have strong experience in building modern data pipelines, event-driven architectures, and Lakehouse solutions using Azure data services.

You will work closely with cross-functional teams to develop secure, reliable, and high-performance data solutions that enable analytics, reporting, and real-time data processing.

Key Responsibilities

• Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Data Factory, Azure Functions, Event Hubs, and Azure Data Lake Storage Gen2.
• Build and support modern Lakehouse architectures following the Medallion (Raw, Valid, Curated) data model.
• Develop and maintain real-time and event-driven data processing solutions using streaming technologies and Change Data Capture (CDC) patterns.
• Create and optimize SQL-based data transformation, validation, and quality processes.
• Develop production-grade Python applications for data processing, automation, and event-driven workloads.
• Design, develop, and deploy Power BI dashboards, semantic models, and reporting solutions connected to curated Lakehouse datasets.
• Manage Infrastructure as Code (IaC) using Terraform to provision and maintain Azure resources.
• Build and maintain Azure DevOps CI/CD pipelines for automated deployment, testing, and release management.
• Implement source control best practices, including feature branching, pull request reviews, and structured code management.
• Collaborate with architects, developers, and business stakeholders to deliver scalable and secure data solutions.
• Monitor, troubleshoot, and optimize platform performance, reliability, and operational efficiency.
• Ensure adherence to data governance, security, and cloud engineering best practices.

Required Qualifications

• Strong hands-on experience with Azure Data Services including:

Azure Data Factory (ADF)

Azure Event Hubs

Azure Functions

Azure Data Lake Storage Gen2 (ADLS Gen2)

Azure Key Vault

Azure Service Bus

• Strong experience in designing and implementing Lakehouse and Medallion Architecture.
• Expertise in Power BI development including dashboards, semantic models, deployment pipelines, and integration with Lakehouse data sources.
• Experience with Infrastructure as Code (Terraform) for Azure resource provisioning.
• Strong experience building Azure DevOps CI/CD pipelines with multi-stage deployments and release management.
• Experience designing and implementing streaming/event-driven data pipelines.
• Strong SQL (T-SQL) development for transformation, validation, and optimization.
• Proficiency in Python (3.10+) for production-grade data engineering applications.
• Strong understanding of Git, feature branching strategies, pull requests, and version control best practices.
• Excellent analytical, troubleshooting, and problem-solving skills.
• Strong communication and collaboration skills in Agile development environments.

Preferred Qualifications

• Experience deploying and managing applications on Azure Kubernetes Service (AKS) including Helm charts.
• Knowledge of secure data engineering practices including PII handling, tokenization, Managed Identity, RBAC, private networking, and Azure security best practices.
• Experience implementing observability using OpenTelemetry, Azure Monitor, and Application Insights.
• Experience working with Debezium or other Change Data Capture (CDC) technologies.
• Knowledge of automated testing frameworks including pytest, integration testing, and contract testing.
• Experience with Python dependency management, packaging, and reproducible deployment practices.
• Exposure to multi-repository enterprise data platforms and collaborative development models.