Azure Data Platform Engineer

Key Responsibilities

Platform & Service Management

• Manage and maintain Azure Data Explorer (Kusto) clusters, databases, and queries.
• Monitor ingestion pipelines, query performance, and cluster health.
• Perform troubleshooting and performance optimization for Kusto workloads.

Data Engineering Support

• Develop, manage, and monitor data pipelines using Azure Data Factory.
• Troubleshoot pipeline failures, integration runtime issues, and data movement challenges.
• Ensure reliability and monitoring of scheduled and event-based pipelines.

Big Data Cluster Operations

• Manage and administer Azure HDInsight clusters.
• Perform cluster configuration, scaling, monitoring, and maintenance.
• Troubleshoot cluster performance, job failures, and resource utilization.

Microservices Platform Management

• Support and manage applications running on Azure Service Fabric.
• Monitor cluster health, service deployments, and application performance.
• Perform upgrades, patching, and operational maintenance of Service Fabric clusters.

Incident & Operational Support

• Provide ICM (Incident & Change Management) support for platform services.
• Monitor alerts, investigate incidents, and perform root cause analysis.
• Participate in on-call rotation and provide weekend/night support.
• Work with engineering teams to resolve critical production issues.

Monitoring & Reliability

• Implement proactive monitoring and alerting for platform services.
• Ensure SLA compliance and service reliability.
• Maintain operational runbooks and documentation.

Required Skills

Technical Skills

• Strong experience in Kusto / Azure Data Explorer cluster management
• Hands-on experience with Azure Data Factory (ADF) pipelines
• Experience managing HDInsight clusters
• Experience supporting Azure Service Fabric clusters (Knowledge is also ok)
• Knowledge of Azure monitoring tools (Log Analytics, Azure Monitor, Alerts)
• Experience with incident management processes and production support
• Experience with Azure DevOps / CI-CD pipelines
• Knowledge of data engineering concepts and big data platforms
• Scripting experience in PowerShell / Python
• Familiarity with cloud security and access management

Experience

• 47 years of experience in Azure Data Platform / Cloud Operations
• Experience supporting production environments and incident management

Soft Skills

• Strong troubleshooting and analytical skills
• Ability to work in a 24x7 operational support model if required
• Good communication and documentation skills
• Ability to coordinate with multiple teams during incidents.