Senior Data Engineer – Azure, Databricks & Dagster

Job Type: Freelance / Part-Time / Contract

Engagement: Approximately 2 hours per day
Duration: Ongoing, based on project requirements
Work Mode: Remote
Opportunity: Suitable for experienced professionals seeking additional freelance income or part-time technical consulting alongside their existing employment.

Senior Azure Data Engineer — Freelance / Part-Time Technical Support

We are looking for an experienced Senior Azure Data Engineer to provide hands-on technical support for an active data engineering project focused on modernizing data pipelines, improving platform reliability, and implementing scalable data engineering practices.

The requirement is approximately 2 hours of support per day, with the possibility of extending working hours depending on project requirements.

This is not a full-time employment opportunity. We are specifically looking for professionals with strong real-world project experience who can provide practical technical expertise, troubleshoot complex issues, and contribute to ongoing data platform development.

Technologies

Candidates should have practical experience with several of the following technologies:

• Cloud Platform: Microsoft Azure, Azure Data Lake Storage (ADLS Gen2)
• Data Engineering: Azure Databricks, Delta Lake, Apache Spark, PySpark
• Workflow Orchestration: Dagster, Apache Airflow
• Data Transformation: dbt Cloud
• Data Governance: Unity Catalog, data lineage, metadata management
• Streaming & Event Processing: Apache Kafka, Confluent
• DevOps & Infrastructure: Docker, Kubernetes (AKS), Terraform
• CI/CD: Automated deployments, version control, pipeline testing
• Data Architecture: Medallion architecture (Bronze, Silver, Gold, Platinum)
• Integrations: REST APIs and third-party enterprise data platforms

Key Responsibilities

• Develop, maintain, and optimize data pipelines using Azure Databricks and related technologies.
• Troubleshoot existing ETL/ELT workflows and resolve pipeline failures.
• Support workflow orchestration, scheduling, and dependency management using Dagster or Airflow.
• Improve data quality, monitoring, logging, alerting, and pipeline observability.
• Assist in establishing and maintaining Development, Testing, and Production environments.
• Support CI/CD automation and infrastructure management.
• Implement data governance, access controls, and data lineage practices.
• Collaborate with project stakeholders to resolve technical challenges and improve delivery efficiency.
• Provide practical technical guidance based on real-world project experience.

Required Experience

• Strong professional experience in Azure data engineering.
• Hands-on experience developing and troubleshooting production data pipelines.
• Good understanding of Python, SQL, ETL/ELT processes, and data architecture.
• Practical knowledge of cloud-based data platforms and data integration.
• Ability to work independently and provide effective technical support within limited daily hours.
• Strong problem-solving and communication skills.

Pay: ₹20,000.00 - ₹25,000.00 per month

Benefits:

• Flexible schedule
• Work from home

Experience:

• Snowflake: 2 years (Required)
• Microsoft SQL Server: 5 years (Required)
• Apache Spark / PySpark: 2 years (Required)
• Azure Data Factory: 5 years (Required)
• Azure Databricks: 5 years (Required)
• Azure Data Lake Storage (ADLS Gen2): 5 years (Required)

Work Location: Remote