Azure Data Engineer
- Source: LinkedIn
- Prophecy Technologies
- On-site
- Hyderabad
- Full-Time
- Mid level
Job Description
Required Skills:
- Strong experience in Data Engineering and ETL/ELT processes
- Proficiency in Python
- Hands-on experience with Apache Spark (PySpark and Spark SQL)
- Experience with Azure Databricks (Unity Catalog), Snowflake, Azure Functions, and Service Bus
- Strong SQL development and optimization skills
- Experience with Infrastructure as Code using Terraform
- Hands-on experience with CI/CD tools such as GitHub Actions, Git, Artifactory, and SonarQube
- Experience with AI/ML concepts and Large Language Models (LLMs)
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT solutions
- Build and optimize data processing workflows using Python and Spark
- Develop cloud-based data solutions using Azure Databricks and Snowflake
- Implement and manage infrastructure using Terraform
- Develop and maintain CI/CD pipelines for data engineering projects
- Collaborate with cross-functional teams to deliver high-quality data solutions
- Support AI/ML and LLM-based initiatives as required
Skills
- microsoft-azure
- Python
- Spark
- pyspark
- apache-spark-sql
- azure-databricks
- databricks
- unity-3d
- Snowflake
- azure-functions
- microsoft-azure-functions
- azure-service-bus
- Terraform
- Github
- github-actions
- sonarqube
