Senior/ Lead Azure Data Engineer

Position Overview:

As a Data Engineer, you will be responsible for the Requirement Gathering, Data Analysis, Development and implementation of Orchestrated data pipeline solutions to support our organization’s data-driven initiatives to ensure data accuracy and enable data-driven decision-making across the organization. The ideal candidate will possess a minimum of 7+ years of hands-on experience in data engineer on high-performing teams. Expertise in DBT, Airflow, Azure Databricks, SQL, Python, Py-spark, Automation is a must and knowledge of reporting tools is addon.

Key Responsibilities:

• 7+ years of hands-on experience using DBT. Airflow, Azure Databricks, Python, Py-spark and SQL , Preferred from Healthcare & Fintech Domain having Automation First Mindset.
• Hands-on experience with Data Collection, Data Analysis, Data modeling, Data Processing using DBT, Airflow, Azure Databricks, Py-spark, SQL, Python.
• Performance Optimization and Automation: Continuously monitor and optimize existing solutions and Debugging DAG failures and resolving.
• Data Processing: Leverage his expertise building Robust Data pipelines using mentioned tech stack with CI/CD.
• Collaboration: Collaborate with cross-functional teams, including data scientists, business analysts, and stakeholders, to understand their data needs and deliver solutions.
• Data Quality: Implement data validation and cleansing processes to ensure data accuracy, consistency, and reliability.

Requirements:

• Bachelors degree in computer science, Information Technology, or a related field.
• Minimum 7+ years of hands-on experience using DBT. Airflow, Azure Databricks, Py-spark, SQL, Python, Automation.
• Flexible to build Data Reports and Dashboards using SQL, Python, Reporting Tools
• Strong Debugging and Automation skills
• Strong understanding of DWH/Data Lake concepts and methodologies. Experience with cloud platforms such as Azure, AWS or GCP
• Excellent problem-solving skills and attention to detail
• Knowledge of data quality, data Validation, data security and compliance standards is a plus.