Data Engineer- With_Azure_Databricks_Snowflake_ADF_Pyspark_SQL_Python

Role & responsibilities :

Position Requirements:

Must Have:

• 4+  years of experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight,  Azure Analysis services
• Should be aware of Data Management concepts and Data Modelling
• Good knowledge of Big Data frameworks and related technologies - Experience in Hadoop and Spark is mandatory
• Should have expertise in extracting data from different source systems like flat files, XML sources, JSON, Parquet files, Big data appliances, RDBMS, etc.
• Good knowledge of Big Data frameworks and related technologies - Experience in Hadoop and Spark is mandatory
• Should have clear understanding of DW lifecycle and contributed in preparing
• Detail Level Design document
• Unit Test plans
• Code review reports
• Must have strong knowledge in Snowflake and sql, understand joins, groups, functions & Stored procedures
• Must have experience in Unit and Integration Test creation and verification
• Proficient in creating documentation related to Project Architecture, Project end User Manual and Operations Hand off guide
• Should have good understanding of Data Migration processes, methods and project lifecycle
• Good in Unix Shell Scripting skills
• Strong analytical skills and troubleshooting skills
• Should be well versed with quality processes and implementation
• Excellent communication skills
• Strong understanding of Cloud data migration processes, methods and project lifecycle
• Good knowledge of Application DevOps tools (Git, CI/CD Frameworks) - Experience in Jenkins or Gitlab with rich experience in source code management like Code Pipeline, Code Build and Code Commit

Desired Knowledge / Skills:

• Certifications in Azure / Databricks/ Snowflake
• Experience in building stream-processing systems, using solutions such as Storm or Spark-Streaming
• Experience in Big Data ML toolkits, such as Mahout, SparkML, or H2O
• Experience in Azure functions
• Knowledge in Python