Lead AWS Data Engineer

Role: Lead Data Engineer

Experience: 7 to 10 yrs

Location: All EXL Location

Work mode: Hybrid J

ob Description

• Design, develop, and maintain scalable and efficient ETL/ELT pipelines using tools such as AWS Glue, DBT, and Airflow.

• Build and automate event-driven data workflows leveraging AWS Lambda, EventBridge, and related services.

• Integrate data replication tools like Hevo or Fivetran for seamless ingestion from external systems.

• Perform data modeling (dimensional and normalized) to support Tableau team and analytics.

• Manage infrastructure as code using Terraform and automate workflows using GitHub Actions.

• Optimize and maintain Snowflake data warehouse environments, including performance tuning and data lifecycle management. • Use GitHub Actions to deploy DBT projects, manage data pipelines, and automate testing/monitoring tasks.

• Collaborate with analysts, data scientists, and engineering teams to gather requirements and deliver robust data solutions.

• Ensure data quality, integrity, and consistency across all systems and processes.

• Containerize data workflows using Docker for portability and repeatability.

• Productionalizes SQL queries from data analysts into structured dbt models within the silver/gold layer

Requirements

• 710 years of experience as a Data Engineer or in a similar role.

• Hands-on experience with AWS data services: Glue, Lambda, EventBridge, S3, IAM, Airflow/MWAA

• Proficiency in SQL and Python for data manipulation and orchestration.

• Snowflake data warehouse design and query optimization

• Solid experience with DBT, Apache Airflow, and Snowflake (or other cloud data warehouses).

• Experience using GitHub Actions for CI/CD in data workflows.

• Familiarity with data replication tools such as Hevo or Fivetran.

• Strong understanding of data modeling principles (e.g., star/snowflake schema).

• Prior experience in eCommerce, retail, or high-volume transactional data environments.

• Infrastructure-as-Code expertise using Terraform.

• Experience containerizing pipelines with Docker, ECR and EKS.

• Strong problem-solving and communication skills in a collaborative, cross-functional environment. Education: Bachelor’s degree in computer science, Software Engineering, MIS or equivalent combination of education and experience

Key Skills: AWS, Snowflake, Pyspark, SQL and Python, AWS Glue