AWS DATA Engineer (Databricks)
- Source: Naukri
- ADP Private Limited
- Hybrid
- Hyderabad
- Full-Time
- Mid level
Job Description
Job ID - IND170694 - EFS FCRM
Data Engineer - AWS Databricks - GenAI
Must-Have Skills
- AWS Databricks (Hands-on implementation experience)
- Apache Spark / PySpark
- Python
- SQL
- Delta Lake
- Databricks SQL
- Unity Catalog
- ETL/ELT Pipeline Development
- Data Lakehouse Architecture
- CI/CD for Data Pipelines
- AWS Cloud Platform
- AI / Generative AI / Agentic AI exposure
Preferred Certifications
- Databricks Certified Data Engineer Associate/Professional
- AWS Certified Data Analytics
- AWS Certified Solutions Architect (Good to have)
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines using Spark on Databricks.
- Build and manage Lakehouse architectures using Delta Lake.
- Implement orchestration using Databricks Workflows.
- Develop CI/CD pipelines using GitHub Actions, Databricks Repos, and AWS tools.
- Optimize pipeline performance and troubleshoot production issues.
- Manage data governance using Unity Catalog.
- Support batch and streaming data processing workloads.
- Contribute to AI/GenAI-enabled data solutions.
Skills
- amazon-web-services
- azure-databricks
- databricks
- Spark
- pyspark
- Python
- delta-lake
- unity-3d
- Github
- github-actions
