Databricks data engineer (Azure/AWS)- PAN INDIA
- Source: Naukri
- Tata Consultancy Services
- On-site
- Bengaluru
- Full-Time
- Mid level
Job Description
Greetings from Tata Consultancy Services (TCS)
What we are looking for: Databricks data engineer (Azure/AWS)
Location : PAN INDIA
Interview Mode : Virtual mode (Microsoft Teams)
Experience : 5-10 Years
- Strong Experience and understanding of Data warehouse, Data Lake platforms architecture and design aspects
- Design and develop scalable end-to-end ETL/ELT pipelines on Azure/AWS Databricks using PySpark, Spark SQL and Delta Lake.
- Implement medallion (BronzeSilverGold) Lakehouse architectures with automated ingestion, schema evolution and performance optimisation.
- Build streaming and batch ingestion using Auto Loader, Lakeflow/Structured Streaming, Delta Live Tables (DLT) and Azure Data Factory.
- Enforce data governance, lineage, access control and cataloguing through Unity Catalog, RBAC and Delta Sharing.
- Optimise Spark workloads for cost and performance (partitioning, caching, cluster sizing, query tuning) across large datasets.
- Support legacy-to-Lakehouse migrations (Hadoop/DataStage/RDBMS) leveraging accelerators such as Lakebridge and reusable frameworks.
- Establish CI/CD for Databricks assets using Azure DevOps / Git, and build automated data validation and testing utilities.
- Collaborate with architects, AI/ML specialists and client teams on solutioning, demos, PoCs and production rollouts.
TECHNICAL SKILLS
Skills
- azure-databricks
- databricks
- microsoft-azure
- amazon-web-services
- mode
- microsoft-teams
- pyspark
- Spark
- apache-spark-sql
- delta-lake
- azure-data-factory
- unity-3d
- Hadoop
- apache-hadoop
- azure-devops
