Aws Data Engineer
- Source: Naukri
- Integrated Personnel Services
- On-site
- Bengaluru
- Full-Time
- Mid level
Job Description
Job role - AWS Data engineer
Experience - 6+
Location - Hyderabad/Chennai/Bangalore/Pune
Mandatory Skill sets
- Strong Python Programming: Core Python, OOP, multithreading/multiprocessing, memory management, exception handling, debugging, and performance optimization.
- PySpark & Apache Spark: DataFrames/RDDs, transformations, joins, partitioning, shuffle, Spark internals (DAG, Catalyst, AQE), and performance tuning
- Advanced SQL: Complex joins, window functions, CTEs, query optimization, indexing, execution plans, stored procedures, and database design.
Good to have
- ETL: ETL/ELT pipeline development, data modelling, batch & streaming processing, data lakes, Parquet/Delta Lake, and data quality frameworks.
- Cloud & Big Data Ecosystem: AWS (S3, EMR, Glue, IAM), Databricks, Airflow, Kafka, CI/CD, Git, Docker, and Infrastructure-as-Code (Terraform/CloudFormation).
- System Design & Production Support: Designing scalable data pipelines, troubleshooting production issues, monitoring, performance tuning, root cause analysis, and best practices for distributed systems.
Skills
- amazon-web-services
- Python
- pyspark
- Spark
- catalyst
- delta-lake
- amazon-s3
- amazon-emr
- aws-glue
- aws-iam
- azure-databricks
- databricks
- Airflow
- apache-airflow
- Kafka
- Docker
- Terraform
- aws-cloudformation
