AWS Data Engineer
- Source: LinkedIn
- PalTech
- On-site
- Hyderabad
- Full-Time
- Mid level
Job Description
Key Responsibilities
- Develop and maintain data pipelines using PySpark and AWS Glue.
- Build serverless data processing solutions using AWS Lambda.
- Design and optimize data storage and analytics solutions on S3, Athena, Redshift, and DynamoDB.
- Implement ETL/ELT processes for data ingestion and transformation.
- Ensure data quality, performance, and reliability across data platforms.
- Collaborate with cross-functional teams to understand and deliver business requirements.
- Monitor, troubleshoot, and support production data workflows.
Required Skills
- Strong experience with Python and PySpark.
- Strong expertise of optimizations
- Hands-on experience with AWS Glue, Lambda, S3, Athena, Redshift, and DynamoDB.
- Strong SQL and data analysis skills.
- Experience building and optimizing large-scale data pipelines.
- Good understanding of data warehousing and data lake concepts.
Good to Have
- Experience with Git and CI/CD pipelines.
- Knowledge of Infrastructure as Code (Terraform or CloudFormation).
- Experience with data quality and data governance frameworks.
- AWS certifications.
- Exposure to real-time or event-driven data processing solutions.
Skills
- amazon-web-services
- pyspark
- aws-glue
- serverless
- aws-lambda
- Python
- amazon-s3
- amazon-athena
- amazon-redshift
- amazon-dynamodb
- Terraform
