Aws Data Engineer

Dear Professionals

Greetings from Tata consultancy Services,

Job Title : AWS Data Engineer With Spark and Scala

Experience : 6-10 years

Location: Hyderabad/Chennai/ Bangalore/Pune/Delhi

Mode of Work : Work from Office

Mode of Interview: Virtual

Key Responsibilities

• Pipeline Development: Design, build, and maintain highly scalable batch and real-time streaming data pipelines using Apache Spark and Scala.
• AWS Data Solutions: Architect and deploy modern data platform components leveraging AWS services including S3, EMR, Glue, Redshift, Athena, and Lambda.
• Performance Tuning: Optimize and tune Spark application engines for high-volume datasets consisting of billions of records.
• Orchestration & Workflow: Build, manage, and monitor workflow dependencies using orchestration tools like Apache Airflow or AWS Step Functions.
• Data Modeling & Quality: Implement efficient data schemas, Medallion Architecture (Bronze/Silver/Gold layers), and automated data quality checks.
• Troubleshooting & Ops: Perform root cause analysis on production pipeline failures and debug complex distributed computing bottlenecks.
• Governance & Compliance: Enforce enterprise-grade data security, IAM policy frameworks, and data governance standards.
• Technical Leadership: Collaborate with architects, manage stakeholder expectations, and provide technical guidance to a small squad of data engineers.

Required Technical Skills

• Core Experience: 6+ years in dedicated Data Engineering or Big Data roles.
• Programming: Absolute proficiency in Scala (Mandatory) along with a strong foundation in SQL.
• Big Data Frameworks: In-depth knowledge of Spark Core, Spark SQL, DataFrames/Datasets, and Hadoop ecosystems (HDFS, Hive, YARN).
• AWS Cloud Services: Extensive experience configuring and optimizing managed big data workloads on AWS EMR, Glue, and Redshift.
• Databases & Storage: Hands-on experience dealing with massive structured/unstructured datasets across SQL, NoSQL (e.g., MongoDB, Cassandra), and modern Lakehouses.
• DevOps / CI-CD: Strong familiarity with Git version control, Jenkins pipelines, Docker, and agile deployment infrastructure.

Preferred/Good-to-Have Skills

• Exposure to real-time streaming tools like AWS Kinesis, Apache Kafka, or Spark Streaming.
• Familiarity with AWS CloudFormation, Lake Formation, and API Gateways.
• An active AWS Certified Data Engineer Associate or Professional certification.

If you are Interested in the above opportunity kindly share your updated resume to [email protected] immediately with the details below (Mandatory)

Name:

Contact No.

Email id:

Skillset:

Total exp:

Relevant Exp:

Fulltime highest qualification (Year of completion with percentage scored):

Current organization details (Payroll company):

Current CTC:

Expected CTC:

Notice period:

Current location:

Preffered Location

Any gaps between your education or career (If yes pls specify the duration):

Will you be able to join within 30/45 days? (Yes/NO)