Sr. Data Engineer / AI Data Platform Engineer
- Source: Naukri
- Info Services
- Hybrid
- Hyderabad
- Full-Time
- Senior
Job Description
Must-haves:
- Python or Scala
- Apache Spark
- Apache Airflow
- Databricks on AWS
- RAG and LLM orchestration frameworks
Required skills:
- Python and/or Scala
- Apache Spark
- Apache Airflow
- Databricks on AWS
- AWS ecosystem (S3, Lambda, Glue)
- Data lake architecture and Medallion (Bronze, Silver, Gold)
- Streaming and batch data pipelines
- REST/GraphQL APIs and data integration patterns
- Data governance, cataloging, monitoring tools
- LLM orchestration frameworks
- RAG architectures
- Model Context Protocol (MCP)
- AWS Bedrock
- AWS Nova Pro
Preferred skills:
- Media & Entertainment or digital platforms
- Social media analytics and sentiment analysis
- Real-time processing (Kafka, Kinesis)
- MLOps and model lifecycle management
Skills
- Python
- Scala
- Spark
- Airflow
- apache-airflow
- azure-databricks
- databricks
- amazon-web-services
- amazon-s3
- aws-lambda
- aws-glue
- GraphQL
- bedrock
- Kafka
