DevOps Engineer – Big Data
- Source: LinkedIn
- Naviteq Ltd
- Remote
- United States
- Full-Time
- Mid level
Job Description
About the Company
A global adtech company with 1,200 employees and offices in the US, Israel, Europe and APAC. It runs an end-to-end advertising platform (DSP + SSP) for leading global brands. Its systems process millions to billions of events per second, so reliable, scalable data infrastructure is core to the business.
The Role
We’re looking for a DevOps Engineer with hands-on Big Data experience to build, run and scale the cloud infrastructure behind our high-throughput data platforms. You’ll combine classic DevOps work on AWS, Terraform and EKS with configuring, tuning and operating large-scale data systems in production.
Responsibilities
- Design, build and maintain cloud infrastructure on AWS using Terraform (IaC)
- Deploy and operate containerised workloads on Kubernetes (EKS)
- Configure, manage, scale and tune Big Data platforms such as Hadoop, Kafka, ClickHouse, Redis or Elasticsearch
- Ensure high availability, performance and data reliability of clusters handling very large data volumes
- Build and improve CI/CD pipelines and automation
- Set up monitoring, alerting and logging; troubleshoot production incidents and drive root-cause analysis
- Plan capacity, upgrades and cost optimization of the data infrastructure
- Work closely with data and backend engineering teams to support their platforms
Requirements
- 5+ years as a DevOps / Infrastructure / SRE engineer
- Strong hands-on experience with AWS (must)
- Strong experience with Terraform (must)
- Production experience with Kubernetes, preferably EKS (must)
- Hands-on experience configuring and managing at least one Big Data platform in production: Hadoop, Kafka, ClickHouse, Redis or Elasticsearch (must)
- CI/CD tools (Jenkins, GitHub Actions, GitLab CI, ArgoCD)
- Scripting in Python, Bash or Go
- Monitoring and observability tools (Prometheus, Grafana, ELK, Datadog)
- Solid Linux and networking fundamentals
- Strong English; communicative, independent, proactive team player
What we offer:
- A great team of experienced engineers who value collaboration and knowledge sharing
- A large-scale global project used by the world’s best-known brands
- Fully remote work with a flexible, distributed team
- Real ownership and influence over architecture and technical decisions
Skills
- amazon-web-services
- Terraform
- Kubernetes
- Hadoop
- apache-hadoop
- Kafka
- clickhouse
- Redis
- Elasticsearch
- Jenkins
- Github
- github-actions
- gitlab
- gitlab-ci
- argo-cd
- Python
- Golang
- Prometheus
- Grafana
- elk
- datadog
