Senior Cloud Engineer - Azure Infrastructure & Databricks Platform
- Source: LinkedIn
- Net2Source (N2S)
- Remote
- New York City
- Full-Time
- Senior
Job Description
Join a Global Leader in Workforce Solutions - Net2Source Inc.
Who We Are
Net2Source Inc. isn’t just another staffing company, we’re a powerhouse of innovation, connecting top talent with the right opportunities. Recognized for 300% growth in the past three years, we operate in 34 countries with a global team of 5,500+. Our mission? To bridge the talent gap with precision—Right Talent. Right Time. Right Place. Right Price.
**Role: Senior Cloud Engineer - Azure Infrastructure & Databricks Platform
Location: US. Remote
Duration: 12+ Months** 1 Position
JD
Senior Cloud Engineer — Azure Infrastructure & Databricks Platform
Alternate titles that reach the same candidate pool: Senior Cloud Platform Engineer (Azure & Databricks) · Senior Azure Infrastructure Engineer — Data Platform · Senior Cloud Engineer, Databricks Platform
About the Role
We are seeking a Senior Cloud Engineer to design, implement, and administer the Azure infrastructure behind our enterprise data and AI platform. This is a cloud infrastructure and platform engineering position centered on Azure Databricks, ADLS Gen2, Azure Data Factory, and Azure Machine Learning.
You will own the platform foundation — infrastructure as code, network and identity architecture, compute governance, observability, and cost management — and partner closely with our Databricks administration, data engineering, and data science teams. Our group defines the platform standards and reference architectures those teams build within, which means depth in Databricks administration matters here as much as depth in Azure. You should be equally at home provisioning a workspace through Terraform and advising on Unity Catalog design, cluster policy, or identity federation.
This role suits an engineer who enjoys being the technical reference point for a platform: setting the architectural direction, partnering with adjacent teams on complex problems, and translating business requirements into the platform capabilities needed to support them.
Part One — Infrastructure Ownership
Design, implement, and administer the Azure foundation the platform runs on.
• Infrastructure as Code (Terraform-first): Author and maintain the reusable Terraform modules that provision the Azure data estate — Databricks workspaces, ADLS Gen2, Data Factory, Key Vault, networking, and Azure ML. Own remote state, module versioning, drift detection, and the pipelines that deploy them.
• Network & Security Architecture: Design and implement the private networking and security posture for the platform — private endpoints, hub-and-spoke topology, VNet injection, NSGs and firewall rules, managed identities, Key Vault-backed secret scopes, RBAC, and data exfiltration controls.
• Data Lake Infrastructure: Own ADLS Gen2 at the infrastructure layer: storage account architecture, hierarchical namespace, container and ACL strategy, lifecycle and tiering policy, encryption and key management, and access patterns for Databricks and downstream consumers.
• Data Factory Platform Operations: Own ADF from the platform side — integration runtime provisioning and scaling (self-hosted and managed VNet), linked service credential and managed identity design, environment promotion, deployment automation, and monitoring. Pipeline development sits with the data engineering team.
• AI/ML Platform Infrastructure: Provision and operate the infrastructure supporting machine learning workloads: Azure ML workspaces and compute clusters, GPU quota and capacity planning, MLflow model registry integration, model serving endpoints, and the identity and networking connecting them to Unity Catalog.
• Observability & Reliability: Build and maintain the monitoring layer — Azure Monitor, Log Analytics, diagnostic settings, alerting, and operational runbooks across the estate.
• FinOps & Capacity Planning: Own cost visibility and optimization across DBU and storage spend — tagging strategy, chargeback and showback, budget alerts, reserved capacity, and forward capacity planning.
• CI/CD & Environment Management: Own the deployment pipelines and promotion path across development, test, and production environments.
Part Two — Databricks Platform Administration
Day-to-day Databricks operations are supported by a dedicated administration team. This role establishes the platform standards and reference configurations that team works from, and serves as a senior technical partner on …
Skills
- microsoft-azure
- azure-databricks
- databricks
- workspace
- Terraform
- unity-3d
- azure-data-factory
- azure-key-vault
- vault
- drift
- ml-flow
- mlflow
- azure-monitor
- chargeback
- microsoft-entra-id
- NodeJS
- aws-auto-scaling
- photon
- serverless
- attribution
- boundary
- Spark
- azure-rbac
- Python
- powershell
- azure-machine-learning
- Kubernetes
- checkov
- Kafka
- hashicorp
You missed this one - but you don't have to keep missing out. Try Club members get matched with open opportunities before they're gone, so you're always first in line, never last.
