Principal Data Architect - Multi-Cloud (GCP & Azure)
- Source: Lever
- Egen
- Hybrid
- Hyderabad
- Full-Time
- Staff
Job Description
We are seeking a Principal Data Architect to define and drive enterprise data architecture, modernization, and governance strategies across complex multi-cloud (Google Cloud Platform and Microsoft Azure) and hybrid environments. This role will design scalable, secure, and high-performing data platforms while partnering with business and engineering leaders to translate strategic objectives into impactful data solutions.
A key focus of this role will be spearheading scalable, agent-driven data migration pipelines to transition legacy on-premises data assets to the cloud, setting the technical direction for engineering teams, and actively managing the customer’s technical backlog and delivery roadmap.
Key Responsibilities
- Multi-Cloud Lakehouse Architecture: Lead the end-to-end architecture, design, and implementation of scalable Data Lakehouse solutions across Google Cloud Platform (BigQuery, GCS, BigLake, Dataplex) and Microsoft Azure (Azure Synapse Analytics, Microsoft Fabric, ADLS Gen2, Azure Databricks).
- On-Prem to Cloud Migration & Agentic Pipelines: Architect and deploy automated, scalable, agent-driven data pipelines (leveraging GenAI agents and metadata-driven frameworks) to orchestrate, validate, and accelerate complex migrations from on-premises legacy warehouses/databases (Oracle, Teradata, SQL Server) to GCP and Azure.
- Client Engagement & Advisory: Collaborate directly with customers to understand business goals, data challenges, and technical requirements; translate them into robust architectural blueprints, technical roadmaps, and actionable delivery plans.
- Customer Technical Backlog Management: Own and actively manage the customer technical backlog—translating business epics into technical user stories, prioritizing technical debt, defining sprint milestones, and aligning backlog items with business ROI.
- Modern Ingestion & Streaming: Design and implement data pipelines supporting both real-time and batch ingestion using modern orchestration (Cloud Composer, Azure Data Factory, Airflow) and streaming/messaging frameworks (Kafka, GCP Pub/Sub, Azure Event Hubs).
- Governance & Lineage: Establish and enforce best practices for unified data cataloging, metadata management, automated lineage, and data quality across multi-cloud and on-prem systems using Dataplex, Microsoft Purview, or open-source governance tools.
- Security & Compliance: Define and implement robust data security, identity management (IAM / Azure Entra ID), encryption, row/column-level access control, and regulatory compliance standards across clouds.
- Team Technical Leadership & Mentorship: Steer the overarching technical direction of engineering teams; establish architectural standards, mentor senior and mid-level engineers, conduct architecture review, and ensure engineering excellence.
- Hands-on Delivery & Prototyping: Balance strategic architecture discussions with hands-on solutioning, architectural spikes, Proof-of-Concepts (POCs), and performance tuning of complex multi-cloud distributed workloads.
- Cross-Functional Partnership: Partner with enterprise architects, delivery leads, DevOps engineers, and customer stakeholders to drive solution adoption, operational scalability, and long-term platform maintainability
Technical Experience & Tooling
Candidates should demonstrate hands-on architecture and design experience across the following domains:
- Cloud Platforms: Deep expertise in both Google Cloud Platform (GCP) and Microsoft Azure enterprise data ecosystems.
- Processing Engines: Apache Spark, PySpark, Databricks, Azure Synapse / Fabric Spark, GCP Dataproc.
- Modern Migration & AI Automation: Building automated, agentic, or metadata-driven data migration pipelines, automated schema conversion, and data validation agents for on-prem to cloud lift-and-shift/re-platforming.
- Messaging & Streaming: Apache Kafka, GCP Pub/Sub, Azure Event Hubs.
- Orchestration: Apache Airflow, Google Cloud Composer, Azure Data Factory (ADF).
- Analytical Storage & Warehouses: GCP BigQuery, Azure Synapse, Microsoft Fabric, Delta Lake/Parquet.
- NoSQL / Cache Storage: MongoDB, Redis, Bigtable, Azure Cosmos DB.
- Containerization & Deployment: Kubernetes (GKE and AKS), Cloud Run, Azure Container Apps.
- API Frameworks: Python (FastAPI, Flask) for data service endpoints.
- Databases & Querying: Advanced SQL validation and extraction from legacy/RDBMS systems (SQL Server, Oracle, PostgreSQL).
- CI/CD & Version Control: Git, Git…
Skills
- google-cloud-platform
- azure-arc
- microsoft-azure
- google-bigquery
- azure-synapse
- microsoft-azure-synapse
- azure-synapse-analytics
- google-analytics
- microsoft-fabric
- azure-databricks
- databricks
- oracle
- teradata
- azure-pipelines
- google-cloud-composer
- Composer
- azure-data-factory
- Airflow
- apache-airflow
- Kafka
- microsoft-entra-id
- level-access
- processing-js
- Spark
- pyspark
- azure-storage
- delta-lake
- MongoDB
- Redis
- azure-cosmos-db
- Kubernetes
- azure-container-apps
- Python
- FastAPI
- Flask
- PostgreSQL
- Github
- gitlab
- gitlab-ci
- azure-devops
- Terraform
- unity-3d
- Snowflake
- amazon-web-services
- amazon-s3
- amazon-redshift
- aws-glue
- amazon-emr
- LangChain
- LlamaIndex
- lever
