AWS Databricks Architect -Sr Staff

Company Qualcomm India Private Limited

Job Area Information Technology Group, Information Technology Group > IT Data Engineer

General Summary The Databricks Architect will be responsible for technical decision-making, code quality, and the successful delivery of complex data projects using Delta Lake, Unity Catalog, and Mosaic AI. The Databricks Architect/Lead owns the end-to-end strategy for data ingestion, storage, and analytics and serve as a liaison between business stakeholders and engineering teams, ensuring that the platform is secure, cost-optimized, and capable of supporting advanced AI/ML workloads.

Key Responsibilities

• Lakehouse Design: Architect end-to-end data solutions using the Medallion Architecture (Bronze/Silver) and Delta Lake.
• Platform Governance: Implement and manage Unity Catalog for centralized data governance, fine-grained access control, and lineage tracking.
• Technical Leadership: Lead and mentor cross-functional teams of data engineers; conduct code reviews and define engineering best practices in Python, PySpark, and SQL.
• Optimization & Performance: Drive cost-efficiency by tuning Spark jobs, managing cluster policies, and leveraging serverless compute and auto-scaling.
• DevOps & DataOps Integration: Integrate Databricks workflows into CI/CD pipelines using tools like Databricks Asset Bundles, Terraform, and Azure DevOps or GitHub.
• AI/ML Support: Provide architectural guidance for integrating machine learning models, including feature engineering and model deployment via MLflow.
• Deep hands-on technical understanding in building complex data models for supporting various analytics needs
• Comfortable in working with cross-functional teams such as Data Engineering, Analytics, SMEs and Business stakeholders
• Passion to understand complex existing footprint of data solutions and build out the strategy for migrating to Databricks medallion architecture
• Accountability and Ownership of the delivery in driving the projects to completion
• 14+ years of experience in data management, data engineering, or data science roles, with at least 7 years in building enterprise level data models as part of data modernization and DW migration efforts.
• 8+ years of work experience with Data Structures and algorithms
• 8+ years of work experience with programming (e.g., Java, Python)
• 8+ years of work experience with SQL or NoSQL Databases
• 3+ years of work experience in a role requiring interaction with senior leadership, leading modernization of Data platforms.

Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field.

• Strong understanding and hands-on experience with lakehouse medallion data architecture and govenance
• Proven implementation experience with migrating legacy DW data models to 3-layer medallion architecture.
• Strong passion in exploring GenAI based solutions for documenting the existing DW footprint and build out strategic data model for Databricks platform
• Experience with machine learning, artificial intelligence, and predictive analytics is a plus.
• Excellent leadership, communication, and collaboration skills in a global hybrid delivery model
• Strong problem-solving skills with strategic thought process and self-initiative and proctive soft skills

Minimum Qualifications

• 7+ years of IT-related work experience with a Bachelor’s degree in Computer Engineering, Computer Science, Information Systems or a related field.

OR

9+ years of IT-related work experience without a Bachelor’s degree.

• 5+ years of work experience with programming (e.g., Java, Python).
• 3+ years of work experience with SQL or NoSQL Databases.
• 3+ years of work experience with Data Structures and algorithms.

Applicants Qualcomm is an equal opportunity employer. If you are an individual with a disability and need an accommodation during the application/hiring process, rest assured that Qualcomm is committed to providing an accessible process. You may e-mail [email protected] or call Qualcomm’s toll-free number found here. Upon request, Qualcomm will provide reasonable accommodations to support individuals with disabilities to be able participate in the hiring process. Qualcomm is also committed to making our workplace accessible for individuals with disabilities. (Keep in mind that this email address is used to provide reasonable accommodations for individu…