Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end-to-end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com.

Job Title: Azure Databricks
Position: Software Engineer
Experience: 8- 12 Years
Category: Software Development/ Engineering
Shift: 1-10PM (Hybrid)
Main location: Bangalore / Chennai
Position ID: J0726-1184
Employment Type: Full Time

Education Qualification: Bachelor’s degree in Computer Science or related field or higher with minimum 3 years of relevant experience.

Position Description: We are looking for an experienced Azure Databricks to join our team. The ideal candidate should be passionate about coding and developing scalable and high-performance applications. The successful candidate will be responsible for designing, developing, optimizing, and supporting enterprise-grade ETL and ELT pipelines primarily using Azure Databricks, PySpark, SQL, and related Microsoft Azure services. This is a hands-on engineering role focused on building scalable, reliable, and high-performing data pipelines across development, testing, and production environments. The engineer will work closely with data architects, business analysts, application teams, quality assurance teams, and production support teams to deliver secure and well-governed data solutions in a regulated financial-services environment.

LI-BK7

Your future duties and responsibilities

• Design, develop, test, deploy, and support scalable ETL/ELT pipelines using Azure Databricks, PySpark, Spark SQL, Delta Lake, and Azure Data Factory.
• Implement end-to-end Lakehouse and Medallion Architecture patterns across Bronze, Silver, and Gold data layers, with strong lineage, replayability, and operational controls.
• Develop advanced ingestion patterns including full loads, incremental loads, CDC, Auto Loader, schema evolution, watermarking, late-arriving data handling, and restartable/idempotent processing.
• Build and maintain Spark Declarative Pipelines / declarative Databricks pipeline patterns with dependency management, data quality expectations, observability, and production-grade orchestration.
• Design and implement Data Vault 2.0 structures including Hubs, Links, Satellites, PIT tables, hash keys, hash diffs, Raw Vault and Business Vault patterns where appropriate.
• Design Kimball dimensional models including facts, dimensions, conformed dimensions, slowly changing dimensions, role-playing dimensions, and bridge tables for downstream analytical consumption.
• Develop reusable PySpark and Spark SQL frameworks for cleansing, standardization, enrichment, deduplication, reconciliation, and business-rule processing.
• Implement enterprise data quality using DQX, including completeness, uniqueness, validity, consistency, referential integrity, auditability, and quarantine/reject patterns.
• Create and maintain audit logging, source-to-target reconciliation, control totals, record-count balancing, SLA monitoring, and production support dashboards/alerts.
• Optimize Spark and Delta Lake workloads using partitioning, clustering, caching, broadcast joins, AQE, skew handling, file compaction, OPTIMIZE/VACUUM, and efficient query design.
• Build and support Azure DevOps CI/CD pipelines for Databricks code, notebooks, declarative pipelines, ADF assets, configuration, and environment promotion.
• Use GitHub Copilot responsibly for development acceleration, code review support, test generation, debugging, and productivity while applying appropriate validation and secure coding practices.
• Troubleshoot and debug failed or slow production jobs, perform root-cause analysis, restore service within SLA, and implement permanent corrective actions.
• Work closely with architects, business analysts, QA/QE, Power BI developers, and application teams to ensure data products meet business, quality, performance, security, and reporting requirements.

Required qualifications to be …