Senior Software Engineer -Data Engineer

Founded in 1976, CGI is among the largest independent IT and business consulting services firms in the world. With 94,000 consultants and professionals across the globe, CGI delivers an end to end portfolio of capabilities, from strategic IT and business consulting to systems integration, managed IT and business process services and intellectual property solutions. CGI works with clients through a local relationship model complemented by a global delivery network that helps clients digitally transform their organizations and accelerate results. CGI Fiscal 2024 reported revenue is CA$14.68 billion and CGI shares are listed on the TSX (GIB.A) and the NYSE (GIB). Learn more at cgi.com.
Job Title: Data Engineer
Position: Senior Software Engineer
Experience: 4-7 Years
Category: Software Development/ Engineering
Shift: General Shift
Main location: India, Karnataka, Bangalore
Position ID: J0626-2182
Employment Type: Full Time
Education Qualification: Bachelor’s degree in Computer Science or related field or higher with minimum 4 years of relevant experience.
#LI-SK38

Your future duties and responsibilities

We are looking for an experienced Data Engineer to join our team. The ideal candidate should be passionate about building scalable data architectures, developing efficient ETL/ELT pipelines, and creating robust, high performance data systems. You will work closely with our Data Science, Analytics, QA, and DevOps teams to deliver seamless data integration and robust solutions that support our business intelligence and strategic objectives.

Responsibilities:

. Design, develop, and maintain scalable data pipelines and architectures using Python, SQL, and big data technologies.
. Build, manage, and optimize modular, testable, and efficient data ingestion and transformation scripts.
. Integrate data from various sources including RESTful APIs, third party services, and internal databases.
. Collaborate with Data Scientists, Analysts, and Backend developers to deliver end to end data solutions.
. Write clean, efficient, and maintainable code following data engineering best practices.
. Optimize pipeline performance, processing speed, and scalability for large scale datasets.
. Implement complex data flows and state management using orchestration tools and reactive data processing frameworks.
. Troubleshoot, debug, and resolve data quality, pipeline failures, and performance bottlenecks.
. Participate in code reviews and maintain high coding standards for data infrastructure.
. Ensure data security, integrity, and protection practices are followed (e.g., encryption, access control, GDPR/SOC2 compliance).
. Contribute to data architecture, modeling, and technical design discussions.
. Support deployment processes and work with data centric CI/CD pipelines (DataOps).
. Stay updated with emerging data technologies, cloud updates, and suggest improvements to the data stack.

Required qualifications to be successful in this role

Must Have Skills:

. Strong proficiency in Python or Scala and advanced SQL (complex joins, window functions, optimization).
. Hands on experience with core data technologies including Spark, Hadoop, or similar distributed processing frameworks.
. Solid understanding of data modeling (Star/Snowflake schema), data warehousing, and lakehouse architectures.
. Experience with workflow orchestration tools like Apache Airflow, Prefect, or Dagster.
. Strong experience in building and consuming APIs for data ingestion and asynchronous data handling.
. Proficiency in Git and version control workflows.
. Understanding of software design principles (modular design, reusability, maintainability applied to data).
. Experience with query optimization, performance tuning, and database indexing.
. Familiarity with data security practices (IAM roles, encryption at rest/transit, data masking).
. Solid understanding of batch and real time data processing patterns.
. Good to Have Skills:
. Experience with cloud data warehouses like Snowflake, BigQuery, or Redshift.
. Exposure to NoSQL databases and search engines (e.g., MongoDB, Cassandra, Elasticsearch).
. Familiarity with data validation frameworks (e.g., Great Expectations, dbt).
. Experience with streaming technologies like Apache Kafka, Flink, or Spark Streaming.
. Working knowledge of Infrastructure as Code (Terraform) and Cloud services (AWS, Azure, or GCP).
. Understan…