Senior Software Engineer - E3 -1.2 -Data Engineering

Job Title: Data Engineer

Group: E3 1.2

BU: CARIAD Business Unit India Location: Whitefield, Bangalore Experience: 3 - 6 Years

Key Responsibilities

• Data Processing & Engineering Build and optimize scalable ETL pipelines using PySpark & Databricks for large-scale automotive datasets; implement CI/CD pipelines for robust deployment.
• Cloud & Big Data Work with Azure Data Lake, Synapse, ADF, and Databricks for data ingestion, transformation, and storage.
• Analytics & Insights Perform statistical analysis, predictive modeling, and trend identification on vehicle test drive data, environmental parameters, and system KPIs.
• Dashboard & Reporting Develop interactive dashboards with Elasticsearch, Power BI, KPI visualizations, and real-time analytics for engineering teams and stakeholders.
• Collaboration & Documentation Partner with data scientists, vehicle engineers, and cross-functional stakeholders, ensuring clear documentation of data pipelines and processes.

Must-Have Skills

• Data Engineering & Cloud :
  • PySpark, Databricks, Azure Data Lake, Azure Synapse, Azure Data Factory .
  • Proficiency in ETL pipeline design, optimization, and CI/CD implementation .
• Analytics & Visualization :
  • Power BI , Elastic Search with expertise in KPI creation, Kibana and real-time reporting .
  • Ability to generate actionable insights from large-scale ADAS datasets .
• Automotive Data Expertise :
  • Strong experience with vehicle trace data analysis (e.g., CAN, LIN, Ethernet).
  • Hands-on with CANoe or similar automotive tools for ECU communication protocol analysis.
  • Knowledge of ECU communication protocols (CAN, LIN, FlexRay, Ethernet)
• Collaboration & Documentation :
  • Strong communication skills to work with cross-functional teams and document solutions effectively.

Nice-to-Have Skills

• AI/ML in ADAS domain :
  • Exposure to AI/ML model training, deployment, and performance monitoring .
  • Familiarity with Azure Machine Learning services .
• Big Data & Streaming Analytics :
  • Knowledge of Kafka, Event Hubs, or real-time data streaming for automotive use cases.
• Automation & DevOps :
  • Experience with CI/CD pipelines (Azure DevOps, GitHub Actions, Jenkins) for data workflows.