Senior Software Engineer Data Platform (Tech Lead)

Job Description

Own and govern the data ingestion and integration architecture for the enterprise data platform.

Define and evolve scalable, reusable, and production-grade data interfaces across all source systems and business domains.

Provide technical leadership and design authority for all data platform ingestion and integration components, ensuring engineering quality, scalability, and reliability.

Establish and enforce software engineering and architecture standards across data platform development (CI/CD, testing, observability, design patterns).

Act as the primary escalation point and technical authority for complex ingestion, integration, and data platform challenges.

Guide and support a distributed team of software engineers and data engineers in building production-grade ingestion services and platform components.

Collaborate closely with platform architecture, cybersecurity, infrastructure, and business IT teams to ensure secure, compliant, and sustainable system design.

Drive consistency and reuse across data platform patterns, frameworks, and engineering practices.

Main Tasks:

Define and evolve ingestion frameworks for batch, streaming, API-based, and event-driven architectures.

Establish and govern standards for structured and unstructured data (JSON, CSV, XML), including schema evolution and compatibility strategies.

Design and review production-grade ingestion and integration patterns, ensuring fault tolerance, observability, and performance.

Define and enforce API design standards, reliability patterns, and contract management.

Lead design reviews for high-risk or complex ingestion pipelines, focusing on scalability, security, and maintainability.

Support engineers in implementing robust error handling, retry mechanisms, orchestration, and monitoring.

Collaborate with system owners to define interface specifications and integration strategies.

Align ingestion architecture with overall platform design, governance, and business requirements.

Enable ingestion patterns for advanced use cases, including data science, machine learning, and AI/LLM integrations.

Maintain and evolve reusable libraries, templates, and framework components to accelerate development.

Promote engineering best practices, including code reviews, testing strategies (unit, integration), and CI/CD pipelines.

Define and maintain architecture blueprints, design guidelines, and engineering standards.

Create reusable architecture patterns for lakehouse-based data platform.

Provide guidance on scalability strategies, resource utilization, and performance optimization.

Support cost transparency, usage optimization, and efficient resource consumption across the platform.

Qualifications

Qualifications

Degree in Computer Science or a related field.

6 10 years of experience in software engineering and data platform development, with a focus on large-scale, enterprise-grade systems.

Proven track record of technical leadership and architectural ownership in distributed systems or data platforms.

Strong hands-on experience with Scala / Java and Python in a production software engineering environment.

Deep understanding of data integration patterns (APIs, CDC, streaming, event-driven architectures).

Experience designing and governing reusable frameworks, platform components, and engineering standards.

Strong knowledge of CI/CD, version control, automated testing, and observability practices.

Experience with Microsoft Azure or other hyperscaler environments.

Familiarity with lakehouse and data warehousing architectures; Databricks experience is a plus.

Ability to guide and mentor engineers while collaborating across global teams and stakeholders.

NOT a fit if your experience is primarily:

Notebook-based data processing (e.g. Jupyter, ad-hoc PySpark) without production deployment

Dashboarding / analytics / BI tools

SQL-only roles without software engineering experience

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.