DISTINGUISHED, SOFTWARE ENGINEER

Position Summary… What youll do…

Role : Distinguished Engineer, Software Engineering

About The Team

Run Time Platforms (RTP), a division of Global Technology Platform (GTP), is dedicated to developing products and platforms that empower application teams to efficiently deploy and manage runtime environments.

The Runtime Platforms pillar within Global Tech Platforms (GTP) owns the foundational infrastructure and developer-facing platform services Expo, Pulse, CCM, and Secrets that every engineering team at Walmart depends on to build, deploy, and run their software safely and at hyper-scale. This pillar is also responsible for an Enterprise Feature Store platform that will be used by US OMNI customers spanning search, personalization, Ads, Marketplace, Fraud, Payments and more. We are talking about a platform that needs to provide and adhere to an SLA p95 of 10ms for online feature retrieval.

What youll do:

Technical Leadership & Strategy

Define and own the short term and long-term technical vision for the ML data and feature store platform, spanning offline pipelines, online serving, and the infrastructure that connects them.

Identify and systematically retire architectural debt across the platform prioritising by production impact, latency, reliability, and engineering velocity.

Serve as the primary technical authority in cross-functional design reviews involving ML engineering, data engineering, platform, and product teams.

Influence the broader engineering organisation through published architectural standards, internal RFCs, and mentorship of senior and staff engineers.

Platform Architecture & Design

Architect end-to-end feature store data systems from data ingestion and feature computation through materialization into online stores and delivery to model inference.

Define and Design high performant, scalable, reliable Data and Control Mgmt planes for both online and offline data processing pipelines.

Define and govern data quality standards, SLOs, and observability frameworks across the platform (feature freshness, materialization lag, training-serving skew detection, drift alerting).

Drive platform decisions around schema design, storage backend selection, consistency trade-offs, and multi-cloud data replication strategies.

Offline Data Systems (Big Data Ecosystem)

Design and optimise large-scale batch data pipelines using Spark, Hive, or equivalent covering feature computation, aggregation, transformation, and materialization into both offline and online stores.

Define offline store schema and partitioning strategies (e.g., Parquet/ORC on cloud object storage) optimised for point-in-time correct training dataset generation.

Establish pipeline reliability and observability standards: data freshness SLOs, quality checks at ingestion, deduplication, late-arrival handling, and lineage tracking.

Provide guidance on orchestration and scheduling of batch workloads (Airflow or equivalent), including dependency management, SLA monitoring, and failure recovery.

Online Data Systems (High-Throughput Serving)

Architect and optimise online feature serving infrastructure targeting high QPM at consistently low latency (p99 in single-digit milliseconds), across multi-region, multi-cloud deployments.

Design data access layers over low-latency stores (Cassandra, Redis, Spanner, or equivalent) including read path optimization, caching strategy, connection pool management, and circuit breaking.

Define online serving reliability patterns: multi-DC failover, graceful degradation, bulkhead isolation, and zero-downtime schema migrations.

Establish capacity planning and load testing practices for online stores; translate traffic forecasts into storage and compute provisioning recommendations.

SDK Design & Developer Experience

Lead the design and evolution of the platforms Java and Python SDKs from the current thick SDK (heavy direct data store coupling) to a thin SDK that handles only routing table fetching, feature name parsing, and request dispatch.

Own the Java SDKs API contract, versioning strategy, backward compatibility guarantees, and deprecation policy (including serializer migration across major versions).

Define ergonomic, well-documented Python SDK interfaces for data science and ML engineer…