Lead Data Engineer – Azure Data Platform & Architecture (ADF)

We need a Lead Data Engineer who can do both halves of the job well: set the architecture and technical direction for our Azure data platform and still be in the pipelines writing code and fixing things when they break. This isn’t a role where you hand off the hard problems to someone else - you’ll be the one deciding how the platform should be built, and also one of the people building it.

You’ll take ownership of our Azure Data Factory environment and Data Lake architecture and play a lead role in our SAP migration (BDC, Datasphere) - making sure data moves reliably and cleanly across systems as we modernize.

What you’ll be doing

Setting direction

• Own the target architecture for our Azure data platform - ADF, Data Lake, SQL, and everything that connects them
• Decide on patterns and standards for pipelines, data modeling, and governance, and make sure the team actually follows them
• Lead design reviews for new integrations, especially around the SAP migration
• Keep an eye on where the platform should go next - Synapse, Databricks, Fabric - and make the call on what’s worth adopting
• Review other engineers’ work and help them get better at this

Actually building it

• Build and administer ADF environments across dev, test, and prod - this is hands-on, not just oversight
• Design and develop ETL/ELT pipelines, and get in there yourself when something’s slow or broken
• Configure linked services, datasets, triggers, integration runtimes
• Own security and access control within ADF
• Run CI/CD and releases for the platform

Pipelines and SQL

• Build pipelines that pull data from multiple sources into the Data Lake and transform it well
• Write and tune complex SQL - stored procedures, queries, the works
• Keep reporting layers consistent by staying involved in the data modeling

Data Lake structure

• Organize and maintain the raw/curated/presentation layers of the Data Lake
• Hold the line on data quality and governance across the platform
• Make sure the architecture can actually scale as the business grows, not just today

SAP work

• Provide the architectural thinking behind our SAP integration (BDC, Datasphere)
• Lead migration planning, mapping, and validation - and pitch in on the actual build when needed
• Build the pipelines connecting Azure and SAP

Working with people

• Partner with analysts, stakeholders, and IT to figure out what the platform needs to do next
• Turn business asks into real architecture, not just quick fixes
• Push modernization forward across the org, not just within your own team

What you’ll need

• 8–10 years in data engineering or data integration, including some time leading technical decisions or mentoring other engineers
• Strong, real hands-on experience with Azure Data Factory - both building and administering it
• Advanced SQL skills, including performance tuning on non-trivial queries
• A track record of architecting solutions on Azure Data Lake and similar modern platforms
• Solid grasp of ETL/ELT design at an enterprise scale
• Comfort doing both things at once: setting direction and rolling up your sleeves to fix a production issue yourself
• Good communication - you’ll need to explain architecture decisions to both engineers and non-technical stakeholders

Nice to have

• Experience with SAP BDC, Datasphere, or similar SAP data platforms
• Familiarity with Synapse, Databricks, or Microsoft Fabric
• CI/CD experience (Azure DevOps or similar)
• Background in data governance, security, and compliance
• You’ve been in a lead or architect-track role before and know what it’s like to balance both sides of it

Pay: $6,000.00 - $7,000.00 per month

Application Question(s):

• Describe your hands-on experience with Azure Data Factory. What types of pipelines have you built and administered, and which ADF components (linked services, datasets, triggers, integration runtimes, security, CI/CD) have you worked with?
• Tell us about an Azure Data Lake architecture you designed or led. How did you approach the raw/curated/presentation layers, ETL/ELT design, data modeling, scalability, governance, and performance?
• Describe a complex data engineering or migration project where you acted as the technical lead while remaining hands-on. What architecture decisions did you make, what technologies did you use (e.g., SQL, SAP, Synapse, Databricks, Fabric), and what production issues did you personally troubleshoot or resolve?
• If you have advanced english and agree to work as a …