Data & AI Engineer

Fund or Department Description

The Data & AI Engineer sits within Carlyle’s Enterprise Technology & Data organization and supports firm-wide data and AI initiatives spanning investment platforms, portfolio operations, investor relations, and corporate functions. The role operates within a federated data operating model, partnering with domain engineering teams to implement shared platforms and reusable patterns for data and AI under the technical direction of the Senior AI & Data Architect.

Position Summary

The Data & AI Engineer is an experienced, hands-on engineer who turns Carlyle’s data and AI architecture into working production systems. Reporting to the Senior AI & Data Architect, this role is responsible for building and operating the pipelines, semantic layers, retrieval systems, and AI-ready data products that power analytics, automation, LLMs, agents, and generative AI applications across the firm.

The role requires deep, hands-on expertise across modern data engineering and applied AI engineering. The Data & AI Engineer will implement retrieval-augmented generation (RAG) patterns, embedding and indexing pipelines, vector stores, and semantic models alongside core ELT, streaming, and analytical pipelines - treating LLMs, agents, and copilots as first-class consumers of the data platform.

This is a senior individual-contributor engineering role that executes against architectural standards, contributes to their evolution through hands-on learning, and partners closely with data science, AI engineering, governance, and domain teams to deliver trusted, AI-consumable data at enterprise scale.

What Success Looks Like: In the first 12 months, this role will deliver foundational AI-ready data pipelines and retrieval components defined in the target-state architecture, productionize one or more priority RAG or agent-grounding use cases, and establish reusable engineering patterns that other domain teams can adopt across the federated data platform.

In-office requirement: 4 days per week

Location: Washington, D.C. or New York, NYAI Data Pipelines & Retrieval Systems (≈35%)

* Build and operate AI-ready data pipelines - embedding generation, chunking, indexing, and refresh workflows - that make Carlyle’s enterprise data reliably retrievable by LLMs, agents, and generative AI applications.

* Implement retrieval-augmented generation (RAG) components, including vector store integrations, hybrid search, re-ranking, and grounding logic, against architectural patterns defined by the Senior AI & Data Architect.

* Develop and maintain tool and function interfaces that allow agents and copilots to query and act on enterprise data safely, with appropriate guardrails, logging, and evaluation hooks.

* Partner with Data Science and AI Engineering teams to operationalize feature stores, evaluation datasets, and reusable AI data products.

* Contribute to semantic and context engineering work that powers natural-language analytics, conversational reporting, and AI-driven insights for business users.

Modern Data Pipeline Engineering (≈30%)

* Design, build, and maintain production-grade ELT, streaming, and transformation pipelines using tools such as dbt, Fivetran and Snowflake.

* Implement ingestion, modeling, and consumption patterns that meet enterprise standards for scalability, performance, security, resiliency, and cost efficiency.

* Write clean, well-tested Python and SQL; apply software engineering best practices including version control, code review, CI/CD, modular design, and automated testing.

* Productionize new sources and domains under the federated operating model, partnering with domain data engineers to apply shared platform capabilities consistently.

Semantic Layer & Data Product Development (≈20%)

* Implement semantic models, data contracts, and analytical/dimensional models that enable trusted self-service analytics and reliable AI grounding.

* Build and maintain reusable data products with clear ownership, documented contracts, and contextual metadata suitable for both human and AI consumers.

* Collaborate with the Senior AI & Data Architect to refine and extend enterprise semantic standards based on what works in production.

* Support discovery and consumption tooling so that analysts, applications, and agents can find and use data products with minimal friction…