Data & AI Engineer

Data & AI Engineer

📍 Location:New York, NY, US

(Hybrid)

🏢 Industry: Investment management

💼 Work Setting: Hybrid

Are you passionate about building modern data platforms, leveraging AI to make data more accessible, and creating scalable analytics solutions that empower business users with actionable insights?

We are seeking a Data Engineer to design, build, and optimize cloud-based data infrastructure that powers reporting, analytics, and AI-driven decision-making. This role combines data engineering, analytics engineering, data warehousing, ETL development, and AI innovation to create trusted, scalable, and intelligent data solutions.

The ideal candidate brings expertise in AWS Redshift, SQL, Python, ETL pipelines, cloud data platforms, and AI/LLM technologies while thriving in a fast-paced environment with evolving business needs.

Key Responsibilities

Data Warehouse & Analytics Platform Development

• Design, build, and maintain a centralized cloud data warehouse environment.
• Consolidate data from multiple business systems and third-party applications.
• Ensure data platforms are scalable, secure, reliable, and optimized for analytics workloads.
• Support enterprise reporting and business intelligence initiatives.
• Maintain data governance, access controls, and platform documentation.

Focus Areas

• Data Warehousing
• Cloud Data Platforms
• Data Architecture
• Analytics Engineering
• Business Intelligence

ETL / ELT Pipeline Development

• Build and maintain ETL/ELT pipelines that ingest, transform, and load data from diverse data sources.
• Integrate operational, transactional, customer, workforce, and reporting data into a unified environment.
• Automate scheduled data processing workflows.
• Ensure data availability, consistency, and reliability.
• Monitor and improve pipeline efficiency and performance.

Technologies

• AWS Glue
• AWS Lambda
• Apache Airflow
• Python
• SQL

Data Modeling & Architecture

• Design scalable schemas and analytical data models.
• Optimize warehouse structures for reporting and data consumption.
• Develop efficient partitioning, distribution, and indexing strategies.
• Support evolving analytical requirements and business growth.
• Improve query performance and system scalability.

Responsibilities

• Data Modeling
• Schema Design
• Performance Optimization
• Data Architecture
• Analytics Readiness

Data Quality, Monitoring & Reliability

• Monitor pipeline health and overall platform performance.
• Implement automated validation and quality checks.
• Develop alerting and monitoring frameworks.
• Troubleshoot data issues and prevent reporting disruptions.
• Improve reliability of mission-critical analytics solutions.

Areas of Focus

• Data Quality
• Data Validation
• Monitoring & Alerting
• Incident Resolution
• Reliability Engineering

Data Warehouse Performance & Cost Optimization

• Optimize query performance and warehouse utilization.
• Monitor infrastructure costs and resource consumption.
• Implement scalable solutions that balance performance and efficiency.
• Improve platform reliability as data volume grows.
• Ensure cost-effective cloud operations.

AI-Powered Analytics Solutions

• Design and develop AI-powered analytics capabilities.
• Build intelligent solutions that enable users to interact with data using natural language.
• Improve accessibility of data insights for non-technical users.
• Enable self-service reporting and analytics experiences.
• Support adoption of AI-driven decision-making tools.

AI Focus Areas

• Conversational Analytics
• Natural Language Querying
• AI Assistants
• Self-Service Analytics
• Intelligent Reporting

LLM & Generative AI Development

• Develop and enhance AI-driven reporting and insight generation workflows.
• Build agent-based solutions that combine structured and unstructured data.
• Implement Retrieval-Augmented Generation (RAG) architectures.
• Design guardrails and evaluation frameworks to ensure accuracy and reliability.
• Support deployment of production-ready AI solutions.

Technologies

• Large Language Models (LLMs)
• Prompt Engineering
• LangChain
• LangGraph
• RAG Frameworks
• Agentic AI Solutions

Automated Reporting & Insights

• Build automated reporting solutions that combine metrics, analysis, and narrative insights.
• Develop scheduled reporting frameworks for recurring business needs.
• Improve speed, accuracy, and consistency of reporting processes.
• Enable leadership teams t…