Lead AI Engineer

Company Description Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money.

We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments.

We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more at experianplc.com.

Job Description We are looking for an AI-Native Developer with deep expertise in building agentic systems, LLM-powered workflows, and autonomous data pipelines on AWS.

This role goes beyond traditional data engineering — you will design and build AI-driven ingestion and transformation systems , leveraging LLMs, code-generation agents, and orchestration frameworks to accelerate development, improve reliability, and reduce manual effort.

You will work extensively with tools like Claude Code, Cursor, and AWS AgentCore , and will be responsible for building production-grade AI agents that can interpret specifications, generate pipelines, validate outputs, and continuously improve through feedback loops.

Agentic System Design & Development

• You will be based in Hyderabad and reporting to Director Engineering
• Design and implement AI agents for data engineering workflows (spec parsing, code generation, validation, reconciliation).
• Build multi-step agent orchestration pipelines using LLMs and tool integrations.
• Develop autonomous systems capable of reverse engineering legacy pipelines and generating cloud-native equivalents.

AI-Augmented Engineering

• Leverage Claude Code, Cursor, and similar tools to accelerate development cycles.
• Build feedback loops for self-healing pipelines and auto-correction of data issues.
• Integrate LLMs for schema inference, transformation logic generation, and anomaly detection.

AWS-Native Development

• Build scalable pipelines using S3, Glue, Lambda, DynamoDB, EventBridge, and streaming services (MSK/SQS).
• Work with AWS AgentCore (or equivalent frameworks) to deploy and manage agentic workloads.
• Design systems that combine event-driven architectures with AI decision layers.

Intelligent Ingestion & Transformation

• Develop AI-driven ingestion frameworks supporting batch and near real-time processing.
• Implement automated data validation, reconciliation, and lineage tracking using AI agents.
• Ensure parity with legacy systems using AI-assisted validation techniques.

CI/CD & AI Integration

• Embed agentic workflows into CI/CD pipelines (Jenkins/Harness).
• Automate testing using LLM-generated test cases and validation strategies.
• Integrate security scanning (SonarQube, Veracode) with AI-assisted remediation suggestions.

Data Governance & Control (AI-Enhanced)

• Implement AI-powered data quality, lineage, and compliance monitoring.
• Build intelligent dashboards with automated evidence generation.
• Enable proactive detection of data issues using anomaly detection agents.

Collaboration & Innovation

• Partner with Product, Architecture, and AI teams to define agentic engineering standards.
• Work closely with legacy system experts to translate logic into AI-understandable specifications.
• Contribute to building an AI-first engineering culture within the organization.

Qualifications Core AI & Agentic Development

• 5+ years of overall experience
• Experience building AI agents / agentic workflows (multi-step reasoning, tool use, orchestration).
• Hands-on experience with Claude Code, Cursor, or similar AI coding environments.
• Experience with AWS AgentCore or equivalent agent orchestration frameworks.
• Understanding of prompt engineering, evaluation, and LLM optimization techniques.

Programming & Data Engineering

• Python (advanced), PySpark
• Strong experience in data pipeline design and distributed processing

AWS & Cloud

• S3, Glue, Lambda, DynamoDB, EventBridg…