Lead AI Engineer

Company Description

Experian is a global data and technology company, powering opportunities for people and businesses around the world. We help to redefine lending practices, uncover and prevent fraud, simplify healthcare, create marketing solutions, and gain deeper insights into the automotive market, all using our unique combination of data, analytics and software. We also assist millions of people to realize their financial goals and help them save time and money.

We operate across a range of markets, from financial services to healthcare, automotive, agribusiness, insurance, and many more industry segments.

We invest in people and new advanced technologies to unlock the power of data. As a FTSE 100 Index company listed on the London Stock Exchange (EXPN), we have a team of 22,500 people across 32 countries. Our corporate headquarters are in Dublin, Ireland. Learn more at experianplc.com.

Job Description

We are looking for an AI-Native Developer with deep expertise in building agentic systems, LLM-powered workflows, and autonomous data pipelines on AWS.

This role goes beyond traditional data engineering — you will design and build AI-driven ingestion and transformation systems, leveraging LLMs, code-generation agents, and orchestration frameworks to accelerate development, improve reliability, and reduce manual effort.

You will work extensively with tools like Claude Code, Cursor, and AWS AgentCore, and will be responsible for building production-grade AI agents that can interpret specifications, generate pipelines, validate outputs, and continuously improve through feedback loops.

Agentic System Design & Development

• You will be #LI-hybrid based in Hyderabad and reporting to Director Engineering
• Design and implement AI agents for data engineering workflows (spec parsing, code generation, validation, reconciliation).
• Build multi-step agent orchestration pipelines using LLMs and tool integrations.
• Develop autonomous systems capable of reverse engineering legacy pipelines and generating cloud-native equivalents.

AI-Augmented Engineering

• Leverage Claude Code, Cursor, and similar tools to accelerate development cycles.
• Build feedback loops for self-healing pipelines and auto-correction of data issues.
• Integrate LLMs for schema inference, transformation logic generation, and anomaly detection.

AWS-Native Development

• Build scalable pipelines using S3, Glue, Lambda, DynamoDB, EventBridge, and streaming services (MSK/SQS).
• Work with AWS AgentCore (or equivalent frameworks) to deploy and manage agentic workloads.
• Design systems that combine event-driven architectures with AI decision layers.

Intelligent Ingestion & Transformation

• Develop AI-driven ingestion frameworks supporting batch and near real-time processing.
• Implement automated data validation, reconciliation, and lineage tracking using AI agents.
• Ensure parity with legacy systems using AI-assisted validation techniques.

CI/CD & AI Integration

• Embed agentic workflows into CI/CD pipelines (Jenkins/Harness).
• Automate testing using LLM-generated test cases and validation strategies.
• Integrate security scanning (SonarQube, Veracode) with AI-assisted remediation suggestions.

Data Governance & Control (AI-Enhanced)

• Implement AI-powered data quality, lineage, and compliance monitoring.
• Build intelligent dashboards with automated evidence generation.
• Enable proactive detection of data issues using anomaly detection agents.

Collaboration & Innovation

• Partner with Product, Architecture, and AI teams to define agentic engineering standards.
• Work closely with legacy system experts to translate logic into AI-understandable specifications.
• Contribute to building an AI-first engineering culture within the organization.

Qualifications

Core AI & Agentic Development

• 5+ years of overall experience
• Experience building AI agents / agentic workflows (multi-step reasoning, tool use, orchestration).
• Hands-on experience with Claude Code, Cursor, or similar AI coding environments.
• Experience with AWS AgentCore or equivalent agent orchestration frameworks.
• Understanding of prompt engineering, evaluation, and LLM optimization techniques.

Programming & Data Engineering

• Python (advanced), PySpark
• Strong experience in data pipeline design and distributed processing

AWS & Cloud

• S3, Glue, Lambda, DynamoDB, EventBridge, MSK/SQS
• Experience building event-driven and serverless architectures

DevOps & Automation

• Jenkins/Harness, Bitbucket
• SonarQube, Veracode
• Experience integrating AI into CI/CD workflows

Data Governance & Security

• Data quality frameworks, lineage tracking
• IAM, encryption, secure configuration

Core Outcomes & Success Metrics

• Production-grade AI-driven ingestion and transformation systems deployed on AWS
• Measurable improvement in developer productivity through AI-assisted engineering
• High accuracy in AI-generated pipelines with validated parity to legacy systems
• Automated data quality and governance with minimal manual intervention
• Robust agentic workflows embedded in CI/CD pipelines

Stakeholder Interactions

• Engineering Leadership: Technical direction and innovation
• Product Owners & BAs: Requirements and prioritization
• AI/ML Engineers: Agent design and optimization
• Data Stewards: Governance validation
• DevSecOps: Secure and scalable deployment

Additional Information

Our uniqueness is that we celebrate yours. Experian's culture and people are important differentiators. We take our people agenda very seriously and focus on what matters; DEI, work/life balance, development, authenticity, collaboration, wellness, reward & recognition, volunteering... the list goes on. Experian's people first approach is award-winning; World's Best Workplaces™ 2024 (Fortune Global Top 25), Great Place To Work™ in 24 countries, and Glassdoor Best Places to Work 2024 to name a few. Check out Experian Life on social or our Careers Site and Glassdoor to understand why.

Experian is proud to be an Equal Opportunity and Affirmative Action employer. Innovation is a critical part of Experian's DNA and practices, and our diverse workforce drives our success. Everyone can succeed at Experian and bring their whole self to work, irrespective of their gender, ethnicity, religion, color, sexuality, physical ability or age. If you have a disability or special need that requires accommodation, please let us know at the earliest opportunity.

Benefits

Experian care for employee's work life balance, health, safety and wellbeing. In support of this endeavor, we offer best-in-class family well-being benefits, enhanced medical benefits and paid time off.

This is a hybrid remote/in-office role.

Experian Careers - Creating a better tomorrow together

Find out what its like to work for Experian by clicking here