Senior Consultant II – AI/ML Engineer - National General

National General is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers’ evolving needs. We offer home, auto and accident and health insurance, as well as other specialty niche insurance products, through a large network of independent insurance agents, as well as directly to consumers.

Job Description Company: National General

• National General Insurance is an Allstate company, meaning it operates under the Allstate umbrella while maintaining its own brand identity and specialized insurance products
• This relationship allows National General to leverage Allstate’s resources, technology, and financial stability while continuing to serve its customers with tailored insurance solutions.
• Employee benefits same as that of Allstate US Corporation employees

About The Role This role supports high-impact AI and data automation initiatives focused on improving data ingestion, transformation, and mapping processes across the organization. This position is responsible for advancing AI-assisted data mapping capabilities to reduce manual effort, enhance accuracy, and deliver scalable, maintainable solutions for development teams.

Initial responsibilities include building and optimizing intelligent import tools that leverage AI/ML techniques to parse, normalize, and validate structured and unstructured data from various sources. Core deliverables involve developing mapping logic, identifying data discrepancies, creating standardized ingestion templates, and producing load-ready outputs for downstream systems. The role also contributes to analytics and dashboarding efforts while supporting source-specific data rules and configurations.

Over time, the scope will expand to broader AI, data-layer, and automation initiatives aligned with evolving business priorities, with a strong emphasis on delivering solutions that improve operational efficiency and overall data quality in high-visibility workflows.

Key objectives: Reduce manual mapping effort, improve data accuracy and processing efficiency, deliver scalable, maintainable AI-driven pipelines

What You’ll Do

• Design and build AI-powered data pipelines to process and transform structured and unstructured data (PDF, Excel, CSV, etc.)
• Develop and optimize LLM-based solutions (prompt engineering, mapping, extraction, and transformation)
• Translate complex data and technical outputs into actionable insights for business stakeholders
• Partner with cross-functional teams (product, engineering, business SMEs) to define and deliver solutions
• Validate and improve solution accuracy through testing, iteration, and stakeholder feedback
• Support integration of ML/AI solutions into enterprise systems and workflows (APIs, databases, cloud platforms)
• Document solutions to ensure scalability, maintainability, and long-term adoption

How your day to day looks:

• Analyze and process carrier and CMS datasets
• Build AI/ML mapping logic using LLMs and prompts
• Develop data pipelines (Python/C# data processing logic, SQL, APIs, Azure)
• Validate outputs against expected results with business stakeholders
• Troubleshoot accuracy, performance, and edge cases
• Document pipelines and ensure production-readiness
• Support integration into enterprise systems (C#, APIs, Azure)

Work Mix (exact mix will shift by project phase)

• 40% ML/prompt and mapping logic
• 35% data engineering / pipeline implementation
• 25% stakeholder collaboration and validation

Required Qualifications

• 3+ years of related experience
• Strong proficiency in Python and SQL for data analysis and engineering
• Experience working with Generative AI / LLMs (prompt engineering, APIs, or similar)
• Hands-on experience with data transformation, file parsing, and pipeline development
• Knowledge of DevOps practices (CI/CD pipelines, version control, cloud deployment)
• Familiarity with cloud platforms (preferably Azure)
• Ability to communicate effectively with both technical and non-technical stakeholders
• Proven ability to work in ambiguous, fast-evolving environments

Preferred Qualifications

• Experience with C# / .NET integration or enterprise application development
• Familiarity with LLM frameworks (LangChain, etc.) or document processing tools
• Knowledge of API design, JSON processing, and system integration patterns
• Background in insurance, healthcare, or ot…