Senior Associate - Agentic AI _ Web Data Platform

Job Description: Senior Associate Agentic AI / Web Data Platform

Role summary: Own and extend a production-bound POC that combines resilient web-crawling engineering with an AWS Bedrock-based LLM verification agent, in a healthcare-data-compliance-sensitive context.

Must-have (core, non-negotiable):

- Senior Python (5+ yrs) — idiomatic, typed, pydantic-based data modeling; comfortable owning a multi-module package(adapters, caching layers, CLI) not just scripts.

- Web scraping / crawling engineering — httpx/requests, BeautifulSoup, sitemap & robots.txt parsing, TLS-fingerprint evasion (curl_cffi-style impersonation), designing fallback/union strategies across multiple discovery methods, handling rate-limiting and backoff against hostile/throttling targets.

- Browser automation — Playwright (or Selenium/Puppeteer equivalent): scripted login flows, cookie/consent gates, JS-rendered SPA waits, pagination/infinite scroll, config-driven step sequences rather than hardcoded per-site logic.

- LLM/agentic engineering with AWS Bedrock — LangChain/Langraph (langchain-aws/ChatBedrockConverse), Claude model usage including extended-thinking/reasoning content blocks, boto3 credential chains (assume-role/STS), and — critically — designing deterministic-first, LLM-second hybrid pipelines (code decides what it can, LLM only judges what it can’t) rather than routing everything through the model.

• AWS core — Bedrock, boto3, IAM roles/STS, Athena/Glue querying.

• PDF/document processing — PyMuPDF or similar, content-anchored text extraction (not blind truncation) for long documents.

Nice-to-have / domain

• Healthcare payer/PBM domain exposure — prior authorization, formularies, medical vs. pharmacy benefit, line-of-business (Medicaid/Medicare/Commercial/Exchange)