Senior Data Engineer Consultant (AI-Forward) — Pharmacy Data, Databricks + Kafka

Who We Are

Proactive Logic Consulting INC is a boutique technology consulting firm specializing in:

• Innovation, zero to one projects
• Assessments & Roadmaps
• Process Automation and AI
• Azure Cloud Migrations
• App Modernization (including Low-Code solutions)

We bring together top-tier independent consultants—experts in their craft who thrive on flexibility and autonomy. Our consultants have a customer-obsessed mindset, high EQ, and deep technical skills. We’re entrepreneurial at heart and fully support consultants who are building their own products on the side.

What We’re Looking For

We need a Senior Data Engineer Consultant (AI-Forward) to build and run the pipelines behind a modern pharmacy platform. You’ll bring together data from pharmacy systems, claims and e-prescribing sources and make it reliable for operations, clinical programs and reporting. You know pharmacy data well enough to spot when something looks wrong. This is a corp-to-corp, 1099 engagement for an independent consultant who works autonomously and uses AI tools daily to multiply their impact.

Key Requirements

• 10+ Years in Data Engineering
• Building and operating production streaming and batch pipelines and lakehouse architectures.
• Strong data modeling and testing; you leave behind clean, well-tested, documented tables.
• Databricks & Spark (2+ years)
• Azure Databricks, PySpark, Delta Lake, declarative pipelines (Delta Live Tables / Lakeflow), Structured Streaming.
• Deployment with Databricks Asset Bundles; job orchestration, monitoring and alerting.
• Kafka & Change Data Capture
• Production Kafka experience; CDC ingestion from operational databases.
• Idempotent, replayable pipelines with clear failure handling.
• SQL & Python
• Strong SQL, including SQL Server.
• Python packaging and testing (pyproject, wheels, pytest).
• Pharmacy Data (Required, 2+ years)
• NCPDP claims, NDCs and drug reference data, prescriptions and fills, prescriber identifiers (NPI, DEA).
• Good PHI habits: minimum-necessary access, no PHI in logs or test data, HIPAA-aligned practices.
• AI-Forward Data Engineering
• Daily practitioner with AI coding tools and agents: Claude Code, OpenAI Codex CLI, GitHub Copilot or Cursor.
• Prompt and instruction design: turn data requirements into structured prompts, tools and context.
• Agentic workflows for data operations: pipeline triage, schema-change diffing, test generation and code review with a human in the loop.
• Safety and compliance: PHI redaction, synthetic test data, secrets in Key Vault, nothing sensitive in prompts or logs.
• Consulting & High EQ
• Excellent client-facing communication; comfortable with engineers, analysts and pharmacy operations staff alike.
• Clear trade-off framing and requirements discovery.
• Entrepreneurial Mindset
• Think like an owner; deliver measurable value. We welcome consultants building their own ventures.
• Business Structure
• Must have an active LLC or S-Corp; 1099 only (Corp-to-Corp). 2+ years running your own consulting company strongly preferred.

What You’ll Do

• Build Pipelines: Streaming and batch pipelines on Azure Databricks with PySpark, Delta Lake and declarative pipelines.
• Ingest Change Data: Bring operational database changes in through CDC and Kafka.
• Model Pharmacy Data: Turn prescription, fill, claim, drug and prescriber data into clean, well-tested tables.
• Close the Loop: Deliver curated data back to operational systems and APIs.
• Ship Safely: Deploy with Databricks Asset Bundles and GitLab CI, with monitoring, alerting and data quality checks.
• Use AI Well: Apply AI tools to speed up delivery, documentation and review without letting PHI leave controlled boundaries.

Preferred Tech & Tooling

• Azure Databricks, PySpark, Delta Lake, Lakeflow / Delta Live Tables, Unity Catalog
• Kafka, CDC (e.g. Debezium), SQL Server
• Python (pyproject, wheels, pytest), GitLab CI, Databricks Asset Bundles
• Agentic IDEs: Claude Code, OpenAI Codex CLI, Copilot/Cursor

Nice to Have

• Pharmacy management system data, or e-prescribing data (NCPDP SCRIPT, medication history).
• Long-term-care or specialty pharmacy background.
• Databricks certification.
• Former pharmacy technician or pharmacy-system analyst.

Why Join Proactive Logic Consulting?

• Flexible Engagements: Shape your schedule a…