CB Databricks AI Engineer

Years of Experience: 8+ Years

Start date: 15th Oct

Duration : 6 Months

Work Timings : 1:30pm to 10:30 pm

Remote Opportunity and Contract Role

Please see the below spec for CB Databricks AI Engineer.

Experience building CI/CD pipelines a bonus, but don’t want to make the request unrealistic.

Role Overview

We are seeking an experienced Databricks AI Engineer to support and enhance a productionised AI information extraction solution. The individual will be responsible for the ongoing development, optimisation, evaluation and support of the solution, working directly with client stakeholders and operating with a high degree of autonomy.

Key Skills & Experience

Deep hands-on experience with Databricks, including Lakehouse architecture, Delta Lake, Unity Catalog, Workflows and production support.

Mandatory skillset : Python, SQL, Spark, Databrick, deltalake, unity catalog, AI

Strong Python, SQL and Spark development skills.

Experience building and supporting AI-driven information extraction and document processing solutions in production environments.

Expertise in AI evaluation frameworks, including:

Human-in-the-loop evaluation approaches

LLM-as-a-Judge methodologies

Gold dataset creation and benchmarking

Accuracy, precision, recall and business outcome measurement

Strong understanding of model selection and evaluation, including assessing multiple foundation models and designing architectures that avoid dependency on a specific LLM provider or model.

Experience implementing robust AI workflows that can adapt to changing model capabilities and market developments.

Strong troubleshooting, optimisation and root-cause analysis skills.

Personal Attributes

Exceptional communication and stakeholder management skills.

Highly self-directed and comfortable working independently within a client environment.

Able to translate business requirements into technical solutions and provide clear recommendations.

Takes ownership of delivery outcomes and proactively identifies opportunities for improvement.

Pay: ₹2.50 - ₹3.00 per month

Work Location: Remote