Software Engineer II - AI

Job Description Job Information

Software Engineer II - AI

Company Information

• Company: McGraw Hill
• Team: AI Platform
• Job ID: 50686

Location & Employment Type

• Location Options: Manhattan, NY
• Remote
• Work Authorization: U.S. (Remote position open to applicants authorized to work for any employer within the United States)
• Employment Type: Full-time

Job Overview Join the AI Platform team at McGraw Hill to build intelligent learning experiences used by millions. This is a software engineering role focused on building the services, APIs, and production infrastructure that turn AI model capabilities (RAG pipelines, LLM orchestration, retrieval, and routing) into shipped features. This role is distinct from data science or MLOps; it focuses on reliability, observability, and scalability rather than model training.

Machine Learning & Artificial Intelligence

Discover more

Software Engineering

Software Development

VPN & Remote Access

Key Responsibilities

• Build services and APIs that deliver AI to users, including RAG pipelines and LLM orchestration layers.
• Integrate with model providers (e.g., Azure OpenAI) and build retrieval and routing logic.
• Develop backend services primarily using Python (FastAPI, async/await, Pydantic) and potentially Go.
• Work with PostgreSQL, async task workers, and service integrations.
• Write tested, readable, and maintainable code; participate in code reviews focusing on quality and reliability.
• Collaborate with data scientists, product managers, designers, and engineers to ship AI-powered features.
• Learn and apply best practices for prompt design, retrieval quality, latency/cost tradeoffs, and graceful failure modes in production.

Required Qualifications

• Experience: 2–5 years of professional software development experience. Proven track record of shipping features to end-users and debugging production issues.
• Backend Fundamentals: Comfort in modern backend languages (Python preferred; Go, Java, Kotlin, or Node acceptable). Experience building APIs and working with relational databases (PostgreSQL, MySQL, or similar).
• AI Curiosity: Some practical experience as a software engineer with AI, such as building applications that call LLMs, experimenting with frameworks like LangChain or LlamaIndex, or forming opinions on prompt design and retrieval.
• Engineering Instincts: Strong testing practices, code readability, and the ability to explain technical decision-making.
• Problem Solving: Comfort with ambiguity, ability to break down medium-sized features into actionable tasks, and identify risks early.
• Communication: Ability to write clear tradeoff analysis, give specific code review feedback, and communicate effectively in public channels.
• Attitude: “Low-ego, high-ownership” disposition with a focus on team success over individual credit.

Nice to Have (Bonus)

• Production experience with AWS or Azure.
• Familiarity with frontend frameworks (React or Angular).
• Experience with vector databases, embeddings, or RAG architectures.
• Exposure to Go or willingness to learn it.
• Familiarity with Kubernetes, Terraform, or modern CI/CD (GitHub Actions).
• Background in edtech, accessibility (WCAG 2.2 AA), or content-rich products.
• Open-source contributions, technical writing, or teaching experience.

Compensation & Benefits

• Salary Range: $93,950 - $135,000 annually (base pay may vary based on knowledge, skills, experience, and location).
• Additional Compensation: Annual bonus plan may be provided.
• Benefits: Full range of medical and other benefits (details available via company links).