Senior Platform Engineer, AI Evaluation Jobs

Senior Platform Engineer, AI Evaluation

Khan Academy

Location:
United States, Mountain View

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

137871.00 - 172339.00 USD / Year

Save Job

Apply Position

Job Description:

We’re looking for an AI Platform Engineer to evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences at Khan Academy. This engineer will have worked with enough eval systems to quickly make sense of Khan's internal eval framework and recognize opportunities for improvement. This is largely a software development role, but domain experience with AI eval is essential for appreciating the hill-climbing and data science workflows we need to support.

Job Responsibility:

Evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences
Work closely with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments
Gather internal requirements, getting buy-in for changes, and then developing documentation and training materials

Requirements:

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems
Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
Familiarity with the architecture of large language models and their industry-standard APIs

Nice to have:

Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human-in-the-loop concerns such as rubric development and inter-rater agreement
Exposure to MLOps practices such as model registry, feature store, or continuous evaluation
Background in education technology or other human-centered AI applications

What we offer:

Competitive salaries
Ample paid time off as needed
8 pre-scheduled Wellness Days in 2026
Remote-first culture
Generous parental leave
401(k) + 4% matching
Comprehensive insurance, including medical, dental, vision, and life

Additional Information:

Job Posted:
December 09, 2025

Employment Type:

Fulltime

Work Type:

Remote work

View All Jobs In This Company

Job Link Share:

Welcome to CrawlJobs.com –
Your Global Job Discovery Platform

At CrawlJobs.com, we simplify finding your next career opportunity by bringing job listings directly to you from all corners of the web. Using cutting-edge AI and web-crawling technologies, we gather and curate job offers from various sources across the globe, ensuring you have access to the most up-to-date job listings in one place.