This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
We’re looking for an AI Platform Engineer to evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences at Khan Academy. This engineer will have worked with enough eval systems to quickly make sense of Khan's internal eval framework and recognize opportunities for improvement. This is largely a software development role, but domain experience with AI eval is essential for appreciating the hill-climbing and data science workflows we need to support.
Job Responsibility:
Evolve and extend our internal evaluation framework for assessing the quality of our AI-driven experiences
Work closely with ML data engineers and platform developers to help internal teams adopt an eval-driven development process incorporating offline benchmark tests and online experiments
Gather internal requirements, getting buy-in for changes, and then developing documentation and training materials
Requirements:
Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
5 years of Software Engineering experience with 2+ of those years working on the evaluation of generative AI systems
Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect)
Familiarity with the architecture of large language models and their industry-standard APIs
Nice to have:
Experience with labeling platforms (e.g., Label Studio, Scale AI, Toloka) and human-in-the-loop concerns such as rubric development and inter-rater agreement
Exposure to MLOps practices such as model registry, feature store, or continuous evaluation
Background in education technology or other human-centered AI applications
What we offer:
Competitive salaries
Ample paid time off as needed
8 pre-scheduled Wellness Days in 2026
Remote-first culture
Generous parental leave
401(k) + 4% matching
Comprehensive insurance, including medical, dental, vision, and life