This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
As an Evaluation Frontend Software Engineer, you will play a key role in helping us make modelling decisions based on experimental outcomes for our large language models (LLMs). Your primary focus will be on building tools to enable easy visualization and analysis of model evaluations which are stored in a central database. You will work closely with cross-functional teams, including researchers and engineers, to surface the insights that are necessary for the future development of our models. This role combines expertise in statistics, data science, frontend development and data visualization.
Job Responsibility:
Design tools and visualizations that enable researchers and engineers to compare and analyse hundreds of model evaluations
Develop an understanding of the relative merits and limitations of each of our model evaluations, as well as suggest new facets of model evaluation
Requirements:
Extremely strong software engineering skills
Strong statistical skills and experience evaluating scientific experiments related to data collection and model performance
Prior experience building front-end visualization systems and dashboards
Familiarity with ML systems evaluations
Proficiency in programming languages such as Python and ML frameworks (e.g., PyTorch, TensorFlow, JAX)
Excellent communication skills to collaborate effectively with cross-functional teams and present findings
One or more papers at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
What we offer:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend