CrawlJobs Logo

Staff Machine Learning Research Scientist, LLM Evals

scale.com Logo

Scale

Location Icon

Location:
United States , San Francisco

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

280000.00 - 380000.00 USD / Year

Job Description:

As a Staff Machine Learning Research Scientist on the LLM Evals team, you will lead the development of novel evaluation methodologies, metrics, and benchmarks to measure the capabilities and limitations of frontier LLMs. You will help define what "good" looks like in generative AI, driving research that informs both our internal roadmap and the broader research community. This role is critical for designing and executing a roadmap that defines best practices in data driven AI development and will accelerate the next generation of generative AI models in partnership with top foundational model labs.

Job Responsibility:

  • Drive research on the effectiveness and limitations of existing LLM evaluation techniques
  • Design and develop novel evaluation benchmarks for large language models, covering areas such as instruction following, factuality, robustness, and fairness
  • Communicate, collaborate, and build relationships with clients and peer teams to facilitate cross-functional projects
  • Collaborate with internal teams and external partners to refine metrics and create standardized evaluation protocols
  • Implement scalable and reproducible evaluation pipelines using modern ML frameworks
  • Publish research findings in top-tier AI conferences and contribute to open-source benchmarking initiatives
  • Mentor and guide research scientists and engineers, providing technical leadership across cross-functional projects
  • Stay deeply engaged with the ML research community, tracking emerging work and contributing to the advancement of LLM evaluation science
  • Thrive in a high-energy, fast-paced startup environment and are ready to dedicate the time and effort needed to drive impactful results.

Requirements:

  • 5+ years of hands-on experience in large language model, NLP, and Transformer modeling, in the setting of both research and engineering development
  • Experience and track of recording in landing major research impacts in a fast-paced environment
  • Experience tech leading a team of research scientists and research engineers
  • Excellent written and verbal communication skills
  • Published research in areas of machine learning at major conferences (NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, etc.) and/or journals
  • Previous experience in a customer facing role.
What we offer:
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • equity based compensation
  • commuter stipend (may be eligible).

Additional Information:

Job Posted:
February 20, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Staff Machine Learning Research Scientist, LLM Evals

Staff / Principal Research Scientist

Inworld is a product-oriented research lab of top AI researchers and engineers, ...
Location
Location
United Kingdom
Salary
Salary:
140000.00 - 200000.00 GBP / Year
inworld.ai Logo
Inworld AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Foundation models: training, new architectures, RL, reward modeling, scaling
  • Evaluation: benchmarks, eval loops, quality measurement, LLM-as-judge, failure analysis
  • Frontier topics: multimodal models, agents, tool use, test-time compute, world models
  • Published research at ICML, ICLR, NeurIPS, EMNLP, ACL, or AAAI
  • PhD in ML/NLP — or equivalent practical experience you can point to
  • Public work: non-trivial AI side projects, interdisciplinary experiments, open-source contributions
  • Full-stack research ownership: you frame the question, run the experiments, write the paper, ship the result
What we offer
What we offer
  • equity and benefits
  • Fulltime
Read More
Arrow Right
New

Curriculum Advisor (First Nations)

This is an identified role for Aboriginal and/or Torres Strait Islander people. ...
Location
Location
Australia , Albury-Wodonga; Bathurst; Dubbo; Orange; Port Macquarie; Wagga Wagga
Salary
Salary:
101421.00 - 110819.00 AUD / Year
csu.edu.au Logo
Charles Sturt University
Expiration Date
April 22, 2026
Flip Icon
Requirements
Requirements
  • Strong experience working respectfully with First Nations communities, Elders and knowledge holders
  • A background in education, curriculum, learning design or professional development
  • The ability to design and deliver culturally safe learning programs and resources
  • Well-developed communication and relationship-building skills
  • Confidence working collaboratively with diverse stakeholders across a large organisation
Job Responsibility
Job Responsibility
  • Help make learning and teaching across the university culturally respectful, inclusive and high quality
  • Be responsible for running and improving the First Nations Cultural Awareness Journey, a key program for staff, students and external partners
  • Make sure the program is delivered well and continues to grow and improve
  • Work closely with First Nations Elders, communities, academics and education specialists
  • Help embed First Nations knowledges, histories and perspectives into courses and professional learning
  • Support culturally safe learning spaces and better outcomes for students across online and face‑to‑face study
What we offer
What we offer
  • 17% superannuation
  • Supportive, collaborative and values‑based environment
  • Fulltime
Read More
Arrow Right
New

Senior Mechanical Engineer

In Command, Control, Communications, Computers, Intelligence, Surveillance, and ...
Location
Location
United States , Dallas
Salary
Salary:
88189.00 - 149922.00 USD / Year
baesystems.com Logo
Baesystems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in mechanical engineering
  • 4+ years work experience (or 2+ years with MS ME) designing, testing, and troubleshooting high-reliability mechanical systems
  • Demonstrated understanding of mechanical principles, test methodologies, and the ability to work with advanced test equipment and cutting-edge technology
  • Strong “hands-on” laboratory experience with instrumentation, equipment, and processes
  • Eligible to obtain a DOD Security Clearance
Job Responsibility
Job Responsibility
  • Support to the design, development, integration/test, and production of dynamic electronic military systems
  • Work closely with hardware integrated into a variety of complex Signal Intelligence systems
  • Execute both individually and in small, yet diverse, team environments through production and sustainment support, and may include other phases of the product life cycle including concept development, detail design, qualification, and manufacturing transition
  • Responsible for solving production issues while developing robust hardware solutions to meet demanding customer requirements
  • Leverage mechanical engineering expertise to perform detailed design and analyses
  • Day to day, activities will include root cause and corrective action activities in a production environment
  • Close interactions with the manufacturing team and suppliers, including generating 3D solid models, drawings, and documentation utilizing GD&T principles
  • Performing tolerance analyses to assure fit & function
  • Performing thermal, structural/stress, and vibration analyses
  • Assessing designs against environmental requirements
What we offer
What we offer
  • Health, dental, and vision insurance
  • Health savings accounts
  • A 401(k) savings plan
  • Disability coverage
  • Life and accident insurance
  • Employee assistance program
  • A legal plan
  • Other perks including discounts on things like home, auto, and pet insurance
  • Paid time off
  • Paid holidays
  • Fulltime
Read More
Arrow Right
New

Director, Accounting and SEC Reporting

This role is essential for ensuring accurate and timely financial reporting in c...
Location
Location
United States , New York
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Accounting/Finance degree
  • CPA
  • 6+ years Accounting/SEC Reporting experience from the Biotech/Pharma Industry
  • Big4/Public Accounting foundation
Job Responsibility
Job Responsibility
  • Oversee and manage monthly, quarterly, and annual financial close processes, including the preparation and review of journal entries, balance sheet reconciliations, and SEC filings such as Forms 10-Q and 10-K
  • Ensure adherence to Sarbanes-Oxley internal control requirements, including consistent testing and documentation of processes
  • Lead and mentor a team, including direct supervision of the Accounts Payable Manager and Senior Accountant
  • Accurately recognize and account for expenses, including R&D and clinical trial accruals, as well as stock-based compensation
  • Review and approve purchase orders, invoices, and travel and expense reports to ensure compliance with company policies
  • Manage accounts payable operations, cash management, and treasury functions, including accounting for investments in U.S. Treasury securities
  • Prepare and file regulatory documents such as Forms 8-K, 14A, and S-8, as well as local tax filings like New York Sales and Use Tax and New York City Commercial Rent Tax
  • Collaborate with payroll, equity administration, and human resources to maintain accurate records and reporting
  • Coordinate with internal teams to analyze contract-related costs and prepare ad hoc financial analyses as needed
  • Drive continuous improvement in accounting processes by leveraging technology solutions and identifying efficiencies
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Read More
Arrow Right
New

Director of Finance

Our client is a growth-focused private equity firm investing in middle-market co...
Location
Location
United States , San Francisco
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-15+ years of accounting and finance experience in investment management
  • CPA, MBA, or similar qualification preferred
  • Strong knowledge of U.S. GAAP and financial reporting
  • Experience with FP&A, investor reporting, and fund operations
  • Advanced Excel and strong analytical skills
  • Ability to operate independently in a fast-paced environment
Job Responsibility
Job Responsibility
  • Oversee treasury operations including capital calls, distributions, and cash management
  • Manage cash flow forecasting and the annual budgeting process
  • Lead monthly closes and financial reporting for funds and GP entities
  • Oversee outsourced accounting operations and expense approvals
  • Manage fund administrator relationships and investment cash flow tracking
  • Prepare quarterly financial reporting and investment performance summaries
  • Lead annual audit and tax processes
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right
New

Executive Assistant

We are currently seeking an experienced and highly organised Executive Assistant...
Location
Location
United Kingdom , Bristol
Salary
Salary:
17.12 GBP / Hour
https://www.office-angels.com Logo
Office Angels
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous experience in an administrative or Executive Assistant role
  • Diary management and travel coordination experience
  • Handling confidential information and understanding GDPR compliance
  • Strong data entry and record-keeping skills
  • Experience liaising with stakeholders or customers
  • Ability to manage workload independently and work proactively
Job Responsibility
Job Responsibility
  • Manage complex scheduling and diary coordination for senior officers
  • Draft, review and send communications on behalf of officers
  • Organise and support meetings, including preparation of documents and logistics
  • Act as a first point of contact for calls and enquiries, relaying key information
  • Manage and prioritise inboxes, responding where appropriate
  • Coordinate travel arrangements and itineraries
  • Maintain accurate records, files and documentation
  • Ensure the Case Management System is up to date and accurate
  • Liaise with internal teams regarding progress on legal cases, campaigns, ballots and activities
  • Support the efficient running of office processes and procedures
What we offer
What we offer
  • Hybrid working model (3 days in office, 2 from home)
  • Central Bristol location with excellent transport links
  • Opportunity to gain experience in a respected organisation
  • Supportive and collaborative team environment
  • Fulltime
Read More
Arrow Right
New

Country Site Merchandising Specialist

Do you get excited when you see data patterns that unlock better customer experi...
Location
Location
Australia , Tempe
Salary
Salary:
Not provided
https://www.ikea.com Logo
IKEA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of experience working with online commerce, sales, marketing, or equivalent, with demonstrated experience in omnichannel customer experience journeys that connect digital and physical touchpoints
  • Strong documented co-creation and collaboration skills, with proven ability to work effectively within teams and with cross-functional teams and stakeholders to deliver shared outcomes
  • Proven analytical skills to monitor performance metrics, interpret data patterns, and identify actionable areas for improvement that drive commercial results
  • Demonstrated ability to act with strong attention to detail and execute tasks with precision, ensuring accuracy and consistency across all digital merchandising activities
  • Proven ability to thrive and perform in complex and dynamic environments, adapting quickly to changing priorities and business needs while maintaining quality standards
  • Strong ability to contribute to driving change in a positive way, influencing and inspiring others to reach business goals through collaborative problem-solving
Job Responsibility
Job Responsibility
  • Ensures timely implementation of range and merchandising plans, including setting up product launches, campaigns, and seasonal activities based on the commercial calendar
  • Regularly updates and optimizes onsite search terms and filters to improve show ability, buy ability, and relevance, ensuring alignment with customer needs and market trends
  • Builds and implements A/B tests for digital customer meeting points, documenting setup, tracking results, and providing data-driven insights for continuous improvement
  • Pulls reports on KPIs such as conversion rates, search behaviour, and add-to-basket performance, identifying patterns and sharing actionable insights with the team
  • Executes updates for product descriptions, images, and localisation with the content team to ensure consistency and relevance across digital touchpoints, meeting local market needs
  • Fulltime
Read More
Arrow Right
New

Compensation & Benefits Specialist

We are looking for a dedicated Compensation & Benefits Specialist to join our te...
Location
Location
United States , Shoreline
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Previous Union/CBA
  • Experience with Complex Excel functions
  • Ideally worked with PeopleSoft previously
  • Proven experience in Human Resources administration with a focus on compensation and benefits
  • Strong proficiency in HRIS systems and payroll management tools
  • Familiarity with employee relations and onboarding processes
  • Knowledge of HIPAA regulations and their application within HR practices
  • Attention to detail and ability to analyze and reconcile complex datasets
  • Excellent communication skills to collaborate effectively across departments
  • Solid understanding of payroll systems and HR documentation procedures
Job Responsibility
Job Responsibility
  • Verify employee compensation details, including pay rates, classifications, step placements, stipends, and supplemental pay, to ensure compliance with approved HR records and payroll configurations
  • Collaborate with HR and Payroll teams to identify and resolve discrepancies in employee data related to hires, terminations, leaves of absence, organizational changes, and role adjustments
  • Conduct analyses for retroactive compensation and corrections, coordinating validation of effective dates, approvals, and necessary adjustments between HR and Payroll
  • Prepare and review reconciliation schedules to ensure alignment between HR source data and payroll processing outputs
  • Maintain and enhance tracking systems for HR documentation and issue logs, ensuring operational transparency and timely resolution of payroll-impacting matters
  • Provide support for accurate documentation of payroll-related adjustments and employee status changes
  • Assist in improving accountability and visibility within HR processes to streamline payroll operations
  • Ensure compliance with regulations and confidentiality standards, including those governed by HIPAA
  • Support the onboarding process by validating compensation details and addressing employee inquiries related to benefits and payroll
  • Collaborate with the HRIS team to optimize data integrity and reporting functionality
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right