CrawlJobs Logo

Research Engineer, Frontier Evals & Environments - Finance

United States, San Francisco 205000.00 - 380000.00 USD / Year · Job Posted February 21, 2026
Apply Position
Job Link Share

Job Description

The Frontier Evals team builds north star model evaluations to drive progress towards safe AGI/ASI. This team builds ambitious evaluations to measure and steer our models, and creates self-improvement loops to steer our training, safety, and launch decisions. Some of the team's open-sourced evaluations include SWE-bench Verified, MLE-bench, PaperBench, and SWE-Lancer, and the team built and ran frontier evaluations for GPT4o, o1, o3, GPT 4.5, ChatGPT Agent, and GPT5. If you are interested in feeling firsthand the fast progress of our models, and steering them towards good, this is the team for you.

Job Responsibility

  • Identify important model capabilities, skills, and behaviors that are crucial to financial workflows, and design methods to quantify performance in these areas
  • Own and pursue a research agenda to identify an important model capability (especially as it relates to financial reasoning) and build evals to measure it
  • Continuously refine evaluations of frontier AI models to assess the extent of frontier capabilities

Requirements

  • Strong engineering and statistical analysis skills (with at least 2-3 years of full-time technical experience)
  • Passionate about evals for real world applications and knowledge work
  • Detail-oriented and thorough
  • Team player / willing to do a variety of tasks to move the team forward
  • Passionate and knowledgeable about AGI/ASI measurement
  • Able to operate effectively in a dynamic and extremely fast-paced research environment as well as scope and deliver projects end-to-end

Nice to have

  • An ability to work cross-functionally
  • Excellent communication skills

What we offer

  • Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
  • Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
  • 401(k) retirement plan with employer match
  • Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
  • Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
  • 13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
  • Mental health and wellness support
  • Employer-paid basic life and disability coverage
  • Annual learning and development stipend to fuel your professional growth
  • Daily meals in our offices, and meal delivery credits as eligible
  • Relocation support for eligible employees
  • Additional taxable fringe benefits, such as charitable donation matching and wellness stipends, may also be provided
  • Offers Equity

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Research Engineer, Frontier Evals & Environments - Finance

8 matching positions

AI Architect

We’re hiring an AI Architect to sit at the intersection of frontier AI research,...
Location
Location
United States , San Francisco; New York
Salary
Salary:
201600.00 - 241920.00 USD / Year
scale.com Logo
Scale
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Deep technical background in applied AI/ML: 5–10+ years in research, engineering, solutions engineering, or technical product roles working on LLMs or multimodal systems, ideally in high-stakes, customer-facing environments
  • Hands-on experience with model improvement workflows: demonstrated experience with post-training techniques, evaluation design, benchmarking, and model quality iteration
  • Ability to work on hard, ambiguous technical problems: proven track record of partnering directly with advanced customers or research teams to scope, reason through, and execute on deep technical challenges involving frontier models
  • Strong technical fluency: you can read papers, interrogate metrics, write or review complex Python/SQL for analysis, and reason about model-data trade-offs
  • Executive presence with world-class researchers and enterprise leaders
  • excellent writing and storytelling
  • Bias to action: you ship, learn, and iterate.
Job Responsibility
Job Responsibility
  • Translate research → product: work with client side researchers on post-training, evals, safety/alignment and build the primitives, data, and tooling they need
  • Partner deeply with core customers and frontier labs: work hands-on with leading AI teams and frontier research labs to tackle hard, open-ended technical problems related to frontier model improvement, performance, and deployment
  • Shape and propose model improvement work: translate customer and research objectives into clear, technically rigorous proposals—scoping post-training, evaluation, and safety work into well-defined statements of work and execution plans
  • Translate research into production impact: collaborate with customer-side researchers on post-training, evaluations, and alignment, and help design the data, primitives, and tooling required to improve frontier models in practice
  • Own the end-to-end lifecycle: lead discovery, write crisp PRDs and technical specs, prioritize trade-offs, run experiments, ship initial solutions, and scale successful pilots into durable, repeatable offerings
  • Lead complex, high-stakes engagements: independently run technical working sessions with senior customer stakeholders
  • define success metrics
  • surface risks early
  • and drive programs to measurable outcomes
  • Partner across Scale: collaborate closely with research (agents, browser/SWE agents), platform, operations, security, and finance to deliver reliable, production-grade results for demanding customers
What we offer
What we offer
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • commuter stipend
  • equity based compensation.
  • Fulltime
Read More
Arrow Right

Controller

The Controller will lead core accounting and financial activities across multipl...
Location
Location
United States , Spring Lake
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Accounting, Finance, or a related field required
  • 5-8 years of progressive accounting or financial management experience
  • Experience managing full-cycle accounting in a small to mid-sized business environment
  • Prior experience supervising accounting or administrative staff preferred
  • Multi-location or multi-entity accounting experience preferred
  • Proven success improving accounting systems, internal controls, and financial processes
  • Strong knowledge of GAAP and financial reporting
  • Ability to translate financial data into actionable business insights
  • Experience with accounting software such as QuickBooks, Sage, or similar platforms
  • Advanced Excel skills
Job Responsibility
Job Responsibility
  • Oversee accounting operations across multiple locations
  • Maintain the general ledger and ensure financial records are accurate and compliant with GAAP
  • Prepare monthly, quarterly, and annual financial statements
  • Review and perform bank and credit card reconciliations
  • Manage fixed asset tracking and capital expenditure reporting
  • Lead the year-end close process and coordinate with external accountants on tax preparation
  • Oversee accounts payable and accounts receivable activities
  • Ensure accurate and timely recording of all revenue streams across the business
  • Promote consistency in accounting practices, policies, and procedures across locations
  • Supervise the Finance Admin responsible for payroll processing, AP/AR and administrative support
What we offer
What we offer
  • Medical
  • Vision
  • Dental
  • Life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right

Software Development Engineer II

Amazon's Financial Technology team is seeking passionate, results-driven, and in...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
Amazon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of non-internship professional software development experience
  • 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
  • Experience programming with at least one software programming language
Job Responsibility
Job Responsibility
  • Design, develop, document, and support massively scalable, distributed real-time systems
  • Own all aspects of end-to-end solutions through full-stack software development
  • Leverage agile methodologies to iteratively build and deliver high-quality results
  • Work across Amazon engineering teams and business teams globally to plan, design, execute, and implement new platform across multiple geographies
  • Review requirements, design services, build new interfaces, integrate with existing architectures, develop and test code, and deliver implementations for Global Tax customers
  • Manage day-to-day activities, participate in designs, design reviews, and code reviews with engineering team
  • Build backend services and customer-facing APIs using AWS technologies
  • Design and code technical solutions using Angular JS, React, Node.js, JQuery, and SQL Server
  • Contribute to a suite of tools hosted on AWS infrastructure
  • Fulltime
Read More
Arrow Right

General Manager

Staycity is an urban aparthotel brand, blending the freedom of apartment living ...
Location
Location
France , Lyon
Salary
Salary:
Not provided
wilde.com Logo
Wilde Aparthotels
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven leadership experience and 3+ years in a similar level role, along with significant experience within the hotel or aparthotel sector
  • Full working proficiency in English
  • Demonstrated success in driving revenues through the strategic management of commercial initiatives
  • Humble and approachable, fostering a positive and inclusive work environment
  • Forward-thinking and innovative, always looking for opportunities to improve operations
  • Self-motivated and able to inspire and lead the team effectively
  • Energetic, flexible, and adaptable to changing circumstances
  • Possesses a keen eye for detail and is highly organized
  • Ability to adapt effectively and make decisions under pressure
  • Proficiency in engaging in strategic business discussions at a higher level
Job Responsibility
Job Responsibility
  • Support all HODs in all aspects of running the property, including front office, housekeeping, maintenance, sales and revenue
  • Ensure that all departments operate within the Staycity Group brand SOPs, monitoring and taking action to any feedback within a timely manner
  • Regularly evaluate individual and team performance against their goals to drive a high performance culture. Offer constructive feedback through on-the-job training, performance appraisals, one-on-one discussions, and team meetings
  • Produce forecasts for revenues and costs and monitor that these are maintained within budgetary constraints and communicate to the EOD any shortcomings as soon as possible
  • Ensure the hotel complies with all legislation and Staycity Group requirements regarding H&S and any other statutory regulations
  • Enhance the team's skills and knowledge by delivering department-specific training plans that align with our strategic goals
  • Cultivate collaborative relationships with other departments, fostering effective communication and teamwork across the organization
  • Monitor and action feedback received through guest satisfaction surveys, 3rd party websites and from guests after, and during their stay and resolve any guest disputes and complaints in a professional manner and within guidelines issued
What we offer
What we offer
  • Bonus scheme that rewards high performers
  • - based on our core values and tied to individual property goals
  • Automatically enrolled into our leadership program: Elevate - designed to harness your skills and make you an even better values driven and commercially astute leader
  • Discounted rates for overnight stays for you, and your family and friends
  • Refer and earn scheme – earn up to £/€550
  • Education Support to help you foster new skills
  • Volunteer days: 2 paid volunteer days per year
  • EAP: Support for you and your family when you need it with our Employee Assistance Program (EAP)
  • Fulltime
Read More
Arrow Right

Project Manager/Sr. Consultant

Location
Location
United States , King of Prussia
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Information Technology, Computer Science, Business, Management, or a related discipline
  • equivalent relevant experience may be considered
  • 5–10 years of progressive experience managing cross-functional business and technology projects in complex environments
  • Demonstrated success leading enterprise application initiatives or large-scale systems projects in areas such as finance, supply chain, manufacturing, or warehouse operations
  • Strong project management expertise with the ability to prioritize work, manage deadlines, and maintain close attention to detail
  • Excellent facilitation, interviewing, and workshop leadership skills, with the ability to gather input from diverse stakeholders and identify root causes
  • Strong written and verbal communication skills, including experience presenting updates and influencing senior leaders or executive audiences
  • PMP certification is preferred and hands-on IT project management experience is required
Job Responsibility
Job Responsibility
  • Direct full project delivery across large-scale business and IT initiatives, maintaining control over scope, budget, timeline, quality standards, and targeted results
  • Lead collaborative sessions with stakeholders across departments to gather requirements, analyze current and future workflows, uncover missing process details, and convert findings into prioritized action plans
  • Build and manage integrated project schedules, staffing plans, and milestone tracking across internal teams, external partners, and third-party vendors
  • Monitor project health by identifying risks, issues, and dependencies early, driving mitigation activities, clearing obstacles, and escalating critical concerns when appropriate
  • Partner with senior leadership, business sponsors, and executive stakeholders to align priorities, support decision-making, and manage expectations across competing initiatives
  • Oversee enterprise application and technology-focused projects, including software deployments, platform upgrades, application development efforts, and infrastructure-related work
  • Coordinate project change control by evaluating proposed adjustments, securing approvals, and ensuring updates to baselines are communicated and executed effectively
  • Drive communication and change readiness efforts through stakeholder planning, training coordination, cutover preparation, and launch readiness activities
  • Establish project charters, business cases, success measures, and post-implementation validation plans, while capturing lessons learned to strengthen future project execution
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
  • Fulltime
Read More
Arrow Right

Ai Data Associate With French

AI is the most transformational technology of our time, capable of tackling some...
Location
Location
Poland , Gdansk
Salary
Salary:
7916.66 PLN / Month
Amazon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • An Associate’s Degree or related work experience
  • C1+ or equivalent fluency in French language
  • Good working knowledge (B2+) in both spoken and written English
  • Strong business writing skills with ability to create reports, proposals, and professional correspondence
  • Advanced reading comprehension with ability to analyze complex business documents
  • Developed analytical thinking and structured problem-solving capabilities
  • Strong ability to interpret and implement detailed instructions across various projects
  • Proficient research skills with experience gathering and synthesizing information from multiple sources
  • Proven attention to detail in managing complex tasks and documents
Job Responsibility
Job Responsibility
  • Maintain and follow strict confidentiality as customer privacy is our most important tenet
  • Work with a range of different types of data including, but not limited to: text, speech, audio, image, and video
  • Deliver high-quality labelled data, using guidelines provided to meet our KPIs and using in-house tools and software, as part of Amazon's commitment to developing and deploying AI responsibly
  • Demonstrate proficiency in generating high quality human insight data across a range of modalities, inclusive of text, image video and audio
  • Capable of making sound judgments and logical decisions when faced with ambiguous or incomplete information while performing tasks
  • Eye for detail and ability to pivot from one category of requirement to another instantaneously
  • Demonstrate support on daily operational deliverables for multiple task types assigned to you and the team
  • Analyze root causes, identify error patterns, and propose solutions to enhance the quality of labeling tasks and their outputs
  • Responsible for identifying day-to-day process and operational issues in Standard Operating Procedure, tools and suggest changes to unblock operations
  • Demonstrate ownership in floor support to clarify internal queries during execution on need basis
What we offer
What we offer
  • performance bonuses
  • pension plans
  • health insurance
  • life and disability insurance
  • business travel insurance
  • MultiSport card
  • employee discount code and program (EDC/Extras)
  • commuter benefits where applicable
  • Employee Assistance Program (private and confidential counselling and referral service to support health and well-being)
  • Fulltime
Read More
Arrow Right

Customer Relations Specialist

We are hiring a Customer Relations Specialist to support clients with travel pla...
Location
Location
United States , Omaha
Salary
Salary:
Not provided
travelingwithtasha.com Logo
Traveling with Tasha
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Detail-oriented
  • Self-motivated
  • Comfortable using online systems and digital tools
  • Strong communication skills
  • Strong organization skills
  • Strong time management skills
  • Ability to work independently
  • Ability to manage multiple clients
Job Responsibility
Job Responsibility
  • Responding to client inquiries
  • Researching travel options
  • Coordinating and confirming reservations
  • Maintaining consistent follow-up communication
  • Ensuring accuracy in booking details
  • Providing support throughout the planning process
What we offer
What we offer
  • Flexible remote work
  • Fulltime
Read More
Arrow Right

Administrative Assistant

Robert Half is seeking a detail-oriented Administrative Assistant for a 1-2‑mont...
Location
Location
United States , Cedar Rapids
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of administrative experience
  • Strong attention to detail with the ability to manage high volumes of data
  • Proficiency with standard office software and scanning tools
  • Clear, professional written and verbal communication skills
  • Ability to multitask and work effectively in a deadline-driven environment
Job Responsibility
Job Responsibility
  • Scan daily work product, abstracts, and supporting documents
  • Retrieve and process information, providing prompt communication to clients
  • Generate searches, reports, and retrieve documents to support research requests
  • Process incoming and outgoing mail and manage courier deliveries related to title work
  • Answer phones, respond to general information requests, and route calls to appropriate departments
  • Provide ongoing clerical and research support to the production team
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right