Research Engineer, Frontier Evals & Environments Job at OpenAI (San Francisco)

Research Engineer, Frontier Evals & Environments - Finance

The Frontier Evals team builds north star model evaluations to drive progress to...

Location

United States , San Francisco

Salary:

205000.00 - 380000.00 USD / Year

OpenAI

Expiration Date

Until further notice

Requirements

Strong engineering and statistical analysis skills (with at least 2-3 years of full-time technical experience)
Passionate about evals for real world applications and knowledge work
Detail-oriented and thorough
Team player / willing to do a variety of tasks to move the team forward
Passionate and knowledgeable about AGI/ASI measurement
Able to operate effectively in a dynamic and extremely fast-paced research environment as well as scope and deliver projects end-to-end

Job Responsibility

Identify important model capabilities, skills, and behaviors that are crucial to financial workflows, and design methods to quantify performance in these areas
Own and pursue a research agenda to identify an important model capability (especially as it relates to financial reasoning) and build evals to measure it
Continuously refine evaluations of frontier AI models to assess the extent of frontier capabilities

What we offer

Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
401(k) retirement plan with employer match
Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
Mental health and wellness support
Employer-paid basic life and disability coverage
Annual learning and development stipend to fuel your professional growth
Daily meals in our offices, and meal delivery credits as eligible

Fulltime

New

Product Manager, Leadership

At Meta, we're shaping innovative experiences in service of giving people the po...

Location

United States of America , Menlo Park

Salary:

249000.00 - 323000.00 USD / Year

Meta

Expiration Date

Until further notice

Requirements

12+ years of experience in Product Management and/or Product Design
12+ years of experience working collaboratively with engineering, design and user research teams
Critical thinking/analytical leadership experience
Demonstrated proficiency using AI-enabled tools to build product artifacts at scale
Experience developing and championing AI-native strategies across organizations
Experience presenting to senior executive audiences
BA/BS in Computer Science or related field

Job Responsibility

Vision & Strategy - Is the primary driver for identifying and shaping transformative, long-term opportunities across multiple large Product areas or portfolios, setting vision and strategy that influences Meta’s broader organizational and industry direction
Generates buy-in and drives consensus across multiple organizations and senior leadership, bringing clarity and structure to highly ambiguous, company-level opportunities
consistently demonstrates initiative and executes with minimal oversight, often setting new standards
Critically evaluate when AI is (and isn't) the optimal solution at portfolio level, setting the standard for rigorous tradeoff analysis
Translate AI capabilities into compelling, differentiated product visions that define market categories
Champion AI-native strategies including comprehensive evals and data strategies that enable org-wide continuous improvement
Leadership & Collaboration - Drive product development with teams of engineers and designers, while maintaining team health
Work closely with cross-functional teams to drive product mission, define product requirements, coordinate resources from other groups (design, legal, etc.), develop roadmaps, and guide the team through key milestones
Reimagine workflows, responsibly using AI tools to transform team velocity and capability at organizational scale
Foster a culture of rapid experimentation and learning that becomes a competitive advantage

What we offer

bonus
equity
benefits

Fulltime

Product Manager, Enterprise

Luma has built frontier multimodal capabilities — foundation models, Canvas, and...

Location

United States , SF Bay Area

Salary:

225000.00 - 325000.00 USD / Year

Luma AI

Expiration Date

Until further notice

Requirements

Staff or Principal-level product management experience (L5/L6 seniority or above) with a track record of building enterprise products in high-ambiguity environments
You’ve built successful enterprise AI applications — especially agentic products with strong real-world adoption, not just traditional SaaS
You possess functional AI/ML fluency — not just academic. You can reason about model capabilities, evals, latency/cost tradeoffs, fine-tuning vs. prompting vs. tool use, and the shape of a frontier roadmap. You are technical enough to prototype a solution on a whiteboard with a customer’s ML team and credible enough that a research lead actually wants you in the room
You operate effectively across complex cross-functional environments involving product, engineering, research, go-to-market, and customer-facing teams
You have a general manager mindset with P&L visibility and experience — you think in terms of business tradeoffs, commercial viability, and revenue outcomes, not just product metrics
You thrive in significant ambiguity with extremely high agency and a strong self-starter mentality — this is the highest-priority trait for this role
You bring consultative PM skills and the ability to play an advisory role with enterprise customers, asking clarifying questions about needs and commercial viability rather than immediately saying yes to requests
You have a mix of large-company and earlier-stage startup experience, and can work in extremely unstructured environments with limited scaffolding
You can partner with and influence research teams in product development environments, helping ensure frontier model capabilities are shaped by market needs

Job Responsibility

Define product opportunities across enterprise segments by identifying customer profiles, workflows, jobs-to-be-done, and unmet needs in marketing, advertising, and entertainment
Translate Luma’s capabilities across models, Canvas, Agents and APIs into clear product definitions that serve real commercial customers and generate revenue
Drive cross-functional execution across research, product, engineering, go-to-market, and forward-deployed teams within a pod-based team structure
Partner directly with enterprise customers to understand requirements, validate opportunities, and shape product direction — showing up in customer meetings to represent research, product strategy, and vision
Play a consultative product role that connects customer feedback to product strategy and roadmap decisions, complementing forward-deployed engineers and creatives rather than duplicating their work
Bridge market signals back to research and product development teams, establishing feedback loops so that model prioritization and evaluation are informed by real commercial needs
Build agentic products — not just traditional SaaS — that serve clear enterprise needs and can generate revenue immediately while scaling to serve broader demand
Operate with a GM-style mindset, making product choices that align with revenue goals, commercial viability, and practical business outcomes

Fulltime

Product Manager (Leadership)

At Meta, we're shaping innovative experiences in service of giving people the po...

Location

Singapore , Singapore

Salary:

Not provided

Meta

Expiration Date

Until further notice

Requirements

12+ years of experience in Product Management and/or Product Design
12+ years of experience working collaboratively with engineering, design and user research teams
Critical thinking/analytical leadership experience
Demonstrated proficiency using AI-enabled tools to build product artifacts at scale
Experience developing and championing AI-native strategies across organizations
Experience presenting to senior executive audiences
BA/BS in Computer Science or related field

Job Responsibility

Vision & Strategy - Is the primary driver for identifying and shaping transformative, long-term opportunities across multiple large Product areas or portfolios, setting vision and strategy that influences Meta’s broader organizational and industry direction
Generates buy-in and drives consensus across multiple organizations and senior leadership, bringing clarity and structure to highly ambiguous, company-level opportunities
consistently demonstrates initiative and executes with minimal oversight, often setting new standards
Critically evaluate when AI is (and isn't) the optimal solution at portfolio level, setting the standard for rigorous tradeoff analysis
Translate AI capabilities into compelling, differentiated product visions that define market categories
Champion AI-native strategies including comprehensive evals and data strategies that enable org-wide continuous improvement
Leadership & Collaboration - Drive product development with teams of engineers and designers, while maintaining team health
Work closely with cross-functional teams to drive product mission, define product requirements, coordinate resources from other groups (design, legal, etc.), develop roadmaps, and guide the team through key milestones
Reimagine workflows, responsibly using AI tools to transform team velocity and capability at organizational scale
Foster a culture of rapid experimentation and learning that becomes a competitive advantage

Fulltime

Applied Research - Forward-Deployed

Prime Intellect builds the infrastructure that frontier AI labs build internally...

Location

United States , San Francisco

Salary:

150000.00 - 300000.00 USD / Year

Prime Intellect

Expiration Date

Until further notice

Requirements

Deep hands-on experience building, evaluating, or deploying LLM-based agents in the past 1–2 years
Strong intuition for evaluation design
Working understanding of RL and post-training concepts (GRPO, RLHF, reward modeling, SFT)
Strong Python skills and comfort with the modern AI stack (Hugging Face, inference engines, agent frameworks)
Experience in a customer-facing or consulting-adjacent technical role, or as a technical founder
Excellent written and verbal communication
High agency and comfort with ambiguity

Job Responsibility

Embed directly with strategic customers to understand their agent architectures, failure modes, and product goals
Design and build custom RL environments, evaluation harnesses, and verifiers that capture what 'good' looks like for each customer's domain
Architect agent scaffolding — tool use, multi-step reasoning, memory, sandbox execution — tailored to customer workflows
Configure and launch training runs on Lab, iterating on reward functions, rollout strategies, and evaluation criteria
Serve as the technical lead for engagements end-to-end: from discovery through deployed, improved models
Identify repeatable patterns from customer engagements and codify them into reference implementations, templates, and documentation
Serve as the voice of the customer internally, shaping the roadmap for Lab, verifiers, the Environments Hub, and training infrastructure
Build high-quality examples and 'recipes' that make it easy for new customers and open-source contributors to extend the stack
Contribute to technical content (blog posts, tutorials, case studies) that demonstrates real-world platform usage
Develop novel evaluation methodologies for agentic behavior — multi-step reasoning, tool use correctness, recovery from failure, long-horizon task completion

What we offer

Cash Compensation Range of $150-300k + equity incentives
Flexible Work (San Francisco or hybrid-remote)
Visa Sponsorship & relocation support
Professional Development budget
Team Off-sites & conference attendance

Fulltime

Product Manager, Central Products

Meta Product Managers work with cross-functional teams of engineers, designers, ...

Location

United States , Menlo Park

Salary:

205000.00 - 277000.00 USD / Year

Meta

Expiration Date

Until further notice

Requirements

10+ years product management and/or Product Design
10+ years of experience working collaboratively with engineering, design and user research teams
Experience navigating through the full product life-cycle, integrating customer feedback into product requirements, driving prioritization, and pre- and post-launch execution
Critical thinking and analytical leadership experience
Demonstrated proficiency using AI-enabled tools to build product artifacts at scale
Experience developing and championing AI-native strategies across organizations
Experience presenting to executive audiences
BA/BS in Computer Science or related field

Job Responsibility

Is the primary driver for identifying significant near and long-term opportunities in a large Product area, and driving product mission, strategies, and roadmaps in the context of broader organizational strategies and goals
Generate buy-in and drive consensus across organizations. Bring clarity and structure to ambiguous opportunities. Consistently demonstrate initiative and execute with limited oversight
Critically evaluate when AI is (and isn't) the optimal solution at portfolio level, setting the standard for rigorous tradeoff analysis
Translate AI capabilities into compelling, differentiated product visions that define market categories
Champion AI-native strategies including comprehensive evals and data strategies that enable org-wide continuous improvement
Drive product development with teams of engineers and designers, while maintaining team health
Work closely with cross-functional teams to drive product mission, define product requirements, coordinate resources from other groups (design, legal, etc.), develop roadmaps, and guide the team through key milestones
Reimagine workflows, responsibly using AI tools to transform team velocity and capability at organizational scale
Foster a culture of rapid experimentation and learning that becomes a competitive advantage
Scale AI best practices (including responsible AI use), workflows, and artifacts across the organization so capability compounds exponentially

What we offer

bonus
equity
benefits

Fulltime

Product Manager - Growth & Monetization

At Meta, we're shaping innovative experiences in service of giving people the po...

Location

Singapore

Salary:

Not provided

Meta

Expiration Date

Until further notice

Requirements

10+ years product management and/or Product Design
10+ years of experience working collaboratively with engineering, design and user research teams
Experience navigating through the full product life-cycle, integrating customer feedback into product requirements, driving prioritization, and pre- and post-launch execution
Critical thinking and analytical leadership experience
Demonstrated proficiency using AI-enabled tools to build product artifacts at scale
Experience developing and championing AI-native strategies across organizations
Experience presenting to executive audiences
BA/BS in Computer Science or related field

Job Responsibility

Is the primary driver for identifying significant near and long-term opportunities in a large Product area, and driving product mission, strategies and roadmaps in the context of broader organizational strategies and goals
Generate buy-in and drive consensus across organizations. Bring clarity and structure to ambiguous opportunities. Consistently demonstrate initiative and execute with limited oversight
Critically evaluate when AI is (and isn't) the optimal solution at portfolio level, setting the standard for rigorous tradeoff analysis
Translate AI capabilities into compelling, differentiated product visions that define market categories
Champion AI-native strategies including comprehensive evals and data strategies that enable org-wide continuous improvement
Drive product development with teams of engineers and designers, while maintaining team health
Work closely with cross-functional teams to drive product mission, define product requirements, coordinate resources from other groups (design, legal, etc.), develop roadmaps, and guide the team through key milestones
Reimagine workflows, responsibly using AI tools to transform team velocity and capability at organizational scale
Foster a mindset of rapid experimentation and learning that becomes a competitive advantage
Scale AI best practices (including responsible AI use), workflows, and artifacts across the organization so capability compounds exponentially

Researcher, Preparedness

The Preparedness team helps us prepare for the development of increasingly capab...

Location

United States , San Francisco

Salary:

295000.00 - 445000.00 USD / Year

OpenAI

Expiration Date

Until further notice

Requirements

Passionate and knowledgeable about short-term and long-term AI safety risks
Ability to think outside the box and have a robust 'red-teaming mindset'
Experience in ML research engineering, ML observability and monitoring, creating large language model-enabled applications, and/or another technical domain applicable to AI risk
Able to operate effectively in a dynamic and extremely fast-paced research environment as well as scope and deliver projects end-to-end

Job Responsibility

Own the scientific validity of frontier preparedness capability evaluations—designing new evals grounded in real threat models (including high-consequence domains like CBRN as well as cyber and other frontier-risk areas), and maintaining existing evals so they don't stale or silently regress
Define datasets, graders, rubrics, and threshold guidance, and produce auditable artifacts (evaluation cards, capability reports, system-card inputs) that leadership can trust during high-stakes launches
Work on identifying emerging AI safety risks and new methodologies for exploring the impact of these risks
Build (and then continuously refine) evaluations of frontier AI models that assess the extent of identified risks
Design and build scalable systems and processes that can support these kinds of evaluations
Contribute to the refinement of risk management and the overall development of 'best practice' guidelines for AI safety evaluations

What we offer

Medical, dental, and vision insurance for you and your family, with employer contributions to Health Savings Accounts
Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses (parking and transit)
401(k) retirement plan with employer match
Paid parental leave (up to 24 weeks for birth parents and 20 weeks for non-birthing parents), plus paid medical and caregiver leave (up to 8 weeks)
Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees
13+ paid company holidays, and multiple paid coordinated company office closures throughout the year for focus and recharge, plus paid sick or safe time (1 hour per 30 hours worked, or more, as required by applicable state or local law)
Mental health and wellness support
Employer-paid basic life and disability coverage
Annual learning and development stipend to fuel your professional growth
Daily meals in our offices, and meal delivery credits as eligible

Fulltime

Select Country

Research Engineer, Frontier Evals & Environments

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?