Staff Product Manager, Evaluations and Automation Job at Airbnb

Job Description

The Staff Product Manager, Evals & Automation, reporting to the Product Lead, Payments Risk, will define the strategy and roadmap for how Airbnb evaluates, learns from, and automates risk decisions—ensuring we protect the marketplace while minimizing friction and insult to good users.

Job Responsibility

Define the product vision, strategy, and execution for critical evaluation workflows for risk detection & mitigation
Design principled evaluation frameworks that determine the right size and shape of holdouts, control groups, and manual review samples—without degrading model performance or decision quality
Drive automation that meaningfully reduces manual reviews, operational burden, and customer friction, while preserving the labels and signals required to keep models accurate and resilient over time
Review how models, holdouts, and manual reviews interact across the decisioning funnel—examining false positives, label coverage, approval lift, and downstream loss impact
Work closely with Data Science, Machine Learning, Risk Engineering, and Operations to understand where current evaluation approaches over-sample good users, introduce bias, or create unnecessary friction
Define the end-to-end product vision for risk evaluations and learning loops, building a multi-year roadmap that balances statistical rigor, operational efficiency, customer experience, and regulatory expectations
Own requirements and execution for systems that: Dynamically size and manage holdouts
Optimize when and how manual reviews are invoked
Preserve high-quality labels without over-reliance on human review
Enable faster, safer iteration on risk models and policies
Partner deeply with Fraud & Safety, Trust, Legal, Policy, Customer Support, and Payments Operations to align on decision principles and ensure evaluation strategies are understood, trusted, and actionable across the company
Collaborate with Payments Operations and Support to reduce manual effort, handle edge cases better, and unlock high-quality decisioning at scale
Partner deeply with other Airbnb platform organizations such as Fraud & Safety, AirCover, Legal, Customer Support, Policy Enforcement & Payments Operations teams to align on the vision and mission and coordinate tactics to deliver mutually aligned outcomes

Requirements

9+ years of Product Management experience, or 6+ years with a BS/MS with significant work in payments, risk/fraud, chargebacks/, or marketplace integrity
Demonstrated experience working with model-driven systems, evaluation frameworks, experimentation, and decisioning under uncertainty
Strong intuition for long-tail risk problems, false positives, and the unintended consequences of sampling and labeling strategies
Systems thinker who can design learning loops that scale globally and remain robust as adversarial behavior evolves
Highly data-literate: comfortable working with loss curves, precision/recall tradeoffs, operational metrics, and model performance diagnostics
Systems thinker with the ability to architect modular processes that scale globally
Ability to make principled, high-judgment decisions in ambiguous, high-stakes environments
Takes ownership of multiple product areas or one with significant impact, establishing clear expectations, driving communication, quality and timely delivery
Experience designing end-to-end user experiences that balance risk mitigation with customer empathy
Adapts communication style according to the audience across teams, extended stakeholders and leadership
Empathetic, clear communicator who can lead cross-functional partners through influence, not authority
Autonomously communicates, influences and aligns stakeholders on the products vision/strategy
With minimal guidance defines products that sustain the changing needs of business, industry, users and partners with Eng/Design to translate to long term architecture and Ecosystems thinking, with clear understanding of platform capabilities to be built
Strong analytical chops—you’re comfortable diving into financial & customer data, operational metrics, and model performance
Proven ability to partner deeply with engineering, design, operations, data science, and legal/compliance teams
Comfortable working in a fast-paced environment with evolving regulatory requirements and complex trade-offs
Authorized to work in the United States

What we offer

bonus
equity
benefits
Employee Travel Credits

Airbnb - All Job Offers

Select Country

Staff Product Manager, Evaluations and Automation

Job Description

Job Responsibility

Requirements

What we offer

Looking for more opportunities?

Staff Product Manager, Evaluations and Automation

Staff Product Manager, Cloud Networking

Staff Technical Product Manager, Lead to Order

Product Manager, AI Solutions and Design

Manager, Product & Engineering

Member of Product Staff

Member of Product Staff

Agentic AI Product Manager

Manager, Staff Engineering - Cryptography

Our AI answers in your language