CrawlJobs Logo

Senior Product Operations Manager, Evaluation

United States, San Francisco 178000.00 - 210000.00 USD / Year · Job Posted February 18, 2026
Apply Position
Job Link Share

Job Description

We’re looking for a technical, systems-minded operator to build and scale the evaluation engine behind Harvey’s platform. As we expand globally, ensuring our models behave reliably, accurately, and jurisdictionally correctly is mission-critical—and evaluation complexity is increasing 10x. As a member of our Product Operations team, you’ll work closely with Applied Legal Researchers, Product, Engineering, AI Research, and human data providers to operationalize evaluation methodologies and embed them into our product development lifecycle. You’ll create the workflows, systems, and tooling that make evaluation a first-class product capability at Harvey. This is a high-ownership role for someone who thrives in ambiguity, loves building structure from ambiguity, and wants to help scale the evaluation infrastructure of a global AI company.

Job Responsibility

  • Build and scale the systems that power model and product evaluations across Harvey
  • Embed evaluation workflows and readiness checkpoints into the product development lifecycle
  • Create the single source of truth for evaluation status, results, history, and launch readiness
  • Turn Expert-designed evaluation methodologies into scalable, repeatable operational processes
  • Manage relationships with human data vendors and ensure evaluation quality meets legal standards
  • Work with Engineering and Research to improve evaluation tooling, automation, and dashboards
  • Drive evaluation readiness for major product and model launches across geographies and jurisdictions
  • Document and operationalize evaluation governance as complexity increases
  • Help define how Harvey ensures model accuracy, reliability, and trust at global scale

Requirements

  • 4–7+ years in technical program management, product operations, research operations, or evaluation/benchmarking roles
  • Experience working with ML/AI evaluations, benchmarking frameworks, or scientific workflows
  • Comfort with statistical methodologies and SQL or Python, or similar tools to interpret evaluation data
  • Ability to work deeply with legal experts and operationalize complex evaluation methodologies
  • Strong cross-functional coordination skills across Product, Engineering, Research, and data providers/vendors
  • High attention to detail and a bias toward clarity, rigor, and reproducibility
  • Ability to navigate extreme ambiguity and bring order to complex systems
  • Strong communication skills and comfort translating technical nuance for diverse stakeholders
  • Desire to do whatever it takes to make evaluation systems successful—from writing documentation to diagnosing pipeline issues

Nice to have

  • Experience in legal tech or working with domain experts in regulated industries
  • Experience managing human data providers or human-in-the-loop evaluation pipelines
  • Background in ML research, data quality management, or evaluation science
  • Early employee at a hyper-growth startup
  • Experience at world-class product or platform operations orgs (ex: Stripe, Ramp)

What we offer

  • Offers Equity
  • Offers Bonus
  • Comprehensive health, dental and vision coverage
  • retirement benefits (401k match up to 4%)
  • flexible PTO

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Product Operations Manager, Evaluation

8 matching positions

Senior Product Manager

Commercial Engineering & AI (CEAI) partners closely with stakeholders to acceler...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 5+ years experience in product/service/program management or software development OR equivalent experience.
Job Responsibility
Job Responsibility
  • Own end-to-end product outcomes from strategy through delivery, ensuring measurable customer and business impact
  • Define product vision, strategy, and roadmap grounded in customer insights, data, and business priorities
  • Lead the design and delivery of AI-powered support experiences and agent-based workflows, including autonomous resolution capabilities
  • Drive product execution across engineering, data, design, and service delivery teams, delivering scalable, high-quality solutions
  • Lead prioritization, backlog management, and cross-team alignment, focusing teams on the highest-impact work
  • Use live-site signals, experimentation, and operational KPIs to drive continuous product improvement, owning evaluation metrics and iteration velocity
  • Rapidly prototype and validate new ideas using AI tools, accelerating learning and reducing time from concept to impact
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation.
  • Fulltime
Read More
Arrow Right

Senior Product Manager

GM is seeking an experienced Product Manager to support workplace modernization ...
Location
Location
United States , Austin, Texas; Warren, Michigan
Salary
Salary:
Not provided
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree required
  • technical discipline preferred, or equivalent experience
  • 7+ years in product management, product analysis, SaaS solutions, enterprise program execution, transformation delivery, or a related business or technology environment
  • Proven experience gathering and analyzing business and user needs and translating them into user stories, use cases, product requirements, and actionable product recommendations
  • Demonstrated ability to become a product expert and communicate product capabilities and business value across a large enterprise
  • Experience contributing to product roadmaps, backlog priorities, feature planning, or enhancement decisions in partnership with engineering and cross-functional teams
  • Experience presenting to large and varied audiences, including audiences with little or no prior product experience
  • Ability to explain complex product capabilities in simple, practical terms for business users
  • Strong stakeholder management and coordination skills across business, IT, vendors, design, support, and engineering teams
  • Familiarity with major enterprise collaboration ecosystems, modern productivity platforms, SaaS solutions, and enterprise technology practices
Job Responsibility
Job Responsibility
  • Clearly document, communicate, and explain the product vision, strategy, and roadmap for new SaaS capabilities supporting workplace modernization and AI-enabled workflow transformation
  • Serve as the product expert and spokesperson for the platform, building deep knowledge of features, workflows, roadmap direction, implementation considerations, and business value
  • Gather and analyze business and user needs, operational challenges, and product opportunities, and translate them into user stories, use cases, product requirements, and actionable product recommendations
  • Influence backlog priorities and feature enhancement decisions with engineering and other cross-functional teams based on business value, customer needs, technical feasibility, enterprise priorities, and business feedback gathered as the voice of the product
  • Collaborate closely with engineering, design, support, vendors, and business stakeholders to align product goals, delivery priorities, and rollout plans
  • Conduct market research, competitive analysis, and product, user, and business research to identify opportunities, threats, gaps, risks, and improvement areas, and use those insights to explain product value and strengthen enterprise fit
  • Partner with SaaS providers and internal stakeholders to stay current on product capabilities, roadmap updates, support materials, implementation considerations, and emerging opportunities
  • Present to large and diverse audiences with confidence, translating complex product capabilities and technical concepts into clear, relevant, and actionable information
  • Introduce new product capabilities to audiences with little or no prior experience through presentations, demonstrations, walkthroughs, and tailored explanations based on business function, role, and level of technical understanding
  • Analyze target audiences, stakeholder feedback, and relevant usage or readiness data to identify barriers, knowledge gaps, and business needs that influence product adoption and value realization
  • Fulltime
Read More
Arrow Right
New

Senior Product Manager

Commercial Engineering & AI (CEAI) partners closely with stakeholders to acceler...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 5+ years experience in product/service/program management or software development OR equivalent experience.
Job Responsibility
Job Responsibility
  • Own end-to-end product outcomes from strategy through delivery, ensuring measurable customer and business impact
  • Define product vision, strategy, and roadmap grounded in customer insights, data, and business priorities
  • Lead the design and delivery of AI-powered support experiences and agent-based workflows, including autonomous resolution capabilities
  • Drive product execution across engineering, data, design, and service delivery teams, delivering scalable, high-quality solutions
  • Lead prioritization, backlog management, and cross-team alignment, focusing teams on the highest-impact work
  • Use live-site signals, experimentation, and operational KPIs to drive continuous product improvement, owning evaluation metrics and iteration velocity
  • Rapidly prototype and validate new ideas using AI tools, accelerating learning and reducing time from concept to impact
  • Drive go-to-market, readiness, and change management, partnering across stakeholder teams to land new agent capabilities and accelerate adoption
  • Communicate strategy, trade-offs, and results to cross-org stakeholders, building alignment and influence across organizational boundaries
  • Champion Responsible AI principles, ensuring agent-based experiences are safe, grounded, transparent, and compliant with Microsoft's AI standards, privacy, and security requirements
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation
  • Fulltime
Read More
Arrow Right
New

Senior Product Manager

NMI's SaaS products power how ISOs and ISVs manage their entire merchant lifecyc...
Location
Location
United States
Salary
Salary:
120000.00 - 145000.00 USD / Year
parking.net Logo
Parking Network B.V.
Expiration Date
July 26, 2026
Flip Icon
Requirements
Requirements
  • 5+ years in product management with at least 3 years in payments, fintech, or B2B SaaS
  • Demonstrated experience taking a product into a new market or geography—you've done the research, built the requirements, and seen a market launch through to adoption
  • Hands-on product owner experience: you write user stories and acceptance criteria (or use AI to do so), manage Jira backlogs, and stay close to engineering through delivery
  • Strong understanding of payment localization: local payment methods, processor integrations, currency handling, regulatory compliance, and operational differences by market
  • GTM instincts: you've supported or led a product launch in a new market, contributed to positioning or messaging, and understand what it takes to enable a sales or channel team in an unfamiliar geography
  • Comfort operating in ambiguity—this market and this roadmap are being built
  • you don't need a finished blueprint to start
  • Data-driven: you define success metrics before you build and you use data to evaluate what's working
  • Excellent written communicator: your stories are clear, your requirements leave no room for interpretation, and your stakeholder updates are concise
  • Fluent with AI tools for product work—market research, spec drafting, synthesis, analysis
Job Responsibility
Job Responsibility
  • Own the international expansion roadmap for Merchant Central, phased by market: Canada first, then UK/EU, then determine what comes next
  • Lead the integration of new strategic processors into Merchant Central from end to end—partner with Engineering on API specs, boarding flows, processor-specific configurations, local payment methods, and reconciliation
  • Research and document market requirements for each new geography: local payment methods, regulatory constraints, currency, language (ex French for Quebec), data residency, and compliance frameworks
  • Write detailed user stories, acceptance criteria, and bug reports for the delivery team—you own delivery and strategy
  • Manage the product backlog: groom, prioritize, and sequence work to keep the team moving with clarity on scope and dependencies
  • Run or actively participate in sprint ceremonies: planning, review, retrospectives
  • own delivery visibility for your roadmap
  • Define the go-to-market approach for each market launch: positioning, partner messaging, launch readiness checklist, and sales/channel enablement
  • Partner with Product Marketing to develop external-facing content and messaging for new geographies—you understand what makes Canada (or the UK) different and you help us say it clearly
  • Build competitive intelligence for each target market: who are the incumbents, what do ISOs and ISVs expect, and where does NMI have a differentiated angle
What we offer
What we offer
  • A remote first culture
  • Competitive compensation and benefits
  • Personal growth and advancement opportunities
  • Flex PTO & dedicated sick time
  • 13 Paid Holidays
  • Health, Dental and Vision Insurance
  • Life, ADD, Short-term and Long-term Disability insurance
  • 401k matching up to 4% after two months of service
  • Flexible Spending Account
  • Gym membership discount
  • Fulltime
Read More
Arrow Right

Senior Product Manager - AI Content Creation

As Senior Product Manager for AI Content Creation, you will lead the evolution o...
Location
Location
Germany , Berlin
Salary
Salary:
Not provided
babbel.com Logo
Babbel
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A demonstrated track record of owning complex digital products end-to-end, from discovery through delivery
  • Strong product judgment and the ability to operate in ambiguous, fast-evolving problem spaces
  • Solid understanding of AI-powered systems and how to productize them. Experience building AI products is a plus
  • Experience working closely with engineers and data teams
  • A high bar for quality and a strong focus on user outcomes
  • Excellent communication and stakeholder alignment skills
  • Able to drive progress in ambiguous environments and communicate decisions clearly to technical and non-technical stakeholders
  • Note that our company's operating language is English, so you will need to be able to work in English.
Job Responsibility
Job Responsibility
  • Own the product strategy for AI-driven content creation, from discovery to execution
  • Create, improve and scale AI generation workflows, including prompting, evaluation, and quality loops
  • Enable faster creation of new languages, levels, and learning experiences, including more complex language systems
  • Balance automation and quality, evolving human-in-the-loop processes where needed
  • Define success metrics and use learner data and feedback to guide decisions
  • Collaborate across Product, Engineering, Data, Design and Learning Content to deliver reliable, learner-ready solutions
What we offer
What we offer
  • Flexible working models
  • Learning budgets
  • Free language learning
  • An inclusive, supportive environment that values curiosity and growth
  • Fulltime
Read More
Arrow Right

Senior Product Manager

We are seeking a Senior Product Manager to lead the transformation of how client...
Location
Location
United States , Phoenix
Salary
Salary:
Not provided
technologent.com Logo
Technologent
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Engineering, or a related discipline
  • 5+ years of experience in product management, with a focus on digital platforms or service delivery solutions
  • Hands-on experience working with enterprise service management platforms, including strong familiarity with IT service workflows and modules
  • Proven ability to manage product roadmaps, backlogs, and feature prioritization aligned to business outcomes
  • Experience designing and optimizing multi-channel service experiences across platforms such as web, chat, mobile, and in-person support
  • Strong understanding of workflow automation, service optimization, and modern support technologies
  • Demonstrated ability to translate user insights into actionable product features and improvements
  • Excellent communication skills with the ability to influence stakeholders at all levels
  • Strong analytical and problem-solving skills with a data-driven approach to decision-making
Job Responsibility
Job Responsibility
  • Define and execute a comprehensive strategy for delivering technology support services to a global user base
  • Communicate product vision, roadmap priorities, and execution progress to senior leadership and key stakeholders
  • Engage directly with users to understand pain points, expectations, and behaviors to inform product decisions and roadmap direction
  • Collaborate with cross-functional teams including engineering, service delivery, and platform owners to identify improvement opportunities and implement new capabilities
  • Drive prioritization of initiatives that maximize user value while ensuring operational efficiency and service excellence
  • Develop and maintain product roadmaps, aligning features and enhancements with business goals and user needs
  • Partner with technical teams throughout the product lifecycle to ensure successful design, development, and delivery of solutions
  • Establish metrics and KPIs to measure service performance, user engagement, and overall impact, leveraging data to guide decision-making
  • Promote adoption and clearly articulate the value of service offerings across the organization
  • Ensure effective measurement frameworks and feedback mechanisms are in place to continuously improve service quality
Read More
Arrow Right

Senior Product Manager

Substrate Storage Fabric powers the dependable, hyperscale storage and retrieval...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 5+ years experience in product/service/program management or software development OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Own roadmap definition, prioritization, and execution for one or more Storage Fabric platform areas across efficiency, trust, resilience, and modernization
  • Identify customer, partner, and platform needs using AI, telemetry, feedback, and competitive signals to shape product direction and improve product-market fit
  • Drive feature and investment prioritization for areas such as availability and durability, intelligent placement and runtime scheduling, fleet efficiency, security and compliance readiness, investigation telemetry, BCDR, and safe onboarding or migration paths
  • Define success metrics and use performance, reliability, and business data to evaluate product outcomes, refine strategy, and drive continuous improvement
  • Partner closely with engineering, design, operations, security, compliance, product marketing, and business planning teams to align customer needs with development and release plans
  • Translate distributed-systems concepts, AI-enabled capabilities, and platform tradeoffs into clear business value, product decisions, and stakeholder communication
  • Engage customers and partners to gather feedback, validate priorities, and improve release readiness, adoption, and long-term product direction
  • Own the end-to-end product lifecycle for key initiatives, from problem framing and feature definition through launch, iteration, and deprecation where needed
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation
  • Fulltime
Read More
Arrow Right

Senior Product Manager, Data Platform

We are seeking a Senior Product Manager to join our Data & AI team and lead prod...
Location
Location
United States , Los Angeles
Salary
Salary:
165000.00 - 200000.00 USD / Year
geniussports.com Logo
Genius Sports
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant product management experience, with meaningful experience owning data platform, infrastructure, API, or enterprise platform products
  • Strong understanding of real-time data systems, data pipelines, ETL/ELT, data warehousing, APIs, and large-scale distributed platforms
  • Experience working closely with engineering teams to deliver complex technical products and platform capabilities
  • Demonstrated ability to define product strategy and execute in cross-functional environments with multiple stakeholders and competing priorities
  • Strong analytical and problem-solving skills, with the ability to evaluate data sources, define business value, and make sound prioritization decisions
  • Excellent written and verbal communication skills, including the ability to translate between technical and non-technical audiences
  • Experience building or supporting products related to analytics, BI, machine learning, or AI-enabled use cases
  • Comfort operating in fast-paced environments where reliability, scalability, latency, and data quality are business-critical
  • Familiarity with live event data collection workflows, including operator-assisted and computer-vision-driven systems
  • Experience with modern data platform technologies, streaming architectures, observability tooling, and cloud-based data ecosystems
Job Responsibility
Job Responsibility
  • Maintain and Modernize Data Pipelines
  • Enable Data Warehousing, Reporting, and Advanced Analytics
  • Build, Maintain, and Evolve Data Distribution Tools
  • Lead Product Strategy and Cross-Functional Execution
What we offer
What we offer
  • Competitive salary and range of benefits
  • Committed to supporting employee wellbeing and helping you grow your skills, experience and career
  • Fulltime
Read More
Arrow Right