CrawlJobs Logo

AI Evaluation Product Manager

plaud.ai Logo

Plaud

Location Icon

Location:
Singapore

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

Plaud is building the future of AI-native productivity — where meetings, conversations, and knowledge become an extension of your memory. As the first dedicated owner of Memory Evaluation, you will shape how AI understands, stores and recalls human knowledge for millions of users worldwide.

Job Responsibility:

  • Define how 'AI memory quality' should be measured for the industry
  • Turn product goals and real user scenarios into measurable evaluation criteria, evaluation frameworks and reproducible test cases
  • Build the global memory evaluation system from the ground up
  • Create evaluation standards, test pipelines, and datasets across markets (US, EU, Japan, China), working with real multilingual and multimodal data
  • Run continuous quality evaluations across Summary, Ask Plaud and other memory-enabled experiences
  • Become the source of truth for ‘is memory working?’
  • Benchmark against global AI meeting and knowledge products (Otter, Notion AI, NotebookLM, etc.) to extract best practices in memory accuracy, retrieval reliability, and user trust — and convert these insights into scientific evaluation methodology

Requirements:

  • Master's degree or above, preferably in Computer Science, Data Science, AI, or related fields
  • 3+ years of product experience, with at least 1 year working on AI or LLM-related products or features
  • Solid analytical skills and familiarity with data processing (SQL/Python preferred)
  • Ability to communicate clearly in English and collaborate across product, engineering, and operations
  • Strong ownership, structured problem solving, and willingness to learn quickly
  • Understanding of English-speaking user scenarios, product expectations, or cultural differences

Nice to have:

  • Hands-on experience designing test plans, evaluation standards, or datasets for LLM or AI features
  • Familiarity with evaluation of memory-related behaviors (extraction, retrieval, contextual usage), or prior work on summary/Ask AI products
  • Experience reviewing real user data and translating findings into actionable product improvements
What we offer:
  • Market-competitive compensation
  • Global exposure
  • Vibrant, creativity-fueled work atmosphere

Additional Information:

Job Posted:
February 21, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for AI Evaluation Product Manager

Product Manager, AI Models

We’re looking for a Product Manager to help build the future of video editing wi...
Location
Location
United States , San Francisco
Salary
Salary:
155000.00 - 205000.00 USD / Year
descript.com Logo
Descript
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2 or more years of product management experience
  • Ideally with a product of technical complexity that incorporates generative AI models
  • Experience in customer discovery and defining features that solve real problems
  • Ability to tackle ambiguous problems, ask smart questions, and translate ideas into action
  • Attention to detail and craft
Job Responsibility
Job Responsibility
  • Develop a deep understanding of our customers, market, and product opportunities
  • Develop a deep understanding of the landscape of modern AI models and conviction on where Descript should build, buy, partner and contribute to the field
  • Drive company-wide AI initiatives that have meaningful business & user impact, and become an influential advocate for our AI strategy
  • Own and drive the roadmap for AI models at Descript, from idea generation through to execution, evaluation and iteration
  • Collaborate with AI research, engineering, design, and other product teams to ship AI models that delight our users and deliver meaningful impact on Descript’s business
What we offer
What we offer
  • Generous healthcare package
  • 401k matching program
  • Catered lunches
  • Flexible vacation time
  • Fulltime
Read More
Arrow Right

Internal Product Manager, AI Tooling

Figma is growing our team of passionate creatives and builders on a mission to m...
Location
Location
United States , San Francisco; New York
Salary
Salary:
136000.00 - 288000.00 USD / Year
figma.com Logo
Figma
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in product management, technical program management, or systems ownership within a cloud/SaaS environment
  • Proven track record shipping and iterating on AI features that drive impact
  • Strong technical fluency in business systems, integrations, and enterprise IT platforms
  • Broad understanding of LLMs and AI-enabled enterprise systems
  • Proven ability to navigate ambiguity, adapt in constantly evolving environments, and drive clarity and momentum across multiple teams
Job Responsibility
Job Responsibility
  • Define the vision and strategy for internal AI tooling across IT and Business Systems
  • Design and operationalize an intake and prioritization framework to evaluate AI tooling requests, balancing business value, technical feasibility, and compliance requirements
  • Partner with key collaborators across Figma to capture AI needs, translate them into technical requirements, and prioritize solutions that deliver the highest impact
  • Lead the full product lifecycle—from discovery and vendor evaluation through pilots, deployment, integrations, and adoption
  • Guide technical conversations on APIs, data flows, and integration design, collaborating closely with architects and engineers
  • Enforce governance and compliance standards to ensure AI adoption is secure, auditable, and responsible
  • Champion change management and enablement by driving documentation, training, and feedback loops that accelerate adoption
  • Stay current on the AI and enterprise IT ecosystem, identifying new opportunities, emerging risks, and best practices for scaling
What we offer
What we offer
  • equity
  • health, dental & vision benefits
  • retirement with company contribution
  • parental leave & reproductive or family planning support
  • mental health & wellness benefits
  • generous PTO
  • company recharge days
  • a learning & development stipend
  • a work from home stipend
  • cell phone reimbursement
  • Fulltime
Read More
Arrow Right

Sr. Product Manager - AI

In this role, you will be joining the Product Management team reporting to the S...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
highspot.com Logo
Highspot
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5–8 years of PM experience in B2B SaaS, ideally in sales enablement or learning tech, with a proven ability to ship products in agile environments
  • 2+ years working with AI/ML teams, including experience in NLP, NLU, or Generative AI applications
  • Skilled at leading cross-functional teams to deliver high-quality, end-to-end product experiences across complex user scenarios with multiple personas
  • Strong problem-solver with a track record of turning ambiguous inputs into structured roadmaps and impactful outcomes
  • Comfortable working with technical teams—able to discuss trade-offs with engineers and collaborate with data scientists on AI strategy
  • Exceptional communication and stakeholder management skills, with the ability to align executives, sales teams, and designers
  • Highly detail-oriented and user-obsessed, with a passion for crafting polished, intuitive experiences that drive adoption
  • Resilient in fast-paced, ambiguous environments, with the ability to prioritize, multitask, and go beyond role boundaries when needed
Job Responsibility
Job Responsibility
  • You will work with customers and stakeholders to create a strategic roadmap, prioritize problems, and define requirements, then manage the backlog through the product development process from discovery to general availability
  • You will become an expert on our software, customer, and our market, as well as be able to identify, prioritize and execute on both tactical and strategic projects
  • You will collect and analyze feedback from customers, stakeholders and other teams to shape requirements, features and end products
  • You will dig deep into customer and usage data to identify opportunities and evaluate the success of our efforts
  • Own the roadmap for AI features related to coaching insights, rep scoring, skill reinforcement, and conversation intelligence
  • Drive discovery with customers, internal SMEs, and sales leaders to uncover pain points and surface areas where AI can improve productivity, engagement, or skill growth
  • Work across systems—analytics, content, LMS, and CRM—to unify data into actionable insights
  • Validate new capabilities with customers using prototypes, mockups, and early betas
  • measure adoption and iterate quickly
  • Define and measure key success metrics (e.g., coaching engagement, learning retention, time-to-proficiency)
  • Fulltime
Read More
Arrow Right

Principal Product Manager, AI

In this role, you will be joining the Product Management team reporting to the D...
Location
Location
United States , Seattle
Salary
Salary:
167000.00 - 278000.00 USD / Year
highspot.com Logo
Highspot
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years product management experience building B2B SaaS products
  • 4+ years working directly with ML/AI products or platforms
  • Demonstrated technical fluency with modern generative AI / ML concepts (LLMs, RAG, embeddings, vector DBs, fine-tuning, prompt engineering, evaluation metrics, model ops)
  • Proven record of shipping complex, cross-functional AI products or platforms to production and driving measurable business impact
  • Strong quantitative and analytical skills
  • Exceptional communication skills
  • Experience mentoring PMs and participating in hiring
  • a track record of raising the bar for product craft
Job Responsibility
Job Responsibility
  • Define vision & strategy
  • Lead execution end-to-end
  • Own model + product success
  • Drive platform & scale thinking
  • Champion safety, fairness & compliance
  • Be the cross-functional glue
  • Mentor & raise the bar
What we offer
What we offer
  • Comprehensive medical, dental, vision, disability, and life benefits
  • Health Savings Account (HSA) with employer contribution
  • 401(k) Matching with immediate vesting on employer match
  • Flexible PTO
  • 8 paid holidays and 5 paid days for Annual Holiday Week
  • Quarterly Recharge Fridays (paid days off for mental health recharge)
  • 18 weeks paid parental leave
  • Access to Coaches and Therapists through Modern Health
  • 2 volunteer days per year
  • Commuting benefits
  • Fulltime
Read More
Arrow Right

Principal Product Manager, AI

In this role, you will be joining the Product Management team reporting to the D...
Location
Location
Canada , Vancouver
Salary
Salary:
146000.00 - 220000.00 CAD / Year
highspot.com Logo
Highspot
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years product management experience building B2B SaaS products
  • 4+ years working directly with ML/AI products or platforms
  • Demonstrated technical fluency with modern generative AI / ML concepts (LLMs, RAG, embeddings, vector DBs, fine-tuning, prompt engineering, evaluation metrics, model ops)
  • Proven record of shipping complex, cross-functional AI products or platforms to production and driving measurable business impact
  • Strong quantitative and analytical skills
  • Exceptional communication skills
  • Experience mentoring PMs and participating in hiring
  • a track record of raising the bar for product craft
Job Responsibility
Job Responsibility
  • Define vision & strategy
  • Lead execution end-to-end
  • Own model + product success
  • Drive platform & scale thinking
  • Champion safety, fairness & compliance
  • Be the cross-functional glue
  • Mentor & raise the bar
What we offer
What we offer
  • Comprehensive medical, dental, vision, disability, and life benefits
  • Group Retirement Savings Plan (RRSP) and matching employer contributions (DPSP) with immediate vesting
  • Flexible PTO
  • Generous Holiday Schedule + 5 Days for Annual Holiday Week
  • Quarterly Recharge Fridays (paid days off for mental health recharge)
  • Flexible work schedules
  • Access to Coaches and Therapists through Modern Health
  • 2 Volunteer days per year
  • Monthly transportation allowance for employees that work in our Vancouver Hub location
  • Stock options
  • Fulltime
Read More
Arrow Right

Senior Product Manager, AI

Hostaway is a SaaS startup transforming the vacation rental industry. Hostaway A...
Location
Location
Spain , Barcelona
Salary
Salary:
Not provided
hostaway.com Logo
Hostaway
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of product management experience, ideally in B2B/B2C SaaS
  • Strong understanding of AI/ML fundamentals to evaluate feasibility with engineers
  • Solid grasp of LLMs, including prompt design and evaluation, managing quality and risk, and awareness of model capabilities, costs, latency, and context limits
  • Experience assessing and integrating off-the-shelf AI solutions
  • LLMs, APIs, SaaS AI services, and third-party ML products
  • Strong understanding of agentic AI concepts (tool use, planning, orchestration)
  • Deep empathy for users — you think in flows, friction points, and moments of delight
  • Able to make decisions when surrounded by uncertainty
  • Able to lead by influencing without authority, aligning cross-functional teams around clear goals and driving outcomes through trust, clarity, and data-informed decision
Job Responsibility
Job Responsibility
  • Drive outcomes and own the results on a multi-million dollar business line
  • Help develop and implement the Hostaway AI vision, strategy, and roadmap, breaking that down into rapidly shippable projects
  • Collect and analyze customer, stakeholder, and team feedback to shape requirements, features, and end products
  • Work with senior management to create product plans and roadmaps
  • Partner with sales, customer success, and engineering to ship features and products
  • Ensure products and releases are launched correctly and on schedule
  • Define and analyze the metrics that determine success
  • Make creative recommendations to expand the product base and vision
What we offer
What we offer
  • Competitive Compensation based on market rates in the country of the applicant
  • 100% Remote work from anywhere within your country of residence
  • Equity in the form of valuable stock options
  • Values-Driven Leadership
  • Professional Growth with learning and development opportunities
  • Annual Paid Leave aligned with country-specific norms
  • Geographic Specific Benefits such as Health Insurance and Pensions where customary
  • Dynamic Team Culture with a global company
  • Fulltime
Read More
Arrow Right

Senior AI Product Manager

Senior AI Product Manager – Nexus Shopping Agent. At Mirakl, we are building the...
Location
Location
France , Paris; Bordeaux
Salary
Salary:
Not provided
mirakl.com Logo
Mirakl
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of Product Management experience in AI products
  • Strong understanding of LLMs, RAG, prompting, structured reasoning, and retrieval systems
  • Experience with AI/ML teams on model evaluation, tuning, and reliability monitoring
  • Strong familiarity with e-commerce, shopping flows, ads systems, or after-sales workflows
  • Comfortable writing specs for AI logic, conversational flows, ranking rules, APIs, data pipelines, and front-end configuration tools
  • Analytical, experimentation-driven mindset
  • Fluent in English and French
Job Responsibility
Job Responsibility
  • Design end-to-end agent experiences across shopping, ads & after-sales
  • Own the roadmap & deliver AI-powered product features
  • Shape and improve the agentic architecture
  • Build dashboards, monitoring & feedback systems
  • Drive adoption & cross-functional collaboration
Read More
Arrow Right

Senior AI Product Manager

At Mirakl, we are building the future of commerce through an Agentic Commerce OS...
Location
Location
France , Paris; Bordeaux
Salary
Salary:
Not provided
mirakl.com Logo
Mirakl
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of Product Management experience in AI products
  • Strong understanding of LLMs, RAG, prompting, structured reasoning, and retrieval systems
  • Experience with AI/ML teams on model evaluation, tuning, and reliability monitoring
  • Strong familiarity with e-commerce, shopping flows, ads systems, or after-sales workflows
  • Comfortable writing specs for AI logic, conversational flows, ranking rules, APIs, data pipelines, and front-end configuration tools
  • Analytical, experimentation-driven mindset
  • Fluent in English and French
Job Responsibility
Job Responsibility
  • Design end-to-end agent experiences across shopping, ads & after-sales
  • Own the roadmap & deliver AI-powered product features
  • Shape and improve the agentic architecture
  • Build dashboards, monitoring & feedback systems
  • Drive adoption & cross-functional collaboration
Read More
Arrow Right