CrawlJobs Logo
Microsoft Corporation Logo Microsoft Corporation · IT - Software Development

Principal Product Manager - Human Data

United States, Multiple Locations 139900.00 - 274800.00 USD / Year · Job Posted March 19, 2026
Apply Position
Job Link Share

Job Description

As a Principal Product Manager, you will lead the human-in-the-loop evaluation program that drives the quality and experience of M365 Copilot Chat for Business—ensuring M365 Copilot is the best AI assistant for work, grounded in M365 data. You'll build and scale the people, processes, and evaluation pipelines that turn expert human judgment into reliable quality signals that directly shape the product.

Job Responsibility

  • Define what great looks like for human data—bringing your own knowledge and perspective on the best approaches to collecting, interpreting, and applying human feedback across product decisions
  • Lead the human-in-the-loop evaluation program—designing evaluation frameworks, scorecards, and quality benchmarks that measure and continuously raise the bar on M365 Copilot's response quality, helpfulness, and user experience
  • Build and manage evaluation workforce operations, including vendor partnerships, annotator onboarding, qualification, training, and continuous performance management
  • Partner with data scientists and engineers to scope evaluation needs, define task instructions, calibrate annotators, and ensure evaluation data is reliable and repeatable

Requirements

  • Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
  • 3+ years of human data or human-in-the-loop evaluation experience
  • Experience at an AI research organization or AI data services provider
  • 4+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework)
  • 6+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn)
  • Experience building and managing workforce programs, including vendor partnerships and annotation operations at scale
  • Proficiency in evaluation pipeline design, annotation frameworks, and quality governance
  • Track record of translating human feedback and evaluation signals into measurable product impact
Microsoft Corporation - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Principal Product Manager - Human Data

8 matching positions

Principal Product Manager

Commercial Engineering & AI (CEAI) partners closely with stakeholders to acceler...
Location
Location
United States , Redmond
Salary
Salary:
142800.00 - 274800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience.
  • Experience defining product strategy, customer scenarios, requirements, roadmaps, and success metrics for technical products or platform experiences.
  • Experience working with AI, automation, agentic workflows, CRM/seller tools, enterprise software, or workflow productivity products.
  • Experience partnering cross-functionally with engineering, design, data science, research, business, and field stakeholders to deliver product direction and measurable outcomes.
  • Experience using customer feedback, telemetry, experimentation, or market insights to evaluate product opportunities and make prioritization decisions.
  • Experience incubating 0-to-1 products or new product capabilities from concept through prototype, validation, and scaled roadmap adoption.
  • Experience with LLM-based products, AI agents, prompt/evaluation workflows, human-in-the-loop systems, responsible AI considerations, or modern AI product development practices.
  • Experience in sales technology, CRM, commercial operations, seller productivity, revenue workflows, partner ecosystems, or customer engagement platforms.
  • Demonstrated ability to operate in ambiguous environments, simplify complex problem spaces, and make clear recommendations using evidence and product judgment.
  • Written and verbal communication skills, including the ability to create concise narratives, product briefs, executive updates, and decision documents.
Job Responsibility
Job Responsibility
  • Define product vision, strategy, and experimentation plans for autonomous selling scenarios, AI agents, and seller workflow automation.
  • Lead discovery for future-facing product opportunities, including customer and seller research, competitive analysis, prototype feedback, telemetry review, and business impact assessment.
  • Partner closely with engineering, design, data science, research, and business stakeholders to create prototypes, evaluate technical feasibility, and iterate quickly based on evidence.
  • Translate ambiguous opportunities into crisp product requirements, PRDs, success metrics, learning agendas, and decision frameworks that guide investment and prioritization.
  • Drive experiments from concept through review, including hypothesis definition, experience design, evaluation criteria, risk assessment, demo/readout, and recommendations to scale, iterate, or stop work.
  • Build a clear path for validated concepts to transition into the broader SPAI product roadmap, including alignment with production owners, engineering plans, telemetry needs, and stakeholder communications.
  • Use data, experimentation, customer feedback, and business outcomes to determine whether a capability should be promoted, reworked, or closed.
  • Collaborate across Microsoft commercial, product, engineering, and field teams to ensure autonomous selling investments are grounded in real seller workflows and customer value.
  • Communicate clearly through written narratives, product briefs, executive-ready updates, and decision documents that create alignment without unnecessary process overhead.
  • Model Microsoft's culture and values, contributing to an inclusive, high-accountability team environment that encourages learning, responsible AI practices, and thoughtful innovation.
  • Fulltime
Read More
Arrow Right

Principal Product Manager

Microsoft AI Monetization is a customer-first organization dedicated to building...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 6+ years experience working in product or service design
  • OR Bachelor's Degree in Industrial Design, Product Design, Human Computer Interaction, User Experience, Interaction Design, or related field AND 8+ years experience working in product or service design
  • OR equivalent experience (e.g., demonstrated experience working in product or service design or using design thinking to solve problems)
  • A portfolio or work samples that demonstrate experience designing world class consumer products from vision through launch
  • Proven track record of setting design vision for innovative products that require bold thinking and risk taking
  • Ability to break large bodies of work into a phased execution plan, and lead a cross-functional team from idea to polished implementation
  • Demonstrate high craft in design work
  • must value craft and translate vision into well-polished design prototypes
  • Experience that demonstrates proven ability to advocate for design thinking, effectively influence design outcomes, discover problems and tackle them in fast-paced and ambiguous environments
  • Experience with diving into research and leveraging both qualitative and quantitative signals for design impact
Job Responsibility
Job Responsibility
  • Design world-class AI-powered experiences that delight advertisers, sellers, and marketers, bringing clarity, trust, and joy to complex workflows while driving positive impact for users and society
  • Own the design vision for a product area, deeply understanding user needs and behaviors to craft intuitive, elegant solutions that simplify CRM and marketing journeys
  • Lead end-to-end design execution, from concept to launch, ensuring every detail reflects craft, quality, and a human-centered approach
  • Navigate ambiguity with creativity, finding innovative paths to deliver impactful experiences quickly and iteratively, even when roadblocks arise
  • Set the bar for design excellence, championing usability, accessibility, and visual polish while inspiring the team to push boundaries
  • Thrive in a fast-paced, design-driven environment, where experimentation and bold ideas shape the future of marketing technology
  • Leverage data and research to inform design decisions, partnering with PMs and researchers to define success metrics, run experiments, and iterate based on insights
  • Collaborate across disciplines: engineering, product, sales, and other designers, to create cohesive, consumer-grade experiences that feel effortless and empowering
  • Explore future-state experiences powered by AI, including conversational interfaces, predictive insights, and personalized workflows that transform how businesses connect and grow
  • Fulltime
Read More
Arrow Right

Principal Product Manager, AI

We are looking for a Lead Product Manager – AI/ML to own the strategy and roadma...
Location
Location
United States , Boston
Salary
Salary:
174000.00 - 256000.00 USD / Year
simplisafe.com Logo
SimpliSafe
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of product management experience
  • At least 3 years shipping AI or ML-driven features at consumer scale
  • Proven ability to define and champion a multi-year technical product vision
  • Sufficient technical depth to work credibly with AI/ML engineers
  • Experience defining quality and success criteria for systems where errors carry real consequences
  • Strong written and verbal communication
  • Comfort operating in ambiguity with incomplete data
  • Experience acting as a strategic peer in a matrixed organization
Job Responsibility
Job Responsibility
  • Define the multi-year strategy and roadmap for SimpliSafe’s AI/ML capabilities layer
  • Translate complex product needs into crisp, actionable requirements for the AI/ML engineering team
  • Establish the quality bar for model performance in production
  • Own platform decisions including model evaluation frameworks, data pipeline architecture, and trade-offs
  • Manage production model health
  • Build and maintain alignment across product, engineering, data science, design, and monitoring operations
  • Drive the strategy for AI safety, governance, and compliance
  • Partner with the Monitoring Product and Operations teams to define AI capability requirements
  • Drive measurable reduction in false alarm rates and response latency
  • Represent AI capability constraints and opportunities clearly
What we offer
What we offer
  • A mission- and values-driven culture and a safe, inclusive environment
  • A comprehensive total rewards package
  • Free SimpliSafe system and professional monitoring for your home
  • Employee Resource Groups (ERGs)
  • Participation in our annual bonus program, equity, and other forms of compensation
  • A full range of medical, retirement, and lifestyle benefits
  • Fulltime
Read More
Arrow Right

Principal Product Manager

Microsoft Teams is used by over 300 million people worldwide for meetings, chat,...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 8+ years experience in product/service/program management or software development
  • 8+ years of product management experience, with at least 3 years in search, information retrieval, or a related ML/AI-powered product area
  • Deep, hands-on experience with evaluation frameworks for search or AI systems
  • Solid understanding of IR fundamentals: ranking, relevance, query understanding, recall/precision trade-offs
  • Experience with LLM-era eval challenges: hallucination detection, groundedness, response quality, and human eval design for generative outputs
  • Demonstrated ability to work cross-functionally with applied scientists, ML engineers, and data teams
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Own Search Quality & Evals Strategy: Define the evaluation framework for Teams Search
  • Lead the design of human evaluation pipelines
  • Build and maintain an offline eval harness
  • Partner with Applied Science and engineering to translate eval signals into roadmap priorities and quality SLAs
  • Drive AI-Powered Search Features: Define product requirements for semantic search, hybrid retrieval, re-ranking, and Copilot-integrated search experiences in Teams
  • Own the PM relationship with the relevance science team
  • Work with the Copilot/AI team to integrate retrieval-augmented generation (RAG) patterns into Teams Search
  • Define Metrics and Success: Own the success metrics hierarchy for Teams Search
  • Instrument A/B experiments end-to-end
  • Build a culture of evidence-based decision making
  • Fulltime
Read More
Arrow Right

Sr. / Principal Product Manager - AI

Shape the future of trust in the age of AI. At Oscilar, we're building the most ...
Location
Location
Salary
Salary:
Not provided
oscilar.com Logo
Oscilar
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of product management experience, with at least 3 years in B2B SaaS platforms
  • Proven track record building workflow automation, orchestration, or integration platforms
  • Experience with AI/ML products, particularly in production environments
  • Background in fintech, risk management, or financial services preferred
  • Demonstrated success launching technical products for non-technical users
  • Deep understanding of workflow orchestration patterns, event-driven architectures, and state machines
  • Experience with visual programming concepts and node-based editors (Node-RED, Retool Workflows, Pipedream)
  • Knowledge of AI/ML concepts including LLMs, embeddings, vector databases, RAG, and agent architectures
  • Understanding of real-time systems, message queuing (Kafka, RabbitMQ), and streaming data processing
  • Familiarity with API design, webhooks, OAuth flows, and integration patterns
Job Responsibility
Job Responsibility
  • Define the product vision for an AI-first risk orchestration platform that revolutionizes how FIs manage risk workflows
  • Develop the roadmap that balances no-code accessibility with sophisticated AI capabilities
  • Research and analyze competitive landscape (n8n, Zapier, Temporal) to identify differentiation opportunities
  • Create the strategy for transitioning from traditional rule engines to AI-orchestrated decisioning
  • Define success metrics and KPIs that demonstrate value across fraud, credit, AML, and onboarding use cases
  • Agentic Workflow Components: Design autonomous AI nodes
  • LLM Integration Framework: Build a flexible system supporting multiple LLM providers
  • Intelligent Orchestration: Create smart workflow features
  • Human-in-the-Loop AI: Design collaborative features
  • Node Library Development: Design 200+ pre-built risk nodes
What we offer
What we offer
  • Competitive salary and equity packages, including a 401k plan
  • Remote-first culture — work from anywhere
  • 100% Employer covered comprehensive health, dental, and vision insurance with a top tier plan for you and your dependents (US)
  • Unlimited PTO policy
  • AI First company
  • both Co-Founders are engineers at heart
  • and over 50% of the company is Engineering and Product
  • Family-Friendly environment
  • Regular team events and offsites
  • Unparalleled learning and professional development opportunities
  • Fulltime
Read More
Arrow Right

Principal Product Manager Lead

Microsoft AI (MAI) is a customer‑first organization building AI‑powered products...
Location
Location
United States , Redmond
Salary
Salary:
163000.00 - 296400.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 10+ years experience in product/service/program management or software development OR equivalent experience
  • 3+ years people management experience
  • Bachelor's Degree AND 15+ years experience in product/service/program management or software development OR equivalent experience
  • 5+ years people management experience
  • 8+ years experience disrupting a market for a product, feature, or experience
  • 6+ years experience taking a product, feature, or experience to market
  • 8+ years experience improving product metrics for a product, feature, or experience in a market
  • Past experience with Evaluation Metrics and Platform
  • Vendor contract and negotiation experience
  • Good Understanding of LLM, Big Data, Machine Learning, and Data Mining
Job Responsibility
Job Responsibility
  • Define the multi‑year product vision and strategy for MAI’s Metrics Evaluation, Agentic Automation, and Testing Platforms
  • Establish the long‑term roadmap for evaluation frameworks, agentic workflows, and human‑in‑the‑loop systems that ensure quality, safety, and performance of LLM‑powered products
  • Define success criteria, OKRs, and KPIs across multiple product surfaces and organizational boundaries
  • Lead cross‑team execution for complex, multi‑dependency initiatives spanning Copilot, Bing, Edge, and MSN
  • Drive platform architecture decisions in partnership with engineering and research teams
  • Partner with Legal, Privacy, Security, HR, and Compliance teams to ensure platforms are compliant‑by‑design
  • Collaborate with external partners and vendors
  • negotiate contracts and manage financial relationships
  • Define and evangelize processes for developers and researchers to improve labeling workflows, model‑as‑a‑judge systems, and agentic evaluation pipelines
  • Research, design, and operationalize new evaluation techniques and metrics that improve the quality and reliability of LLM and ML‑based products
  • Fulltime
Read More
Arrow Right

Principal Product Manager - Ecosystem

We’re a fast-scaling legal tech SaaS company. Our platform is deeply embedded in...
Location
Location
United Kingdom , London; Edinburgh
Salary
Salary:
Not provided
opus2.com Logo
Opus 2
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7–10+ years of product management experience in SaaS or platform environments
  • Proven experience leading complex, cross-product or ecosystem-level initiatives
  • Strong understanding of APIs, integrations, and extensible platform design, including evolving AI-enabled interaction models
  • Experience working with both technical and non-technical users, translating needs across a broad capability spectrum
  • Comfort operating in ambiguity where technology, workflows, and customer expectations are rapidly changing
  • Familiarity with AI-enabled solutions and their implications for platform design, developer experience, and product strategy
  • Deep familiarity with legal or professional services product lifecycles, strong ability to translate compliance/security needs into product features used by legal teams
Job Responsibility
Job Responsibility
  • Identify whitespace opportunities across the broader ecosystem
  • Lead deep discovery with customers, partners, and internal teams to understand how data enters, exits, and interoperates with the platform across varying technical maturity levels
  • Translate insights into clear strategic recommendations and validated ecosystem initiatives
  • Define the long-term vision for the platform ecosystem, including APIs, integrations, data exchange, and extensibility
  • Connect the dots between traditional API design, AI-powered integrations, and emerging paradigms such as agent-driven or autonomous tool interactions
  • Identify where extending workflows beyond the platform create meaningful customer and business value and where it does not
  • Ensure ecosystem investments align with company strategy, platform principles, and client trust requirements
  • Own the end-to-end strategy, roadmap, and delivery of ecosystem capabilities, including APIs, integrations, data ingress and egress, and platform interoperability
  • Define and sequence initiatives that balance foundational platform improvements with experimentation, innovation pilots, and strategic partnerships
  • Lead complex, cross-team initiatives from discovery through launch and iteration, managing dependencies across Product, Engineering, Design, AI/ML, Architecture, and GTM
What we offer
What we offer
  • Contributory pension plan
  • 26 days annual holidays, flexible working, and length of service entitlement
  • Health Insurance
  • Enhanced Maternity and Paternity
  • Employee Assistance Programme
  • Electric Vehicle Salary Sacrifice
  • Cycle to Work Scheme
  • Calm and Mindfulness sessions
  • A day of leave to volunteer for charity or dependent cover
  • Accessible and modern office space and regular company social events
  • Fulltime
Read More
Arrow Right

Principal Product Manager

We’re building next generation AI-powered skilling experiences that help people ...
Location
Location
United States , Multiple Locations
Salary
Salary:
139900.00 - 274800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 8+ years experience in product/service/program management or software development OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check
  • Bachelor's Degree in Computer Science, Engineering, Human Computer Interaction, Data/AI, or related technical field or equivalent practical experience AND 12+ years of product leadership with enterprise SaaS/platform experience OR equivalent experience
  • 6+ years experience improving product metrics for a product, feature, or experience in a market
  • 6+ years experience disrupting a market for a product, feature, or experience
  • 6+ years of experience end to end product management experience shipping largescale, cross functional products or platforms
  • 6+ years of experience leading product teams building consumer products
  • 6+ years of experience in machine learning and/or AI
  • 4+ years experience taking a product, feature, or experience to market
Job Responsibility
Job Responsibility
  • Set the North Star – Define the vision and strategy for AI skilling across learner, manager, and enterprise scenarios
  • translate it into clear success metrics
  • Turn Insight into Product – Use qualitative and quantitative research to uncover unmet needs and convert them into crisp scenarios, roadmaps, and MVPs
  • Lead the Portfolio – Prioritize data informed investments
  • make tradeoffs across content, platform, telemetry, cost/latency, privacy, and compliance
  • Build AI First Experiences – Partner with Design and Engineering to ship LLM powered features (recommendations, coach, pathways, assessment) that are usable, accessible, and trustworthy
  • Raise the Bar on Quality – Establish measurement for engagement, learning outcomes, credential attainment, reliability, and performance
  • iterate based on evidence
  • Launch and Grow – Collaborate with Product Marketing and field teams to craft narratives, land launches (e.g., major events), and drive adoption and retention
  • Win with Partners – Work with internal/external partners (e.g., content providers, platform teams) to expand catalog depth, credentials, and enterprise integrations
  • Fulltime
Read More
Arrow Right