Principal Product Manager - Observability & Evaluations Job at Microsoft Corporation (New York)

AI Product Manager

We’re scaling AI and machine learning across our products, devices, and operatio...

Location

United States , Boston

Salary:

121300.00 - 177900.00 USD / Year

SimpliSafe

Expiration Date

Until further notice

Requirements

8+ years of product management experience, including significant ownership of AI/ML or data-intensive products
Clear track record of shipping production ML systems (not just integrating third-party AI APIs), in close partnership with data science, ML engineering, and MLOps
Principal-level impact: leading cross-team initiatives, shaping strategy, and influencing senior stakeholders
Strong understanding of core ML concepts and lifecycle: data, labeling, training/validation, evaluation metrics, deployment, monitoring, and retraining
ML experience with at least one of following: computer vision or sensor data, LLM-powered applications (prompting, RAG, fine-tuning, evaluation) and/or hardware or edge products (e.g., on-device models, connectivity/latency trade-offs)
Familiarity with modern ML infrastructure (cloud platforms, model serving, CI/CD for ML, monitoring/alerting)
Comfortable going deep into data, metrics, and model behavior—not just the UX layer
Excellent communicator who can make complex AI topics clear to diverse audiences
Strong alignment with our values: customer-obsessed, low ego, highly collaborative, comfortable with ambiguity, and biased toward learning and iteration.

Job Responsibility

Define and communicate the multi-year roadmap for key AI/ML capabilities across SimpliSafe
Identify and prioritize AI opportunities where models and data can materially improve safety, customer experience, or efficiency—on both devices and cloud services
Make build-vs-buy decisions for AI capabilities in partnership with data science and engineering
Partner with data scientists, ML engineers, and MLOps to design and deliver end-to-end ML solutions—from problem framing through data, training, evaluation, deployment, and monitoring
Work with hardware and embedded teams to shape edge AI/ML experiences (e.g., on-device detection, low-latency decisions, bandwidth-aware designs)
Define model-level requirements (metrics, latency, cost, guardrails) and connect them to business outcomes (e.g., false alarm reduction, detection accuracy, handle time, CSAT)
Translate product needs into requirements for ML platform capabilities (model serving, observability, experiment tracking, human-in-the-loop tools)
Lead product direction for LLM and multimodal use cases (e.g., text, vision, sensor data)
Decide when to use prompt engineering, RAG, fine-tuning, or traditional ML—and how to evaluate quality, safety, and hallucinations
Design workflows that incorporate human review and escalation where needed

What we offer

A mission- and values-driven culture and a safe, inclusive environment where you can build, grow, and thrive
A comprehensive total rewards package that supports your wellness and provides security for SimpliSafers and their families
Free SimpliSafe system and professional monitoring for your home
Employee Resource Groups (ERGs) that bring people together, give opportunities to network, mentor and develop, and advocate for change
Participation in our annual bonus program, equity, and other forms of compensation, in addition to a full range of medical, retirement, and lifestyle benefits.

Fulltime

New

Principal Product Manager, AI

LogicMonitor® is the AI-first hybrid observability platform powering the next ge...

Location

India , Pune

Salary:

Not provided

LogicMonitor

Expiration Date

Until further notice

Requirements

8 to 9 years of product management experience, ideally in B2B software
Strong hands on fluency with AI powered products and how modern AI systems work
Proven experience owning products end to end in ambiguous environments
Strong customer instincts and comfort being customer facing
Ability to think strategically while executing tactically
Clear communicator who can align teams without relying on authority
Bias for action and comfort with scrappy execution

Job Responsibility

End to end ownership of major Edwin AI product areas from discovery to delivery to adoption
Deep customer engagement to uncover real problems and validate solutions
Clear product strategy and roadmap grounded in customer value, data, and business outcomes
Hands on collaboration with engineering, design, GTM, and applied AI teams
Product decisions across AI capabilities including agents, workflows, and intelligent experiences
Success metrics like adoption, usage, customer impact, and business outcomes
Spend real time with customers through calls, demos, feedback sessions, and follow ups
Translate messy customer problems into crisp product bets and shipped features
Work closely with engineers on scope, tradeoffs, and delivery details
Get hands on with AI concepts like prompting, evaluations, agent behavior, and failure modes

Architectural Project Designer - Interior Specialist

Location

United States

Salary:

Not provided

Architectural Design & Construction, Inc.

Expiration Date

Until further notice

Requirements

Learn to create realistic project schedule: estimate time frame for design development, structural design, construction documents
Manage client expectations: schedule, budget, and project scope
Establish due dates for all finish selections for client
Educate client concerning the process and importance of obtaining selections on schedule
Obtain selections on schedule and before construction starts
Record all selections
Obtain client approval in writing before construction starts
Provide client with regular project status reports throughout the design phase
Coordinate with consultants to meet project requirements
Meet project schedule and budget for design services to maintain profitability

Job Responsibility

Reports to the Principal/Construction Project Manager
Properly communicate information regarding selections to the Principal/Construction Project Manager, Project Architect, Project Manager, and client as appropriate and in a timely manner
Prior to all client meetings, share all information that will be presented to a client in the meeting with the Project Team members
Promote, discuss and offer ADC’s Interior Design services when clients have not selected an Interior Designer for their project
Maintain and enhance client relationship with ADC
Walk client through the selections process
Ensure that they understand the phases of construction
Notify suppliers when ADC clients will visit them, if possible
Provide Construction Manager with selections prior to client sign-off on Final Budget Estimate
Assist the Construction Manager putting the selections out to bid with trade contractors and suppliers

Principal Product Manager - AI Governance (Azure AI Foundry)

AI is transforming how organizations operate, and with it comes a new frontier o...

Location

United States , Redmond

Salary:

139900.00 - 274800.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor’s Degree AND 8+ years in product management, program management, or technical leadership roles OR equivalent experience
Ability to meet Microsoft, customer and/or government security screening requirements
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Deep familiarity with AI/ML systems, model governance, or enterprise compliance frameworks
Proven experience driving cross-functional initiatives with engineering, security, and policy teams
Excellent communication skills with demonstrated ability to influence across organizational boundaries
10+ years of product management or applied AI experience, ideally in enterprise cloud or responsible AI domains
Knowledge of AI governance frameworks (EU AI Act, NIST AI RMF, ISO 42001, SOC 2, etc.)
Experience building or integrating security, compliance, or observability products
Familiarity with agentic AI systems and associated risk classes (e.g., sensitive data leakage, prohibited actions, task drift, jailbreaks)

Job Responsibility

Lead the AI Governance product area within Azure AI Foundry — defining the long-term vision, strategy, and roadmap for policy management, compliance automation, and regulatory readiness
Design and deliver core governance experiences, including agent-level policies, data sensitivity signals, prohibited action controls, and AI system compliance dashboards
Integrate Foundry governance with Microsoft’s broader security and compliance stack — Entra (identity and A2A policies), Purview (data classification and DLP), and Defender (threat insights)
Translate emerging AI regulations (EU AI Act, ISO 42001, NIST AI RMF) into actionable platform capabilities and customer experiences
Partner with Responsible AI researchers and engineering teams to operationalize ethical principles into measurable safeguards and evaluators
Collaborate with customers, industry bodies, and policymakers to help shape standards for trustworthy AI deployment
Establish and track success metrics (governance coverage, compliance posture, customer adoption), ensuring measurable impact and clarity across engineering and partner orgs
Represent the product in executive and customer forums, evangelizing Microsoft’s approach to responsible, governed AI systems

Fulltime

Principal AI Engineer

We are looking for a Principal AI Engineer to lead the design and deployment of ...

Location

United States

Salary:

200000.00 - 300000.00 USD / Year

Apollo.io

Expiration Date

Until further notice

Requirements

10+ years of software engineering experience
at least 3 years in applied LLM or agentic AI systems (2023–present)
proven success in deploying LLM-powered products used by real users at scale
deep backend & systems engineering expertise with Python, distributed systems, and scalable APIs
familiarity with LangChain, LlamaIndex, or similar orchestration frameworks
experience with RAG pipelines, vector DBs, embedding models, and semantic search tuning
experience managing performance across cloud providers (e.g., AWS Bedrock, OpenAI, Anthropic, etc.)
demonstrated experience building multi-step agents, planning workflows, chaining reasoning steps, and integrating APIs with agent memory/state
comfort with advanced prompting strategies, few-shot and chain-of-thought reasoning, and embedding retrieval setups
strong understanding of AI system evaluation, human ratings, A/B experimentation, and feedback loop pipelines

Job Responsibility

Architect and lead the development of multi-agent systems capable of long-horizon planning, reasoning, and API orchestration
build reusable agentic components that integrate deeply into sales and marketing processes
own and evolve our in-house platform for scalable, low-latency, and cost-efficient LLM and agent deployments
lead design of interfaces powered by natural language understanding and retrieval-augmented generation (RAG)
build embedding-based, intent-aware search and personalization systems tuned to business user needs
drive innovation in personalized outreach generation using context-aware generation pipelines
tune inference pipelines, caching layers, and model selection logic for high-scale, cost-aware performance
define and drive robust offline and online testing methodologies (A/B, sandboxing, human evals) across agents and LLM flows
architect human-in-the-loop systems and telemetry to improve accuracy, UX, and explainability over time

What we offer

equity
company bonus or sales commissions/bonuses
401(k) plan
at least 10 paid holidays per year
flex PTO
parental leave
employee assistance program
wellbeing benefits
global travel coverage
life/AD&D/STD/LTD insurance

Fulltime

Principal Software Engineer - Copilot Security

Copilot Security is at the core of Microsoft’s mission to deliver trusted, human...

Location

United States , Redmond, WA

Salary:

163000.00 - 296400.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, Go, or Python OR equivalent experience
8+ years in technical engineering roles building large-scale services
8+ years hands-on experience designing and operating security-critical or AI-powered systems at scale, including agentic AI, secure orchestration, or advanced threat defenses
Proven ability to design, build, and ship agentic AI features or frameworks
Ability to clearly explain complex systems and security concepts to technical and non-technical stakeholders and influence cross-org roadmaps
Experience building production agent systems using frameworks such as LangGraph, Amazon Strands SDK, or similar platforms
familiarity with agentic design patterns including tool calling, multi-agent coordination, and secure delegation patterns
Hands-on experience with distributed training frameworks (Ray, Slurm, HPC), containerization and orchestration technologies (Docker, Kubernetes) for ML model deployment, and ML lifecycle management in production environments
Experience designing evaluation frameworks for LLM-based applications and implementing observability for agent systems using tools such as Phoenix, MLFlow, LangFuse, or custom eval harnesses
understanding of AI safety evaluation methodologies including adversarial testing and red-teaming

Job Responsibility

Develop and ship agentic AI-powered security features that protect users from threats such as prompt injection, adversarial manipulation, and abuse of agentic workflows
Design and implement secure orchestration frameworks that enable Copilot to safely delegate, coordinate, and execute actions across devices, services, and platforms
Invent and apply new intelligent agents that leverage information flow analysis and apply common sense and judgement guardrails for security and privacy
Collaborate with product, engineering, security, privacy, and AI teams to drive adoption of agentic security patterns and best practices across Copilot and MAI
Monitor key metrics for agentic AI security and innovation, using data-driven insights to improve defenses and enablement
Align with central Microsoft security and AI roadmaps, landing platform capabilities in Copilot and MAI consumer scenarios
Document secure agentic AI patterns, ensuring they address novel risks, support safe delegation, and enable responsible orchestration of actions

Fulltime

Principal Software Engineer - Copilot Security

Copilot Security is at the core of Microsoft’s mission to deliver trusted, human...

Location

United States , Redmond

Salary:

139900.00 - 274800.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
OR equivalent experience
8+ years in technical engineering roles building large-scale services
6+ years hands-on experience designing and operating security-critical or AI-powered systems at scale, including agentic AI, secure orchestration, or advanced threat defenses
Proven ability to design, build, and ship agentic AI features or frameworks
Ability to clearly explain complex systems and security concepts to technical and non-technical stakeholders and influence cross-org roadmaps
Experience building production agent systems using frameworks such as LangGraph, Amazon Strands SDK, or similar platforms
familiarity with agentic design patterns including tool calling, multi-agent coordination, and secure delegation patterns
Hands-on experience with distributed training frameworks (Ray, Slurm, HPC), containerization and orchestration technologies (Docker, Kubernetes) for ML model deployment, and ML lifecycle management in production environments
Experience designing evaluation frameworks for LLM-based applications and implementing observability for agent systems using tools such as Phoenix, MLFlow, LangFuse, or custom eval harnesses

Job Responsibility

Develop and ship agentic AI-powered security features that proactively protect users from threats such as prompt injection, adversarial manipulation, and abuse of agentic workflows
Design and implement secure orchestration frameworks that enable Copilot to safely delegate, coordinate, and execute actions across devices, services, and platforms
Invent and apply new intelligent agents that leverage information flow analysis and apply common sense and judgement guardrails for security and privacy
Collaborate with product, engineering, security, privacy, and AI teams to drive adoption of agentic security patterns and best practices across Copilot and MAI
Establish and monitor key metrics for agentic AI security and innovation, using data-driven insights to continuously improve defenses and enablement
Align with central Microsoft security and AI roadmaps, influencing platform capabilities and landing them in Copilot and MAI consumer scenarios
Document and evangelize secure agentic AI patterns, ensuring they address novel risks, support safe delegation, and enable responsible orchestration of actions
Mentor engineers and foster a culture of secure innovation, balancing rapid development with rigorous protection for customers

Fulltime

Director, Global Indian Team Lead, Private Bank

Join Relationship Management, to build and maintain relationships with customers...

Location

Singapore , Singapore

Salary:

Not provided

Barclays

Expiration Date

Until further notice

Requirements

Management of relationship management initiatives, including oversight of colleagues and their performance, implementation of departmental goals and objectives, oversight of department efficiency and effectiveness
Management of client relationships to identify the clients financial goals, challenges, and risk tolerance to support the analysis of data obtained from various sources, including the investment portfolio and cash flow, to identify trends, insights, areas for improvement and additional services to support client needs
Research and understanding of the client's industry trends, regulatory landscape, and competitive environment to inform strategic recommendations
Design of customised solutions that address the client's specific needs and objectives, incorporating a range of products and services from the bank's portfolio
Communication of the value proposition of proposed solutions, justification of recommendations, and negotiation of terms that are beneficial for both the client and the bank
Provision of guidance to clients to support their financial decisions, offering expert investment advice, risk management and wealth management strategies support, and updates on market trends to ensure a positive and continuous relationship
Assessment of financial, legal, and operational risks associated with client relationships, and implementation of measures to minimise potential losses
Documentation of all client interactions, transactions, and agreements to ensure transparency and auditability, and communicate findings effectively to support product development, service offerings, and the overall bank strategy
Monitoring of client satisfaction, revenue generated, and other relevant metrics to evaluate the effectiveness of relationship management efforts
To manage a business function, providing significant input to function wide strategic initiatives

Job Responsibility

Establish and nurture profitable partnerships with both corporate and individual clients
Be the bridge between the bank's offerings and clients' needs, ensuring mutual benefit and long-term success

What we offer

Structured approach to hybrid working
Supportive and inclusive culture and environment
Commitment to providing a supportive and inclusive culture and environment
Office is carefully designed to optimise the well-being of our colleagues
Integrated live-work-play development, with shops, restaurants, gyms and an alfresco dining area

Fulltime

Principal Product Manager - Observability & Evaluations

Microsoft Corporation

Location:
United States , New York

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Additional Information:

Job Posted:
February 20, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Principal Product Manager - Observability & Evaluations

AI Product Manager

Principal Product Manager, AI

Architectural Project Designer - Interior Specialist

Principal Product Manager - AI Governance (Azure AI Foundry)

Principal AI Engineer

Principal Software Engineer - Copilot Security

Principal Software Engineer - Copilot Security

Director, Global Indian Team Lead, Private Bank

Principal Product Manager - Observability & Evaluations

Microsoft Corporation

Location:United States , New York

Category:IT - Software Development

Contract Type:Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Additional Information:

Job Posted:February 20, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Principal Product Manager - Observability & Evaluations

AI Product Manager

Principal Product Manager, AI

Architectural Project Designer - Interior Specialist

Principal Product Manager - AI Governance (Azure AI Foundry)

Principal AI Engineer

Principal Software Engineer - Copilot Security

Principal Software Engineer - Copilot Security

Director, Global Indian Team Lead, Private Bank

Location:
United States , New York

Category:
IT - Software Development

Contract Type:
Not provided

Job Posted:
February 20, 2026