Principal Product Manager - Observability & Evaluations Job at Microsoft Corporation (New York)

AI Product Manager

We’re scaling AI and machine learning across our products, devices, and operatio...

Location

United States , Boston

Salary:

121300.00 - 177900.00 USD / Year

SimpliSafe

Expiration Date

Until further notice

Requirements

8+ years of product management experience, including significant ownership of AI/ML or data-intensive products
Clear track record of shipping production ML systems (not just integrating third-party AI APIs), in close partnership with data science, ML engineering, and MLOps
Principal-level impact: leading cross-team initiatives, shaping strategy, and influencing senior stakeholders
Strong understanding of core ML concepts and lifecycle: data, labeling, training/validation, evaluation metrics, deployment, monitoring, and retraining
ML experience with at least one of following: computer vision or sensor data, LLM-powered applications (prompting, RAG, fine-tuning, evaluation) and/or hardware or edge products (e.g., on-device models, connectivity/latency trade-offs)
Familiarity with modern ML infrastructure (cloud platforms, model serving, CI/CD for ML, monitoring/alerting)
Comfortable going deep into data, metrics, and model behavior—not just the UX layer
Excellent communicator who can make complex AI topics clear to diverse audiences
Strong alignment with our values: customer-obsessed, low ego, highly collaborative, comfortable with ambiguity, and biased toward learning and iteration.

Job Responsibility

Define and communicate the multi-year roadmap for key AI/ML capabilities across SimpliSafe
Identify and prioritize AI opportunities where models and data can materially improve safety, customer experience, or efficiency—on both devices and cloud services
Make build-vs-buy decisions for AI capabilities in partnership with data science and engineering
Partner with data scientists, ML engineers, and MLOps to design and deliver end-to-end ML solutions—from problem framing through data, training, evaluation, deployment, and monitoring
Work with hardware and embedded teams to shape edge AI/ML experiences (e.g., on-device detection, low-latency decisions, bandwidth-aware designs)
Define model-level requirements (metrics, latency, cost, guardrails) and connect them to business outcomes (e.g., false alarm reduction, detection accuracy, handle time, CSAT)
Translate product needs into requirements for ML platform capabilities (model serving, observability, experiment tracking, human-in-the-loop tools)
Lead product direction for LLM and multimodal use cases (e.g., text, vision, sensor data)
Decide when to use prompt engineering, RAG, fine-tuning, or traditional ML—and how to evaluate quality, safety, and hallucinations
Design workflows that incorporate human review and escalation where needed

What we offer

A mission- and values-driven culture and a safe, inclusive environment where you can build, grow, and thrive
A comprehensive total rewards package that supports your wellness and provides security for SimpliSafers and their families
Free SimpliSafe system and professional monitoring for your home
Employee Resource Groups (ERGs) that bring people together, give opportunities to network, mentor and develop, and advocate for change
Participation in our annual bonus program, equity, and other forms of compensation, in addition to a full range of medical, retirement, and lifestyle benefits.

Fulltime

Principal Product Manager, AI

LogicMonitor® is the AI-first hybrid observability platform powering the next ge...

Location

India , Pune

Salary:

Not provided

LogicMonitor

Expiration Date

Until further notice

Requirements

8 to 9 years of product management experience, ideally in B2B software
Strong hands on fluency with AI powered products and how modern AI systems work
Proven experience owning products end to end in ambiguous environments
Strong customer instincts and comfort being customer facing
Ability to think strategically while executing tactically
Clear communicator who can align teams without relying on authority
Bias for action and comfort with scrappy execution

Job Responsibility

End to end ownership of major Edwin AI product areas from discovery to delivery to adoption
Deep customer engagement to uncover real problems and validate solutions
Clear product strategy and roadmap grounded in customer value, data, and business outcomes
Hands on collaboration with engineering, design, GTM, and applied AI teams
Product decisions across AI capabilities including agents, workflows, and intelligent experiences
Success metrics like adoption, usage, customer impact, and business outcomes
Spend real time with customers through calls, demos, feedback sessions, and follow ups
Translate messy customer problems into crisp product bets and shipped features
Work closely with engineers on scope, tradeoffs, and delivery details
Get hands on with AI concepts like prompting, evaluations, agent behavior, and failure modes

Architectural Project Designer - Interior Specialist

Location

United States

Salary:

Not provided

Architectural Design & Construction, Inc.

Expiration Date

Until further notice

Requirements

Learn to create realistic project schedule: estimate time frame for design development, structural design, construction documents
Manage client expectations: schedule, budget, and project scope
Establish due dates for all finish selections for client
Educate client concerning the process and importance of obtaining selections on schedule
Obtain selections on schedule and before construction starts
Record all selections
Obtain client approval in writing before construction starts
Provide client with regular project status reports throughout the design phase
Coordinate with consultants to meet project requirements
Meet project schedule and budget for design services to maintain profitability

Job Responsibility

Reports to the Principal/Construction Project Manager
Properly communicate information regarding selections to the Principal/Construction Project Manager, Project Architect, Project Manager, and client as appropriate and in a timely manner
Prior to all client meetings, share all information that will be presented to a client in the meeting with the Project Team members
Promote, discuss and offer ADC’s Interior Design services when clients have not selected an Interior Designer for their project
Maintain and enhance client relationship with ADC
Walk client through the selections process
Ensure that they understand the phases of construction
Notify suppliers when ADC clients will visit them, if possible
Provide Construction Manager with selections prior to client sign-off on Final Budget Estimate
Assist the Construction Manager putting the selections out to bid with trade contractors and suppliers

Principal Product Manager - AI Governance (Azure AI Foundry)

AI is transforming how organizations operate, and with it comes a new frontier o...

Location

United States , Redmond

Salary:

139900.00 - 274800.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor’s Degree AND 8+ years in product management, program management, or technical leadership roles OR equivalent experience
Ability to meet Microsoft, customer and/or government security screening requirements
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Deep familiarity with AI/ML systems, model governance, or enterprise compliance frameworks
Proven experience driving cross-functional initiatives with engineering, security, and policy teams
Excellent communication skills with demonstrated ability to influence across organizational boundaries
10+ years of product management or applied AI experience, ideally in enterprise cloud or responsible AI domains
Knowledge of AI governance frameworks (EU AI Act, NIST AI RMF, ISO 42001, SOC 2, etc.)
Experience building or integrating security, compliance, or observability products
Familiarity with agentic AI systems and associated risk classes (e.g., sensitive data leakage, prohibited actions, task drift, jailbreaks)

Job Responsibility

Lead the AI Governance product area within Azure AI Foundry — defining the long-term vision, strategy, and roadmap for policy management, compliance automation, and regulatory readiness
Design and deliver core governance experiences, including agent-level policies, data sensitivity signals, prohibited action controls, and AI system compliance dashboards
Integrate Foundry governance with Microsoft’s broader security and compliance stack — Entra (identity and A2A policies), Purview (data classification and DLP), and Defender (threat insights)
Translate emerging AI regulations (EU AI Act, ISO 42001, NIST AI RMF) into actionable platform capabilities and customer experiences
Partner with Responsible AI researchers and engineering teams to operationalize ethical principles into measurable safeguards and evaluators
Collaborate with customers, industry bodies, and policymakers to help shape standards for trustworthy AI deployment
Establish and track success metrics (governance coverage, compliance posture, customer adoption), ensuring measurable impact and clarity across engineering and partner orgs
Represent the product in executive and customer forums, evangelizing Microsoft’s approach to responsible, governed AI systems

Fulltime

Principal Software Consultant - AI/ML Engineer

As an ML Team Lead, you will be responsible for leading the technical direction ...

Location

Pakistan , Lahore, Karachi, Islamabad

Salary:

Not provided

10Pearls

Expiration Date

Until further notice

Requirements

Bachelor's or Master's degree in computer science, Artificial Intelligence, Data Science, Software Engineering, or a related field
7+ years of professional software engineering experience with at least 5 years of hands-on experience building and deploying ML systems into production
Prior experience as a Tech Lead, Staff Engineer, or hands-on lead for AI/ML engineering teams
Strong expertise in classical machine learning domains such as forecasting, ranking, classification, and optimization
Hands-on experience building modern LLM and agentic AI systems including RAG pipelines, tool-using agents, multi-step workflows, and evaluation systems
Strong proficiency in Python and backend system development
Experience with ML frameworks such as PyTorch or TensorFlow
Strong understanding of scalable distributed systems, APIs, system integration, architecture design, and production engineering practices
Experience operating ML services at scale, including SLO management, monitoring, on-call practices, and incident response
Experience working with Kubernetes-based deployments, CI/CD pipelines, and modern cloud-native engineering practices

Job Responsibility

Lead the technical direction for the team’s ML and LLM systems, including architecture patterns, platform choices, evaluation frameworks, and engineering standards
Stay hands-on by designing and implementing complex ML and agentic AI systems, writing production-grade code, and leading through technical execution
Design, develop, and deploy scalable ML and LLM-powered applications and services in production environments
Build and optimize AI-powered solutions such as RAG systems, multi-step agents, AI assistants, chatbots, forecasting systems, ranking models, classification models, and optimization systems
Drive architecture and design reviews to ensure scalability, reliability, security, and maintainability of AI/ML systems
Own the technical roadmap for ML/LLM initiatives and translate business objectives into execution plans and scalable solutions
Collaborate closely with Product Managers, Engineers, Data Engineers, MLOps Engineers, QA Engineers, and cross-functional stakeholders to deliver business-aligned AI solutions
Establish engineering best practices for prompt engineering, model evaluation, regression testing, observability, and production readiness
Define and implement quality standards, evaluation suites, acceptance metrics, and regression plans for all AI/ML features
Ensure high availability, scalability, and resilience of tier-1 ML services through SLOs, monitoring, incident response, failover strategies, circuit breakers, and multi-zone deployments

Fulltime

Platform Lead E2E Architect

The Platform Lead Architect is a senior technical authority responsible for defi...

Location

United Kingdom , Newbury

Salary:

Not provided

Vodafone

Expiration Date

Until further notice

Requirements

MSc or equivalent in Computer Science, Engineering, Telecommunications, or related discipline
10+ years of experience in architecture roles, with significant exposure to cloud, APIs, event-driven systems, microservices, and modern DevOps/SRE practices
Demonstrated architectural leadership within large, multi-squad engineering organisations
Track record delivering platforms integrated with telecom BSS/OSS components
Business-fluent English
other European languages beneficial.

Job Responsibility

End-to-end ownership of the platform architecture, covering digital channels, backend services, orchestration, integration patterns, data models, network interfaces, and operational architecture
Define and evolve the long-term technology strategy and architectural roadmap, aligning to business goals, multi-market needs, and platform-scalability requirements
Govern architectural standards, guardrails, and principles, ensuring consistency, reusability, and coherence across all value streams and platform teams
Lead high-impact architectural decision-making, balancing innovation with standardisation and managing trade-offs across performance, security, cost, and delivery speed
Partner with Product Owners, Delivery Leads, Technical Principals, and Network Architects to ensure features are technically feasible, sequenced effectively, and built on solid architectural foundations
Drive platform evolution - including Identity, Product Catalogue, Charging, Billing, SIM/eSIM lifecycle management, Mediation, Observability, and Event Infrastructure
Champion AI-enabled automation, architectural modernisation, and continuous simplification of technical landscapes
Represent Digital & IT in cross-functional governance, senior-level design forums, partner engagements, and roadmap planning
Line Management - Experience guiding senior technical professionals, fostering a high-performance engineering and architecture culture, and providing coaching, feedback, and professional development across multi-disciplinary teams
Leadership - Ability to drive collaboration across organisations, align to a common goal and ensure project delivery against obstacles

What we offer

Perks and benefits

Fulltime

Principal Software Engineer

Are you looking for an opportunity to work with the latest Azure offerings and p...

Location

India , Bangalore

Salary:

Not provided

Microsoft Corporation

Expiration Date

Until further notice

Requirements

10–12+ years of experience in software engineering, with significant experience building scalable backend or distributed systems
Strong programming expertise in one or more languages such as Python, Go, Java, or C#, with experience designing production-grade services and APIs
Experience building AI-powered applications, including integrating LLMs, implementing agent or Copilot workflows, and orchestrating multi-step AI interactions
Hands-on experience with LLM application frameworks and orchestration tools such as Semantic Kernel, LangChain, or similar agent frameworks
Familiarity with retrieval-augmented generation (RAG) architectures, vector databases, embeddings, and semantic search systems
Experience evaluating and improving model performance through prompt design, evaluation frameworks, fine-tuning, or feedback loops
Solid understanding of distributed systems concepts including scalability, reliability, observability, caching, and asynchronous processing
Experience deploying and operating AI workloads in cloud environments (preferably Azure), including containerized services and GPU-enabled infrastructure
Understanding of Responsible AI practices, including model governance, safety, privacy, and evaluation of AI behaviour in production systems
Ability to work across product, research, and engineering teams to translate product scenarios into scalable AI system architectures

Job Responsibility

Design, build, and operate scalable AI systems that power intelligent product experiences, including Copilot and agent-driven workflows
Architect and implement backend services that support multi-step AI interactions, including orchestration pipelines, context management, memory/state persistence, and tool execution
Integrate large language models (LLMs), APIs, and internal services to enable context-aware, human-in-the-loop experiences across customer scenarios
Build and maintain data and inference pipelines that support model training, fine-tuning, evaluation, and real-time inference across diverse data sources
Evaluate, benchmark, and tune AI/ML models (LLMs and traditional models) to meet product requirements for accuracy, latency, reliability, and safety
Implement robust retrieval, grounding, and knowledge integration mechanisms (e.g., RAG systems, semantic indexing, vector search) to power intelligent applications
Collaborate with product managers, software engineers, and researchers to translate product vision into production-ready AI capabilities and measurable outcomes
Ensure reliability, observability, and governance of AI systems, including monitoring model performance, data quality, and responsible AI practices
Build reusable platforms, APIs, and tools that enable teams to rapidly develop AI-powered features and self-service intelligent applications

Fulltime

Principal AI Engineer

We are looking for a Principal AI Engineer to lead the design and deployment of ...

Location

United States

Salary:

200000.00 - 300000.00 USD / Year

Apollo.io

Expiration Date

Until further notice

Requirements

10+ years of software engineering experience
at least 3 years in applied LLM or agentic AI systems (2023–present)
proven success in deploying LLM-powered products used by real users at scale
deep backend & systems engineering expertise with Python, distributed systems, and scalable APIs
familiarity with LangChain, LlamaIndex, or similar orchestration frameworks
experience with RAG pipelines, vector DBs, embedding models, and semantic search tuning
experience managing performance across cloud providers (e.g., AWS Bedrock, OpenAI, Anthropic, etc.)
demonstrated experience building multi-step agents, planning workflows, chaining reasoning steps, and integrating APIs with agent memory/state
comfort with advanced prompting strategies, few-shot and chain-of-thought reasoning, and embedding retrieval setups
strong understanding of AI system evaluation, human ratings, A/B experimentation, and feedback loop pipelines

Job Responsibility

Architect and lead the development of multi-agent systems capable of long-horizon planning, reasoning, and API orchestration
build reusable agentic components that integrate deeply into sales and marketing processes
own and evolve our in-house platform for scalable, low-latency, and cost-efficient LLM and agent deployments
lead design of interfaces powered by natural language understanding and retrieval-augmented generation (RAG)
build embedding-based, intent-aware search and personalization systems tuned to business user needs
drive innovation in personalized outreach generation using context-aware generation pipelines
tune inference pipelines, caching layers, and model selection logic for high-scale, cost-aware performance
define and drive robust offline and online testing methodologies (A/B, sandboxing, human evals) across agents and LLM flows
architect human-in-the-loop systems and telemetry to improve accuracy, UX, and explainability over time

What we offer

equity
company bonus or sales commissions/bonuses
401(k) plan
at least 10 paid holidays per year
flex PTO
parental leave
employee assistance program
wellbeing benefits
global travel coverage
life/AD&D/STD/LTD insurance

Fulltime

Select Country

Principal Product Manager - Observability & Evaluations

Job Description

Job Responsibility

Requirements

Looking for more opportunities?