Principal AI Engineer Job at LogicMonitor (San Francisco)

Principal AI Engineer

LogicMonitor

Location:
United States , San Francisco

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

215160.00 - 268950.00 USD / Year

Save Job

Apply Position

Job Description:

LogicMonitor is the AI-first hybrid observability platform powering the next generation of digital infrastructure. Edwin AI is LogicMonitor’s enterprise AIOps platform, designed to help organizations detect, understand, and resolve incidents faster by reasoning across metrics, logs, topology, and change intelligence. As Edwin AI evolves toward AI-driven action and remediation, we are building enterprise-grade AI agents that integrate deeply with systems like ServiceNow to automate incident workflows with safety, reliability, and auditability. We are seeking a Principal AI Engineer to design and build AI agents that operate across observability data and ITSM systems. This role focuses on turning AI insights into safe, deterministic actions—building systems that can enrich incidents, orchestrate workflows, and reduce MTTR in large-scale, multi-tenant enterprise environments. This is a hands-on senior individual contributor role with high ownership and technical authority.

Job Responsibility:

Design and implement AI agents within Edwin AI that reason over metrics, logs, topology, and change data
Build and operate deep integrations with ServiceNow (Incidents, Problems, Changes, CMDB)
Translate observability insights and RCA outputs into ServiceNow-native actions and workflows
Implement event-driven, asynchronous pipelines for long-running and approval-based actions
Define and enforce guardrails, validation, auditability, and human-in-the-loop controls for AI actions
Ensure enterprise standards for security, reliability, and tenant isolation
Partner with Product, Platform, and Observability teams to shape Edwin AI agent capabilities
Provide technical leadership and mentorship on agentic AI system design

Requirements:

8+ years of backend or platform engineering experience
Proven experience building distributed, microservices-based systems at scale
Hands-on experience integrating with ServiceNow or enterprise ITSM platforms
Production experience building AI agents or LLM-powered systems that invoke tools or workflows
Experience with AIOps, observability platforms, or cloud management systems
Experience building AI systems that trigger real operational actions
Familiarity with Model Context Protocol (MCP) or equivalent agent-tool architectures
Strong proficiency in Java and/or Python
Experience with Kubernetes, containers, and cloud-native systems
Deep understanding of async execution, retries, idempotency, and failure handling
Experience designing audit-safe, compliance-friendly enterprise automation

What we offer:

Comprehensive health, dental and vision coverage
generous parental leave policies
access to our Employee Assistance Program and various Wellness programs
a 401K with company matching
a Lifestyle Spending Account
unlimited vacation policy

Additional Information:

Job Posted:
January 12, 2026

Expiration:
March 05, 2026

Employment Type:

Fulltime

Work Type:

Hybrid work

LogicMonitor - All Job Offers

Job Link Share: