CrawlJobs Logo
Uber Logo Uber · IT - Software Development

Program Lead: Product Operations - AI Observability

United States, Sunnyvale 162000.00 - 180000.00 USD / Year · Job Posted April 12, 2026
Apply Position
Job Link Share

Job Description

The AI Observability Program Leader will own the end-to-end strategy, design, and implementation of the frameworks used to monitor, understand, and improve Uber’s GenAI-powered agentic systems. This role sits within the Global Digital Experience team, the operational arm of Uber’s customer support tech organization, and is a critical driver of accuracy, safety, and reliability across Uber’s next-generation AI solutions. This leader will bridge the gap between raw AI logs and actionable product insights.

Job Responsibility

  • Architect Observability Frameworks: Own the strategy for understanding AI agentic reasoning, enabling deep analysis of step-by-step agent decision-making
  • Drive Autoeval Strategy: Design and roll out automated evaluation systems (LLM-as-a-judge) to provide a scalable, high-confidence "pulse" on AI performance across conversational and voice interfaces
  • Define Micrometrics: Develop granular signals within agentic activity—identifying latent failures, reasoning loops, or tool-calling inefficiencies—to drive product improvements
  • Lead Pre-Launch Simulation: Partner with Product & Engineering to build and maintain simulation environments that test AI agents against edge cases before deployment, and democratise these tools with Operations teams
  • Cross-Functional Technical Partnership: Act as the primary liaison between Product, Engineering, and Data Science to ensure observability tooling is integrated into the development lifecycle and directly informs release "Go/No-Go" decisions
  • Insight Synthesis: Package complex technical observability data into clear, actionable narratives for leadership, highlighting specific failure patterns and opportunities for CX improvement
  • Operational Excellence: Establish the standards and tooling for how AI performance is reported globally, ensuring consistency across different regions and support modalities.

Requirements

  • 5+ years of experience in Technical Program Management, Product Operations, AI Quality, or Observability
  • Bachelor’s degree in Engineering, Computer Science, Data Science, or a related technical field.

Nice to have

  • AI Literacy: Deep understanding of GenAI systems, including LLM orchestration, agentic workflows, and the nuances of reasoning chains (e.g., Chain of Thought)
  • Systems Thinking: Proven experience designing technical frameworks or evaluation pipelines (e.g., autoevals, RAG evaluation, or model benchmarking)
  • Analytical Rigor: Ability to define and track complex technical metrics (micrometrics) and correlate them with high-level business KPIs
  • Influence without Authority: Demonstrated ability to drive complex initiatives in an IC capacity by building strong partnerships with Engineering and Product teams
  • Advanced AI Expertise: Experience with "LLM-as-a-judge" frameworks, prompt engineering for evaluations, and fine-tuning feedback loops
  • Simulation & Testing: Background in building simulators, "digital twins," or robust A/B testing frameworks for conversational AI or autonomous agents
  • Tooling Proficiency: Familiarity with AI observability tools
  • Problem Solving: Exceptional ability to turn "noisy" AI logs into structured failure pattern analysis
  • Communication: Strong ability to translate highly technical agent behaviors into business-relevant insights for non-technical stakeholders
  • Domain Knowledge: Experience in Customer Support technology, Voice UX, or high-volume automated workflows.

What we offer

  • Eligible to participate in Uber's bonus program
  • May be offered an equity award & other types of comp
  • All full-time employees are eligible to participate in a 401(k) plan
  • Eligible for various benefits (details at link).
Uber - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Program Lead: Product Operations - AI Observability

8 matching positions

Technical Lead – AI & IOT

We are seeking a Technical Lead with strong retail technology experience to driv...
Location
Location
India , Pune
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12+ years in engineering or architecture roles, with at least 3–5 years in retail technology. Hands-on experience with Azure Services (Synapse, IoT Hub, Event Hub, Functions, APIM, AKS, AAD, Blob Storage, Monitor). Strong knowledge of microservices, API design, event-driven systems, DevOps, and cloud security.
  • Understanding of store operations, loss prevention, planogram execution, inventory workflows, demand forecasting, and supply chain processes.
  • Familiarity with AI/ML platforms, CV-based analytics, and streaming data architectures (Event Hub, Kafka, etc.). Experience with RAG/GenAI is a plus.
  • Proven ability to lead technical teams, collaborate with external partners, and deliver complex programs in fast-paced environments.
  • Azure Solutions Architect (AZ-305)(preferred, not mandatory)
  • Azure Developer (AZ-204) or Azure DevOps (AZ-400)(preferred, not mandatory)
  • Relevant retail technology or cloud certifications (preferred, not mandatory)
Job Responsibility
Job Responsibility
  • Own and drive the end-to-end technical execution of cloud, AI, and IoT-based retail modernization initiatives
  • Develop scalable, secure Azure-native architectures and oversee implementation through production rollout
  • Work closely with engineering partners and cross-functional teams to deliver prioritized retail use cases
  • Integrate data across POS, ERP, CRM, inventory, supply chain, and IoT devices using Azure services
  • Implement reliable real-time and batch data pipelines for analytics, monitoring, and automation
  • Ensure security, governance, and observability across environments
  • Convert retail workflows such as shrink reduction, planogram compliance, store operations optimization, footfall analytics, and inventory accuracy into scalable technical solutions
  • Enable data-driven decision making through automation, AI, and analytics
  • Act as the technical owner and primary point of contact for cloud and AI initiatives
  • Manage backlog, technical milestones, architecture reviews, and cross-team coordination
  • Fulltime
Read More
Arrow Right

Ai Enablement Program Manager

We are the global test and automation specialists, powering next-generation tech...
Location
Location
United States , North Reading
Salary
Salary:
173800.00 - 278100.00 USD / Year
teradyne.com Logo
Teradyne
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven people leadership capability with experience managing sizable teams of data professionals
  • Demonstrated ability to operate effectively alongside senior executive stakeholders
  • Strong program and product management mindset
  • Deep understanding of cloud data platforms and AI/ML concepts, with hands-on familiarity with Azure data services (Azure Data Factory, Fabric, or equivalent), Snowflake, Informatica, Semarchy (or MDM equivalent) and BI/analytics tooling
  • Proficiency with AI tools and platforms including Azure AI Foundry, Microsoft Copilot, Anthropic Claude, or Google Gemini
  • Excellent written and verbal communication skills
  • Demonstrated ability to translate complex business and technical requirements into clear product strategies, backlog items, and delivery plans using Agile/Scrum methodologies
  • Strong analytical and data-driven judgment
  • Ability to build trust and influence across cross-functional teams
  • Familiarity with enterprise data architecture concepts such as data lakes, data warehouses, data pipelines, and MDM
Job Responsibility
Job Responsibility
  • Partner with the Global CIO, Chief AI Officer, and Head of Enterprise Architecture and Data to drive the enterprise AI program, maintaining the integrated program roadmap and ensuring coordinated execution across all AI program workstreams
  • Define and maintain the AI program backlog in collaboration with program partners, prioritizing capabilities based on strategic alignment, business impact, and ROI
  • Facilitate cross-functional collaboration to identify high-impact AI use cases, assess feasibility, and sequence delivery for maximum enterprise-wide value
  • Champion AI adoption through organizational change management, workshops, and knowledge-sharing sessions that build AI literacy and foster a culture of innovation across Teradyne’s divisions
  • Track and communicate program progress, milestones, and outcomes
  • stay current on emerging AI technologies and industry trends, translating insights into actionable program strategies
  • Define and drive the Enterprise Data Platform vision, strategy, and roadmap—spanning data integration, master data management, and analytics/AI—translating stakeholder needs and business requirements into actionable delivery plans for the data engineering team
  • Institute AI capabilities throughout the data management lifecycle—spanning ingestion, transformation, quality, observability, and delivery—in partnership with Semiconductor Test, Robotics, Operations, and Enterprise Architecture
  • Drive stakeholder management and prioritization across Teradyne’s divisions, communicating platform strategy and progress to both technical and non-technical audiences
  • Establish success metrics, KPIs, and adoption tracking for the Enterprise Data Platform
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • Flexible Spending Accounts
  • Retirement savings plans
  • Life and disability insurance
  • Paid vacation & holidays
  • Tuition assistance programs
  • Fulltime
Read More
Arrow Right

Finance Systems AI Transformation Lead

We are looking for a Finance Systems AI Transformation Lead to join a Long-term ...
Location
Location
United States , Menlo Park
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 3 years of experience developing AI, machine learning, or data-focused applications, including practical work with LLMs or agent-based solutions
  • Strong Python programming skills with experience building APIs, automations, system integrations, and data pipelines from end to end
  • Familiarity with modern frameworks or orchestration tools used to create production-ready AI applications
  • Experience working with finance organizations, financial systems, or similarly analytical business environments
  • Solid understanding of budgeting, forecasting, financial reporting, variance analysis, and related planning concepts
  • Knowledge of enterprise platforms such as ERP systems, finance applications, or data warehouses, along with structured data integration practices
  • Ability to evaluate solution performance, troubleshoot issues effectively, and improve reliability through testing and iteration
  • Experience supporting user acceptance testing and translating business needs into practical technical outcomes
Job Responsibility
Job Responsibility
  • Design and implement AI-driven solutions that simplify core finance activities such as forecasting support, variance review, close coordination, allocation workflows, and workforce planning analysis
  • Build intelligent, multi-step automations that interpret financial information, interact with connected tools, and produce traceable outputs suitable for business use
  • Develop integrations between AI applications and enterprise finance platforms, planning systems, and data environments to support reliable information exchange
  • Create data retrieval pipelines that connect large language model capabilities with structured finance data, including reporting histories, ledger structures, cost center details, and planning records
  • Establish controls and validation methods that allow automated solutions to read, verify, and update finance-related data with consistency and audit readiness
  • Define and execute evaluation approaches to measure solution quality, including output accuracy, stability, explainability, and risk of failure in finance use cases
  • Monitor production performance, investigate issues, and refine workflows based on user feedback, testing results, and observed system behavior
  • Partner closely with finance stakeholders, financial systems teams, and technical counterparts to convert manual finance processes into scalable AI-supported operations
  • Lead user acceptance testing activities and confirm that delivered solutions meet finance, controllership, and audit expectations before broader rollout
What we offer
What we offer
  • Medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Read More
Arrow Right

Sr. Manager- AI Platform Lead

We are developing an Enterprise AI Platform to help all employees build, deploy,...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in computer science, Engineering, or equivalent practical experience with a total 12-17 years of industry experience
  • 8+ years of engineering experience building/platforming cloud services or developer platforms, with 3+ years leading engineering teams or technical programs
  • Proven experience designing and operating cloud-native platforms (Kubernetes, containers, microservices, service meshes)
  • Hands-on experience with LLM serving or model-serving patterns (hosting models, request routing, batching, scaling, latency/cost tradeoffs) — or adjacent experience (large-scale inference endpoints, model CI/CD)
  • Practical knowledge of API/Gateway patterns, authentication/authorization, and secure integrations
  • Familiarity with cost attribution and FinOps concepts for compute/AI workloads and toolchains for measuring and controlling model/agent costs
  • Strong track record working with product managers and senior technical stakeholders to deliver platform capabilities and roadmaps
  • Excellent communication skills: able to explain technical tradeoffs to technical and non-technical audiences
  • Experience with observability and SRE practices (metrics, tracing, logging, incident management)
Job Responsibility
Job Responsibility
  • Provide technical leadership and clear, pragmatic product-centric architecture for our AI platforms
  • Translate product and business requirements into scalable platform capabilities (agent hosting, LLM serving, gateway/integration architecture, observability and operations)
  • Drive platform decisions around LLM serving (model endpoints, caching, batching, latency vs. cost tradeoffs), AI Gateways (routing, policy, rate-limiting, auditing), and agent hosting patterns (single/multi-tenant, sandboxing, lifecycle)
  • Own platform reliability, scalability and cost: define SLIs/SLOs, capacity planning, cost attribution and FinOps practices
  • Collaborate with Product Owners, Principal Engineers and stakeholders to define the roadmap, acceptance criteria, and delivery milestones
  • Lead, coach and grow a high-performing engineering team focused on platform services, integrations (low/no-code tooling such as n8n, and pro-code agent hosting frameworks like AgentCore or equivalents), CI/CD for agents/models, and marketplace features
  • Establish standards for security, compliance and model governance (data handling, access controls, logging and auditability), particularly for regulated environments
  • Be hands-on when needed — prototype architectures, review designs, troubleshoot production incidents, and participate in code/design reviews
What we offer
What we offer
  • Competitive and comprehensive Total Rewards Plans that are aligned with local industry standards
  • Fulltime
Read More
Arrow Right

Global Product Manager, SaaS & AI

At Vodafone, we're not just shaping the future of connectivity for our customers...
Location
Location
Spain , Madrid
Salary
Salary:
Not provided
vodafone.com Logo
Vodafone
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven product management experience, ideally with AI Agents and/or Agent Marketplaces
  • Deep understanding of AI Agent frameworks (including agent-to-agent protocols), AI governance, and compliance
  • Strong technical acumen in cloud, API integration, identity/security, and observability
  • Expertise in customer-centric design and Design Thinking
  • Track record of leading innovation, rapid experimentation, and operationalizing new technologies
  • Commercial mindset with pricing, packaging, and business case development
  • Excellent cross-functional collaboration and stakeholder management
  • Ability to distil customer/partner feedback into clear PRDs and acceptance criteria
  • Experience taking AI features from concept, prototype, production, scaled adoption with KPIs and measurable ROI
  • Degree (Upper Second Class or higher preferred)
Job Responsibility
Job Responsibility
  • Manage first‑ and third‑party AI agents, ensuring quality, safety, publishing standards and partner readiness
  • Own vision, strategy, roadmap, pricing, packaging, commercial models, revenue share and partner frameworks for AI Agents and AI‑powered SaaS, aligned to Vodafone growth priorities
  • Drive market analysis, customer research and competitive benchmarking to inform direction
  • Lead deployment, release and full lifecycle management of agents—onboarding, configuration, governance, updates and retirement
  • Monitor and optimise AI agent KPIs, defining standards for onboarding, catalogue rules, versioning, safe updates, rollback and EOL
  • Develop voice‑enabled agent solutions (call, chat, omnichannel)
  • Build pre‑built SME templates/workflows (e.g., appointment‑booking agent)
  • Oversee creation and maintenance of all PLCM artefacts (innovation canvases, PRDs, release notes, adoption guides)
  • Ensure robust documentation, version control and stakeholder communication
  • Guide junior product specialist on artefact best practices
What we offer
What we offer
  • Hybrid work model
  • Bonus on top of the gross salary
  • Flexible working hours from Monday to Thursday, and an intensive schedule on Fridays
  • Intensive Summer Schedule during July and August
  • Up to 20 days per year of 100% remote work from other locations
  • Private Health and Life Insurance for employees and Family
  • 25 vacation days, plus December 24th and 31st off
  • Optional Pension Plan
  • Access to an online learning platform for continuous training
  • Fulltime
Read More
Arrow Right

Lead Software Engineer - AI Engineering

Join RTB House and lead our AI Engineering Lab, a team dedicated to pioneering i...
Location
Location
Poland
Salary
Salary:
Not provided
rtbhouse.com Logo
RTB House
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 6 years of professional experience in Software Engineering, with a strong background in building and deploying complex, large-scale systems
  • Distributed Systems Expertise: Proven, hands-on experience designing, developing, and operating distributed systems at scale (e.g., microservices, event-driven architectures, stream processing)
  • Programming Languages: Proficiency in at least two programming languages, with Python being mandatory
  • AI/ML Engineering: Basic understanding of the Machine Learning lifecycle, MLOps practices, and experience in integrating ML models (especially LLMs) into production applications
  • Technical Leadership: Demonstrated experience in technical leadership, including defining technical roadmaps, mentoring junior engineers, leading code reviews, and driving architectural decisions
  • Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
Job Responsibility
Job Responsibility
  • Lead, mentor, and grow a team of talented Frontend/Full Stack and Backend engineers, fostering a culture of technical excellence and high code quality
  • Serve as a Full Stack tech-leader (often hands-on), contributing to the design and development of key architectures and full stack solutions that support various platforms (Web, Mobile, CTV)
  • Define and execute the team's charter, focusing on end-to-end customer interactions and the reliable display of ads globally
  • Develop and oversee state-of-the-art observability systems for the Ad Display platform, tracking crucial metrics like reliability, viewability, latency, and providing deep debugging insights for ad creation teams
  • Provide governance for cross-team ad rollout, including defining best practices and tooling for rigorous testing and deployment strategies (A/B testing, Canary deployments)
  • Lead complex technical projects at massive scale, ensuring our solutions can handle millions of requests and maintain high performance worldwide
  • Collaborate intensely with a Staff Frontend Engineer, stakeholders from Ads layouts creation teams (designers, graphic specialists), and the core Bidding Platform backend teams.
What we offer
What we offer
  • Projects focused on high code quality – solid code reviews are our standard
  • Collaboration within an interdisciplinary, self-sufficient team including: DevOps (ensuring a great Developer Experience), database experts, backend developers, product designers, and QA engineers
  • Hardware and software tailored to your preferences – e.g. MacBook, AI tool licenses
  • Access to modern technologies and the opportunity to apply them in large-scale, high-impact projects
  • Flexible working conditions – no core hours, fully remote cooperation.
Read More
Arrow Right

Lead Software Engineer - AI Engineering

Join RTB House and lead our AI Engineering Lab, a team dedicated to pioneering i...
Location
Location
Poland
Salary
Salary:
Not provided
rtbhouse.com Logo
RTB House
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 6 years of professional experience in Software Engineering, with a strong background in building and deploying complex, large-scale systems
  • Distributed Systems Expertise: Proven, hands-on experience designing, developing, and operating distributed systems at scale (e.g., microservices, event-driven architectures, stream processing)
  • Programming Languages: Proficiency in at least two programming languages, with Python being mandatory
  • AI/ML Engineering: Basic understanding of the Machine Learning lifecycle, MLOps practices, and experience in integrating ML models (especially LLMs) into production applications
  • Technical Leadership: Demonstrated experience in technical leadership, including defining technical roadmaps, mentoring junior engineers, leading code reviews, and driving architectural decisions
  • Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
Job Responsibility
Job Responsibility
  • Lead, mentor, and grow a team of talented Frontend/Full Stack and Backend engineers, fostering a culture of technical excellence and high code quality
  • Serve as a Full Stack tech-leader (often hands-on), contributing to the design and development of key architectures and full stack solutions that support various platforms (Web, Mobile, CTV)
  • Define and execute the team's charter, focusing on end-to-end customer interactions and the reliable display of ads globally
  • Develop and oversee state-of-the-art observability systems for the Ad Display platform, tracking crucial metrics like reliability, viewability, latency, and providing deep debugging insights for ad creation teams
  • Provide governance for cross-team ad rollout, including defining best practices and tooling for rigorous testing and deployment strategies (A/B testing, Canary deployments)
  • Lead complex technical projects at massive scale, ensuring our solutions can handle millions of requests and maintain high performance worldwide
  • Collaborate intensely with a Staff Frontend Engineer, stakeholders from Ads layouts creation teams (designers, graphic specialists), and the core Bidding Platform backend teams.
What we offer
What we offer
  • Projects focused on high code quality – solid code reviews are our standard
  • Collaboration within an interdisciplinary, self-sufficient team including: DevOps (ensuring a great Developer Experience), database experts, backend developers, product designers, and QA engineers
  • Hardware and software tailored to your preferences – e.g. MacBook, AI tool licenses
  • Access to modern technologies and the opportunity to apply them in large-scale, high-impact projects
  • Flexible working conditions – no core hours, fully remote cooperation.
Read More
Arrow Right

Ai And Devops Platform Support Lead

Engineer the future of global finance. At Citi, our Tech team doesn't just suppo...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of relevant experience in a hands-on technical or support leadership role
  • Experience contributing to architecture discussions and ensuring solutions align with enterprise standards and long-term maintainability
  • Experience working with senior stakeholders or technology partners
  • Demonstrated experience supporting IT service improvements or platform stability initiatives
  • Strong communication and presentation skills, with the ability to convey technical concepts clearly
  • Experience supporting or contributing to technical roadmaps or operational workstreams
  • Experience participating in resilience-related activities such as incident simulations, disaster recovery exercises, or stability testing
  • Ability to collaborate with cross-functional support teams and technology groups
  • Strong organizational and workload-planning skills
  • Consistently demonstrates clear and concise written and verbal communication skills
Job Responsibility
Job Responsibility
  • Demonstrates a strong understanding of how application support contributes to the overall technology function and organizational objectives
  • Assist with vendor relationship management, including coordination with offshore managed services
  • Support efforts to improve service levels for end users by enhancing operational efficiencies and strengthening incident management, problem management, and knowledge-sharing practices
  • Partner with development teams to guide improvements in application stability and supportability
  • Contribute to frameworks for managing capacity, throughput, and latency
  • Assist in defining and implementing application onboarding guidelines and standards
  • Support team members by fostering a collaborative environment and encouraging skill development
  • Participate in cost-reduction efforts through Root Cause Analysis reviews, knowledge management, performance tuning, and user training
  • Participate in business review meetings to help align technology tools and strategies with business requirements
  • Ensure adherence to support processes and tool standards, and assist in enhancing processes to promote consistency and quality across the support program
  • Fulltime
Read More
Arrow Right