CrawlJobs Logo

UX Engineer, LLM Experimentation Platform

arize.com Logo

Arize

Location Icon

Location:
United States

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

175000.00 - 200000.00 USD / Year

Job Description:

You'll work on our LLM evaluation and prompt experimentation platform, taking functional features and refining them into intuitive, polished experiences. We're a technical team building for technical users (ML engineers, researchers) who are drowning in complexity. Your job is to make that complexity manageable and maybe even enjoyable.

Job Responsibility:

  • Refine and iterate on UX for existing features
  • Watch real users struggle in Fullstory, identify friction points, and fix them
  • Prototype interaction alternatives rapidly (in Figma or code) to test different approaches
  • Work closely with our Senior engineers who own feature architecture
  • Own the 'feel' of the product: transitions, feedback, error handling, progressive disclosure
  • Build reusable interaction patterns that the team can leverage
  • Run lightweight usability tests to validate improvements

Requirements:

  • You've made products noticeably better through interaction refinement
  • You're comfortable with modern frontend frameworks and animation libraries
  • You've used session replay tools and incorporated user feedback into designs
  • You can work in Figma to prototype before committing to code
  • You think about edge cases, loading states, and empty states automatically
  • You've worked on complex tools (dev tools, data tools, B2B SaaS) where UX really matters

Nice to have:

You've built or contributed to design systems

What we offer:
  • competitive equity package
  • comprehensive benefits package, including medical, dental, vision
  • a 401(k) plan
  • unlimited paid time off
  • a generous parental leave plan
  • additional support for mental health and wellness
  • WFH monthly stipend to pay for co-working spaces

Additional Information:

Job Posted:
December 20, 2025

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for UX Engineer, LLM Experimentation Platform

Director of AI Engineering

We are entering a hyper-growth phase of AI innovation and are hiring a Director ...
Location
Location
Canada; United States
Salary
Salary:
300000.00 - 450000.00 USD / Year
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10–15+ years in software engineering, with significant leadership experience owning AI/ML or applied LLM systems at scale
  • Proven history shipping LLM-powered features, agentic workflows, or AI assistants used by real customers in production
  • Deep understanding of LLM orchestration frameworks (LangChain, LlamaIndex), RAG pipelines, vector search, embeddings, and prompt engineering
  • Expert in backend & distributed systems (Python strongly preferred) and cloud infrastructure (AWS/GCP)
  • Strong experience with telemetry, observability, and cost-aware real-time inference optimizations
  • Demonstrated ability to lead senior engineers, define technical roadmaps, and deliver outcomes aligned to business metrics
  • Experience building or scaling teams working on experimentation, optimization, personalization, or ML-powered growth systems
  • Exceptional ability to simplify complex problems, set clear standards, and drive alignment across Product, Data, Design, and Engineering
  • Strong product sense, ability to weigh novelty vs. impact, focus on user value, and prioritize speed with guardrails
  • Fluent in integrating AI tools into engineering workflows for code generation, debugging, delivery velocity, and operational efficiency
Job Responsibility
Job Responsibility
  • Define the multi-year technical vision for Apollo’s AI stack, spanning agents, orchestration, inference, retrieval, and platformization
  • Prioritize high-impact AI investments by partnering with Product, Design, Research, and Data leaders to align engineering outcomes with business goals
  • Establish technical standards, evaluation criteria, and success metrics for every AI-powered feature shipped
  • Lead the architecture and deployment of long-horizon autonomous agents, multi-agent workflows, and API-driven orchestration frameworks
  • Build reusable, scalable agentic components that power GTM workflows like research, enrichment, sequencing, lead scoring, routing, and personalization
  • Own the evolution of Apollo’s internal LLM platform for high-scale, low-latency, cost-optimized inference
  • Oversee model-driven experiences for natural-language interfaces, RAG pipelines, semantic search, personalized recommendations, and email intelligence
  • Partner with Product & Design to build intuitive conversational UX that hides underlying complexity while elevating user productivity
  • Implement rigorous evaluation frameworks, including offline benchmarking, human-in-the-loop review, and online A/B experimentation
  • Ensure robust observability, monitoring, and safety guardrails for all AI systems in production
What we offer
What we offer
  • Equity
  • Company bonus or sales commissions/bonuses
  • 401(k) plan
  • At least 10 paid holidays per year
  • Flex PTO
  • Parental leave
  • Employee assistance program and wellbeing benefits
  • Global travel coverage
  • Life/AD&D/STD/LTD insurance
  • FSA/HSA
  • Fulltime
Read More
Arrow Right

Principal AI Engineer

We are looking for a Principal AI Engineer to lead the design and deployment of ...
Location
Location
United States
Salary
Salary:
200000.00 - 300000.00 USD / Year
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of software engineering experience
  • at least 3 years in applied LLM or agentic AI systems (2023–present)
  • proven success in deploying LLM-powered products used by real users at scale
  • deep backend & systems engineering expertise with Python, distributed systems, and scalable APIs
  • familiarity with LangChain, LlamaIndex, or similar orchestration frameworks
  • experience with RAG pipelines, vector DBs, embedding models, and semantic search tuning
  • experience managing performance across cloud providers (e.g., AWS Bedrock, OpenAI, Anthropic, etc.)
  • demonstrated experience building multi-step agents, planning workflows, chaining reasoning steps, and integrating APIs with agent memory/state
  • comfort with advanced prompting strategies, few-shot and chain-of-thought reasoning, and embedding retrieval setups
  • strong understanding of AI system evaluation, human ratings, A/B experimentation, and feedback loop pipelines
Job Responsibility
Job Responsibility
  • Architect and lead the development of multi-agent systems capable of long-horizon planning, reasoning, and API orchestration
  • build reusable agentic components that integrate deeply into sales and marketing processes
  • own and evolve our in-house platform for scalable, low-latency, and cost-efficient LLM and agent deployments
  • lead design of interfaces powered by natural language understanding and retrieval-augmented generation (RAG)
  • build embedding-based, intent-aware search and personalization systems tuned to business user needs
  • drive innovation in personalized outreach generation using context-aware generation pipelines
  • tune inference pipelines, caching layers, and model selection logic for high-scale, cost-aware performance
  • define and drive robust offline and online testing methodologies (A/B, sandboxing, human evals) across agents and LLM flows
  • architect human-in-the-loop systems and telemetry to improve accuracy, UX, and explainability over time
What we offer
What we offer
  • equity
  • company bonus or sales commissions/bonuses
  • 401(k) plan
  • at least 10 paid holidays per year
  • flex PTO
  • parental leave
  • employee assistance program
  • wellbeing benefits
  • global travel coverage
  • life/AD&D/STD/LTD insurance
  • Fulltime
Read More
Arrow Right

Senior Product Manager

The Response and conversation experiences (RCX) core team owns the user experien...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience as a Technical product / program manager shipping platforms or large‑scale user experiences
  • Systems thinking, demonstrated technical depth in APIs, contracts, front end engineering, telemetry and performance fundamentals
  • Experience building extensibility frameworks (plugins, SDKs, component kits)
  • Proven ability to design and operationalize product evaluation & instrumentation (quality metrics, automated gates, AB testing, scorecards)
  • Strong track record of driving partner integrations at scale across multiple teams
Job Responsibility
Job Responsibility
  • UX platform ownership: Define contracts, APIs, runtime controls and shared UX patterns that power Copilot across surfaces
  • UX extensibility: Land partner integrations across M365 apps, frameworks to add domain‑specific capabilities at scale
  • Instrumentation & evaluation: Frameworks to ensure faster metrics onboarding, automated LLM based evaluations of UX quality
  • Performance & reliability: Measurably improve product performance, memory management and live site handling
  • Agility: Own the products, tools, and processes to ensure faster experimentation
  • Fulltime
Read More
Arrow Right

Senior Design Technologist

At Capgemini Invent, we believe difference drives change. As inventive transform...
Location
Location
United States , San Francisco
Salary
Salary:
100000.00 - 150000.00 USD / Year
frog.co Logo
frog
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Professional experience designing and developing user interfaces, UX concepts, web sites and applications on a variety of platforms, frameworks, and methodologies
  • Ability to collaborate with a multi-disciplinary team of designers and strategists
  • A background in data management and manipulation to support data specific needs and concepts
  • Ability to communicate with clients directly and complete program and workstream tasks autonomously in some cases
  • Exceptional written and verbal communications skills along with experience in preparing and delivering documentation, proposals, presentations, and structured working sessions to all levels in a client organization
  • Bachelor’s Degree in the sciences, engineering, or creative discipline or equivalent experience
  • Proficiency in: Modern web front-end application development
  • Working within Cloud platforms (e.g., AWS, GCP, Azure, etc.)
  • Source control (e.g., GitHub, GitLab) and toolchain integrations
  • Third-party API integrations
Job Responsibility
Job Responsibility
  • Design Technologists at frog blend futurist vision and practical thinking, serving as hands-on organizers of technical insights, experiments, proofs, and models
  • Bridge the gap between creative design services, technology strategy, and software engineering
  • Drive technology platform research, experimentation, concept exploration, prototyping, and ideation within the creative process of designing user experiences
  • Leverage and manage data to support technical needs ranging from AI applications to impact assessments
  • Gather, ingest, and utilize data from diverse modalities to advance project goals
  • Identify and analyze current and emerging UX technology patterns and trends
  • Conduct in-depth investigations and experiments with new platforms and frameworks
  • Spot new opportunities for business value and UX innovation through technology enablers
  • Work closely with the creative team to translate design assets into user experience simulations, interactive prototypes, technical proofs-of-concept, and UI/presentation layers
  • Embrace a maker and tinkerer mindset—technically adept, passionate about design, and eager to tackle challenges hands-on
What we offer
What we offer
  • Flexible work
  • Healthcare including dental, vision, mental health, and well-being programs
  • Financial well-being programs such as 401(k) and Employee Share Ownership Plan
  • Paid time off and paid holidays
  • Paid parental leave
  • Family building benefits like adoption assistance, surrogacy, and cryopreservation
  • Social well-being benefits like subsidized back-up child/elder care and tutoring
  • Mentoring, coaching and learning programs
  • Employee Resource Groups
  • Disaster Relief
  • Fulltime
Read More
Arrow Right

Senior Design Technologist

At Capgemini Invent, we believe difference drives change. As inventive transform...
Location
Location
United States , San Francisco
Salary
Salary:
100000.00 - 150000.00 USD / Year
frog.co Logo
frog
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Professional experience designing and developing user interfaces, UX concepts, web sites and applications on a variety of platforms, frameworks, and methodologies
  • Ability to collaborate with a multi-disciplinary team of designers and strategists
  • A background in data management and manipulation to support data specific needs and concepts
  • Ability to communicate with clients directly and complete program and workstream tasks autonomously in some cases
  • Exceptional written and verbal communications skills along with experience in preparing and delivering documentation, proposals, presentations, and structured working sessions to all levels in a client organization
  • Bachelor’s Degree in the sciences, engineering, or creative discipline or equivalent experience
  • Proficiency in: Modern web front-end application development
  • Working within Cloud platforms (e.g., AWS, GCP, Azure, etc.)
  • Source control (e.g., GitHub, GitLab) and toolchain integrations
  • Third-party API integrations
Job Responsibility
Job Responsibility
  • Design Technologists at frog blend futurist vision and practical thinking, serving as hands-on organizers of technical insights, experiments, proofs, and models
  • Bridge the gap between creative design services, technology strategy, and software engineering
  • Drive technology platform research, experimentation, concept exploration, prototyping, and ideation within the creative process of designing user experiences
  • Leverage and manage data to support technical needs ranging from AI applications to impact assessments
  • Gather, ingest, and utilize data from diverse modalities to advance project goals
  • Identify and analyze current and emerging UX technology patterns and trends
  • Conduct in-depth investigations and experiments with new platforms and frameworks
  • Spot new opportunities for business value and UX innovation through technology enablers
  • Work closely with the creative team to translate design assets into user experience simulations, interactive prototypes, technical proofs-of-concept, and UI/presentation layers
  • Embrace a maker and tinkerer mindset—technically adept, passionate about design, and eager to tackle challenges hands-on
What we offer
What we offer
  • Flexible work
  • Healthcare including dental, vision, mental health, and well-being programs
  • Financial well-being programs such as 401(k) and Employee Share Ownership Plan
  • Paid time off and paid holidays
  • Paid parental leave
  • Family building benefits like adoption assistance, surrogacy, and cryopreservation
  • Social well-being benefits like subsidized back-up child/elder care and tutoring
  • Mentoring, coaching and learning programs
  • Employee Resource Groups
  • Disaster Relief
  • Fulltime
Read More
Arrow Right

Senior Software Engineer

Project Sophia is a new generation business application, built ground up from ma...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 3+ years of extensive experience with one or more modern web technologies such as .NET / Node / React / Angular, building RESTful APIs
  • 2+ years of solid experience in an OO Language like C# or Java
  • 1+ year experience with large language models (LLMs) and generative AI
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Design, implement, and ship AI-first product capabilities end-to-end from rapid prototype to production, spanning LLM-powered services, retrieval/grounding pipelines, and intelligent UX experiences that delight users through Sophia’s AI canvas
  • Own architecture and implementation across the full stack integrating front-end experiences, back-end services, and AI orchestration layers that connect models, context, and tools to deliver cohesive, extensible, high-performance systems
  • Collaborate with design, research, and platform teams to adapt or fine-tune LLMs/SLMs and multimodal models for real-world customer scenarios, ensuring outcomes are contextual, transparent, and human-centered
  • Build agentic, tool-using, and multimodal workflows that reason across data and services
  • optimize for safety, latency, reliability, and cost efficiency
  • Contribute to engineering excellence secure-by-design, accessibility compliance, automated testing, and code craftsmanship across the product lifecycle
  • Instrument and evaluate AI features with telemetry, experimentation, and continuous feedback loops to refine reasoning quality and user experience
  • Drive live-site reliability and operational excellence, participating in On-Call rotations while maintaining a sustainable, high-ownership engineering culture
  • Mentor peers and influence cross-team practices, sharing knowledge across AI, UX, and system-design disciplines
  • Fulltime
Read More
Arrow Right
New

Software Engineer II

The Follow-on Suggestions team in STCI, in the Microsoft AI (MAI) organization d...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience, and experience with generative AI
  • Drive experimentation through A/B testing and offline evaluation to evaluate system performance
  • Comfortable driving complex server and client architecture across large product teams
  • Hands-on experience with modern LLM evaluation techniques, including LLM-as-a-Judge, agentic evaluations and RAG assessments
  • A track record of delivering successful, large scale applied ML projects in an industry setting
  • Experience with MLOps practices, model versioning, automated testing, monitoring and CI/CD for machine learning
  • Experience with proficient coding, debugging, and problem-solving skills
  • Outstanding communication and collaboration skills
Job Responsibility
Job Responsibility
  • Design, implement, and ship AI-first product capabilities end-to-end from rapid prototype to production, spanning LLM-powered services, retrieval/grounding pipelines, and intelligent UX experiences
  • Own implementation across the full stack integrating front-end experiences, back-end services, and AI orchestration layers that connect models, context, and tools to deliver cohesive, extensible, high-performance recommendation systems
  • Collaborate with design, research, and platform teams to adapt or fine-tune LLMs/SLMs for follow-on scenarios
  • Build agentic, tool-using workflows that reason across data and services
  • optimize for security, safety, latency, reliability, and cost efficiency
  • Contribute to engineering excellence secure-by-design, accessibility compliance, automated testing, and code craftsmanship across the product lifecycle
  • Instrument and evaluate AI features with telemetry, experimentation, and continuous feedback loops to improve user experience
  • Drive live-site reliability and operational excellence, participating in On-Call rotations while maintaining a sustainable, high-ownership engineering culture
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Agentic

Join us in building the future of finance. Our mission is to democratize finance...
Location
Location
United States , Menlo Park, CA; Bellevue, WA
Salary
Salary:
196000.00 - 230000.00 USD / Year
robinhood.com Logo
Robinhood
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of full-stack software engineering experience, ideally in AI, ML, or data-heavy environments
  • Strong experience building frontend and backend applications using JavaScript/TypeScript, Node.js, and modern frameworks (React, Vue, Angular)
  • Experience working with LLMs and agentic systems, including prompt engineering, orchestration, or RAG-style patterns
  • Solid product and UX instincts, especially for internal or enterprise tools
  • Comfort working in large codebases or monorepos (e.g., Bazel) and shipping production-grade systems
  • Strong collaboration skills and a mindset focused on building developer- and user-friendly platforms
Job Responsibility
Job Responsibility
  • Build and maintain full-stack tools and frameworks that enable teams to develop and deploy AI-powered internal applications
  • Design and implement client- and server-side systems to support scalable, production-ready agentic AI solutions
  • Develop workflows and tooling for agent development, enabling rapid prototyping, experimentation, and iteration
  • Build internal AI products ranging from workflow automation to full internal platforms used across the company
  • Collaborate closely with AI engineers and internal stakeholders to deliver reusable libraries, abstractions, and tooling
  • Help shape how agentic AI is applied across Robinhood, with a focus on reliability, usability, and long-term impact
What we offer
What we offer
  • Performance-driven compensation with multipliers for outsized impact, bonus programs, equity ownership, and 401(k) matching
  • 100% paid health insurance for employees with 90% coverage for dependents
  • Lifestyle wallet — a highly flexible benefits spending account for wellness, learning, and more
  • Employer-paid life & disability insurance, fertility benefits, and mental health benefits
  • Time off to recharge including company holidays, paid time off, sick time, parental leave, and more
  • Exceptional office experience with catered meals, events, and comfortable workspaces
  • Fulltime
Read More
Arrow Right