CrawlJobs Logo

Staff LLM Systems Engineer

darwinrecruitment.com Logo

Darwin Recruitment GmbH

Location Icon

Location:
United States , New York

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

We are seeking a Senior / Staff LLM Systems Engineer to lead the development, optimization, and deployment of large language model inference pipelines. This role focuses on high-throughput, low-latency serving and production reliability, bridging ML research and platform engineering. This is not a training-focused role – the emphasis is on serving models at scale, optimizing systems, and enabling production ML reliability.

Job Responsibility:

  • Design, implement, and optimize inference pipelines for large language models
  • Improve throughput and latency of model serving in production environments
  • Collaborate closely with infrastructure, platform, and ML research teams to ensure smooth deployment
  • Build monitoring, observability, and alerting systems for inference performance and reliability
  • Identify and solve scaling challenges across GPUs, TPUs, or distributed environments
  • Evaluate and adopt new technologies, frameworks, and architectures to improve inference efficiency
  • Mentor other engineers and contribute to technical strategy for production ML systems

Requirements:

  • 5+ years of software engineering experience, including hands-on ML systems experience
  • Strong background in distributed systems, performance tuning, and low-latency architectures
  • Experience with model serving frameworks (e.g., Triton, vLLM, Ray, TorchServe)
  • Familiarity with GPU/TPU infrastructure, multi-node deployment, and system-level optimization
  • Understanding of ML workloads and trade-offs between accuracy, latency, and cost
  • Proven ability to deliver production-grade ML systems at scale
  • Excellent collaboration and problem-solving skills
What we offer:
  • Competitive compensation
  • Flexible work arrangements

Additional Information:

Job Posted:
January 05, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Icon
Darwin Recruitment GmbH - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Staff LLM Systems Engineer

LLM - Senior Staff Engineer - Python + Machine Learning

AquSag is seeking a hands-on Machine Learning Senior Staff Engineer to lead cros...
Location
Location
Salary
Salary:
40.00 - 60.00 USD / Hour
aqusag.com Logo
AquSag Technologies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 9+ yrs of strong background in Machine Learning, NLP, and modern deep learning architectures (Transformers, LLMs)
  • Hands-on experience with frameworks such as PyTorch, TensorFlow, Hugging Face, or DeepSpeed
  • Hands-on experience in Docker for Production deployment
  • Proven experience managing teams delivering ML/LLM models in production environments
  • Knowledge of distributed training, GPU/TPU optimization, and cloud platforms (AWS, GCP, Azure)
  • Familiarity with MLOps tools like MLflow, Kubeflow, or Vertex AI for scalable ML pipelines
  • Excellent leadership, communication, and cross-functional collaboration skills
  • Bachelor’s or Master’s in Computer Science, Engineering, or related field (PhD preferred)
  • Overlap of 6 hours with PST time zone is mandatory
  • Commitments Required: 8 hours per day with overlap of 6 hours with PST
Job Responsibility
Job Responsibility
  • Lead and mentor a cross-functional team of ML engineers, data scientists, and MLOps professionals
  • Oversee the full lifecycle of LLM and ML projects — from data collection to training, evaluation, and deployment
  • Collaborate with Research, Product, and Infrastructure teams to define goals, milestones, and success metrics
  • Provide technical direction on large-scale model training, fine-tuning, and distributed systems design
  • Implement best practices in MLOps, model governance, experiment tracking, and CI/CD for ML
  • Manage compute resources, budgets, and ensure compliance with data security and responsible AI standards
  • Communicate progress, risks, and results to stakeholders and executives effectively
  • Fulltime
Read More
Arrow Right

Staff AI Engineer

As a Staff AI Engineer on our AI Engineering team, you will be responsible for b...
Location
Location
India
Salary
Salary:
Not provided
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of software engineering experience with a focus on production systems
  • 1.5+ years of hands-on LLM experience (2023-present) building real applications with GPT, Claude, Llama, or other modern LLMs
  • Demonstrated experience building customer-facing, scalable LLM-powered products with real user usage
  • Experience building multi-step AI agents, LLM chaining, and complex workflow automation
  • Deep understanding of prompting strategies, few-shot learning, chain-of-thought reasoning, and prompt optimization techniques
  • Expert-level Python skills for production AI systems
  • Strong experience building scalable backend systems, APIs, and distributed architectures
  • Experience with LangChain, LlamaIndex, or other LLM application frameworks
  • Proven ability to integrate multiple APIs and services to create advanced AI capabilities
  • Experience deploying and managing AI models in cloud environments (AWS, GCP, Azure)
Job Responsibility
Job Responsibility
  • Design and Deploy Production LLM Systems
  • Agent Development
  • Prompt Engineering Excellence
  • System Integration
  • Evaluation & Quality Assurance
  • Performance Optimization
  • Cross-functional Collaboration
Read More
Arrow Right

Staff AI Engineer

As a Staff AI Engineer on our AI Engineering team, you will be responsible for b...
Location
Location
United States
Salary
Salary:
200000.00 - 280000.00 USD / Year
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of software engineering experience with a focus on production systems
  • 1.5+ years of hands-on LLM experience (2023-present) building real applications with GPT, Claude, Llama, or other modern LLMs
  • Demonstrated experience building customer-facing, scalable LLM-powered products with real user usage
  • Experience building multi-step AI agents, LLM chaining, and complex workflow automation
  • Deep understanding of prompting strategies, few-shot learning, chain-of-thought reasoning, and prompt optimization techniques
  • Expert-level Python skills for production AI systems
  • Strong experience building scalable backend systems, APIs, and distributed architectures
  • Experience with LangChain, LlamaIndex, or other LLM application frameworks
  • Proven ability to integrate multiple APIs and services to create advanced AI capabilities
  • Experience deploying and managing AI models in cloud environments (AWS, GCP, Azure)
Job Responsibility
Job Responsibility
  • Design and Deploy Production LLM Systems
  • Create sophisticated AI agents that can chain multiple LLM calls, integrate with external APIs, and maintain state across complex workflows
  • Develop and optimize prompting strategies
  • Build robust APIs and integrate AI capabilities with existing Apollo infrastructure and external services
  • Implement comprehensive evaluation frameworks, A/B testing, and monitoring systems
  • Optimize for cost, latency, and scalability across different LLM providers and deployment scenarios
  • Work closely with product teams, backend engineers, and stakeholders to translate business requirements into technical AI solutions
  • Build sophisticated multi-agent systems that can reason, plan, and execute complex sales workflows
  • Develop systems that maintain conversational context across complex multi-turn interactions
  • Build scalable large language model and agentic platforms
What we offer
What we offer
  • equity
  • company bonus or sales commissions/bonuses
  • 401(k) plan
  • at least 10 paid holidays per year, flex PTO, and parental leave
  • employee assistance program and wellbeing benefits
  • global travel coverage
  • life/AD&D/STD/LTD insurance
  • FSA/HSA and medical, dental, and vision benefits
  • Fulltime
Read More
Arrow Right

Staff Backend Engineer

As a Staff Software Engineer at Apollo, you’ll be responsible for leading the te...
Location
Location
India
Salary
Salary:
Not provided
apollo.io Logo
Apollo.io
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of deep experience building and scaling backend systems in production environments
  • Proven ability to develop features with performance, reliability, and scalability in mind
  • Strong product intuition—balancing technical scope with MVP delivery
  • Expertise with message queues, background jobs, and distributed system patterns
  • Familiarity with observability tools (e.g., GCP Logging, Prometheus, NewRelic)
  • Demonstrated ability to leverage AI tools (e.g., code generation, debugging, automation) and stay current with emerging AI trends
  • Experience with LLM architecture, pipelines, or applied AI tooling is a strong bonus
Job Responsibility
Job Responsibility
  • Architect and build scalable, high-availability systems that run 24/7 and support mission-critical workflows
  • Own and deliver multi-tier, high-volume applications with performance, accessibility, and resilience in mind
  • Write high-quality, maintainable code with a strong focus on developer experience and system performance
  • Drive and uphold engineering best practices, including code reviews, test coverage, monitoring, and observability
  • Provide technical leadership across cross-functional teams, guiding architecture, trade-offs, and execution
  • Mentor and sponsor engineers to grow their impact and deepen engineering culture
  • Lead large-scale, cross-team projects from concept to production
  • Deliver clear, objective feedback to peers and junior engineers through structured reviews and 1:1s
Read More
Arrow Right

Staff Software Engineer, Backend

The Staff Engineer will work closely with AI/ML engineers, product managers, app...
Location
Location
United States , NYC
Salary
Salary:
160000.00 - 190000.00 USD / Year
conductor.com Logo
Conductor
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Completed studies in Computer Science, Mathematics, engineering or a related field or equivalent professional experience
  • 8+ years of experience in software development, with experience in product-driven companies
  • Strong expertise in system design, distributed computing, and scalable architecture patterns for handling large datasets and high-throughput applications
  • Proficiency in multiple programming languages with strong Python coding skills. Experience with Java is highly valued
  • Strong database experience including both SQL and NoSQL systems, with knowledge of data modeling and optimization techniques
  • Experience with AI/ML technologies including LLMs, vector databases (e.g., Milvus), embeddings, and ML frameworks
  • Knowledge of MLOps practices, model deployment, and AI system integration in production environments
  • Experience working across the full software development lifecycle including CI/CD, monitoring, testing, and production deployment
  • Proven track record of technical leadership, mentoring engineers, and driving engineering excellence within teams
  • Up-to-date with rapidly-evolving technologies and demonstrated ability to evaluate and adopt new tools and frameworks
Job Responsibility
Job Responsibility
  • Lead the technical architecture, design, and implementation of large-scale distributed systems and data platforms to support customer needs and business growth
  • Oversee the planning, execution, and successful delivery of complex engineering projects, ensuring adherence to engineering best practices and quality standards
  • Design and build scalable, high-performance backend systems and APIs that handle millions of requests and large datasets efficiently
  • Architect robust data processing pipelines and ETL workflows using modern cloud technologies and distributed computing frameworks
  • Drive technical decision-making across the engineering organization, evaluating trade-offs and establishing engineering standards and practices
  • Lead cross-functional collaboration with product, AI/ML engineering, data engineering, and infrastructure teams to deliver comprehensive solutions
  • Build and maintain CI/CD pipelines, monitoring systems, and deployment automation to ensure reliable software delivery
  • Implement AI/ML capabilities including LLM integration, vector databases, and intelligent content processing workflows
  • Mentor senior and junior engineers, fostering technical excellence and knowledge sharing within the engineering organization
What we offer
What we offer
  • 100% covered employee medical plan
  • a dental & vision plans
  • 401(k) with employer contribution
  • an unlimited vacation policy
  • 10 sick days
  • short-term disability
  • long-term disability
  • generous paid parental leave
  • employee assistance program
  • flexible savings accounts
  • Fulltime
Read More
Arrow Right

Staff Software Engineer, Core AI

As a Staff AI Engineer on our Core AI team, you will be a cornerstone of FloQast...
Location
Location
United States , San Jose
Salary
Salary:
164000.00 - 246000.00 USD / Year
floqast.com Logo
FloQast
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional software engineering experience
  • 4+ years focused on building backend for production applications
  • Mastery of Python
  • Familiarity with some AI application frameworks, context engineering, and scalable system design for AI products
  • Expertise in designing products that integrate with multiple technologies, APIs, and data sources in cloud-native environments (AWS preferred)
  • Strong desire to develop deep hands-on experience with LLM APIs, retrieval-augmented generation (RAG), conversational AI, document processing, and MCP integrations
  • Proven ability to lead tech product initiatives, establish technical standards and communicate complex system designs to both technical and business stakeholders
Job Responsibility
Job Responsibility
  • Architect and lead development of production AI products including intelligent chatbots, document processing systems, and agentic workflows using Python and modern AI frameworks
  • Design and implement our centralized AI platform including model routing, provider management, vector search, and AI application frameworks with seamless MCP (Model Context Protocol) integrations
  • Build scalable AI products that integrate with diverse technologies including accounting systems, document repositories, and external APIs while maintaining robust monitoring and observability
  • Master context engineering and system design for AI applications, ensuring optimal information retrieval, context assembly, and multi-turn conversation management
  • Collaborate with Product, Engineering, and Security teams to ensure AI products are robust, compliant, and aligned with business objectives in the regulated accounting space
  • Provide technical leadership and mentorship to the growing AI team, establishing best practices for AI product development, deployment, and governance
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • Family Forming benefits
  • Life & Disability Insurance
  • Unlimited Vacation
  • Fulltime
Read More
Arrow Right

Staff AI Innovation Engineer, Employee Experience

Architect the next generation of Airbnb’s internal operating system by designing...
Location
Location
United States
Salary
Salary:
180000.00 - 225000.00 USD / Year
airbnb.com Logo
Airbnb
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 9+ years in product management, AI engineering, and/or platform architecture, with hands-on experience building and launching AI/ML-powered products
  • Proven 0→1 builder with extensive experience developing GenAI prototypes, tools, and platforms from scratch
  • Ability to produce functional prototypes (using Claude Code or similar) within hours and stakeholder-ready demos within days
  • Strong systems-thinking skills with the ability to map complex workflows, diagnose root causes, and design buildable solutions
  • Expertise in prompt engineering, context orchestration, and adapting LLM behavior to different users, workflows, and decision contexts
  • Data-informed decision-maker who can define success criteria, measure ROI, and refine models and workflows
  • Skilled in communicating technical concepts to non-technical audiences
  • Highly collaborative, systems-oriented, and adept at connecting dots across people, tools, and processes
  • Strong visual and narrative communication skills (Figma, Miro, slideware)
  • Portfolio of prototypes or relevant 0→1 work preferred but not required
Job Responsibility
Job Responsibility
  • Translate employee and manager workflows into well-defined AI product opportunities
  • Rapidly prototype and test working solutions (LLM agents, workflow automations, microapps, embedded tools, etc.)
  • Architect systems integrations across our HR tech stack (Workday, Greenhouse, Qualtrics, Airtable, RightAnswers, Asana, and more)
  • Lead discovery sessions with stakeholders to uncover automation and augmentation opportunities
  • Evaluate third-party AI tools and determine build vs. buy options
  • Design and execute fast-cycle pilots across knowledge management, decision support, communications, and resource planning
  • Define success metrics, measure impact, and iterate based on user feedback
  • Translate complex AI architectures into clear narratives that help stakeholders understand value and adoption paths
  • Ensure alignment with Airbnb’s privacy, security, and ethical AI guardrails
  • Continuously evolve EX’s AI strategy as organizational needs and technologies shift
What we offer
What we offer
  • bonus
  • equity
  • benefits
  • Employee Travel Credits
  • Fulltime
Read More
Arrow Right

Member of Technical Staff - Platform Engineer

Platform Engineer to join our team building backend infrastructure for new ML-po...
Location
Location
United States , Palo Alto
Salary
Salary:
175000.00 - 350000.00 USD / Year
inflection.ai Logo
Inflection AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Backend engineering experience with Python, TypeScript, or Node.js
  • Hands-on experience working with production PyTorch models, model checkpoints, and inference logic
  • Strong knowledge of building APIs and services that are scalable, stable, and secure
  • Passion for bridging backend engineering and ML systems, especially at the infrastructure layer
  • Familiarity with tools such as FastAPI, Postgres, Redis, Kubernetes, and React
  • Desire to be hands-on and contribute to shaping the foundation of a new enterprise ML product
  • Have a bachelor’s degree or equivalent in a related field to the offered position requirements
Job Responsibility
Job Responsibility
  • Build and maintain backend services to support LLM integration, inference orchestration, and data flow
  • Write clean, reliable Python code for experimentation, model integration, and production systems
  • Collaborate closely with ML researchers to rapidly iterate on product ideas and deploy features
  • Design and implement infrastructure to handle scalable inference workloads and enterprise-level use cases
  • Own system components and ensure reliability, observability, and maintainability from day one
What we offer
What we offer
  • Diverse medical, dental and vision options
  • 401k matching program
  • Unlimited paid time off
  • Parental leave and flexibility for all parents and caregivers
  • Support of country-specific visa needs for international employees living in the Bay Area
  • Competitive stock options
Read More
Arrow Right
Jobs by Category
Art and Entertainment Jobs Banking Jobs Call Center Jobs Construction Jobs Consulting Jobs Customer Service Jobs Education Jobs Energy Jobs Finance Jobs Gastronomy Jobs Health and Beauty Jobs Hospitality and Tourism Jobs Human Resources Jobs Insurance Jobs IT - Administration Jobs IT - Software Development Jobs Legal Jobs Logistics Jobs Manufacturing Jobs Marketing, Digital Marketing, SEO, SEM Jobs Media Jobs Nursing Jobs Office Administration Jobs Pharmacy Jobs Physical Work Jobs Public Relations Jobs Public Sector Jobs Purchasing Jobs Quality Control Jobs Real Estate Jobs Research and Development Jobs Sales Jobs
Job Positions Jobs by Country
United States Jobs United Kingdom Jobs India Jobs Canada Jobs Australia Jobs Poland Jobs Ireland Jobs Germany Jobs
Company
Blog About Us Contact
Follow Us
Facebook
X (Twitter)
LinkedIn
Instagram
© 2026 CrawlJobs Ltd. All Rights Reserved
Terms & Conditions Privacy Policy