ML Model Serving Engineer Job at Sesame (San Francisco)

Ai infrastructure engineer, model serving platform

As a Software Engineer on the ML Infrastructure team, you will design and build ...

Location

United States , San Francisco; New York

Salary:

179400.00 - 224250.00 USD / Year

Scale

Expiration Date

Until further notice

Requirements

4+ years of experience building large-scale, high-performance backend systems
Strong programming skills in one or more languages (e.g., Python, Go, Rust, C++)
Experience with LLM serving and routing fundamentals (e.g. rate limiting, token streaming, load balancing, budgets, etc.)
Experience with LLM capabilities and concepts such as reasoning, tool calling, prompt templates, etc.
Experience with containers and orchestration tools (e.g., Docker, Kubernetes)
Familiarity with cloud infrastructure (AWS, GCP) and infrastructure as code (e.g., Terraform)
Proven ability to solve complex problems and work independently in fast-moving environments

Job Responsibility

Build and maintain fault-tolerant, high-performance systems for serving LLMs workloads at scale
Build an internal platform to empower LLM capability discovery
Collaborate with researchers and engineers to integrate and optimize models for production and research use cases
Conduct architecture and design reviews to uphold best practices in system design and scalability
Develop monitoring and observability solutions to ensure system health and performance
Lead projects end-to-end, from requirements gathering to implementation, in a cross-functional environment

What we offer

Comprehensive health, dental and vision coverage
retirement benefits
a learning and development stipend
generous PTO

Fulltime

ML Engineer - Inference Serving

Luma’s mission is to build multimodal AI to expand human imagination and capabil...

Location

United States; United Kingdom , Palo Alto; London

Salary:

187500.00 - 395000.00 USD / Year

Luma AI

Expiration Date

Until further notice

Requirements

Strong Python and system architecture skills
Experience with model deployment using PyTorch, Huggingface, vLLM, SGLang, tensorRT-LLM, or similar
Experience with queues, scheduling, traffic-control, fleet management at scale
Experience with Linux, Docker, and Kubernetes
Python
Redis
S3-compatible Storage
Model serving (one of: PyTorch, vLLM, SGLang, Huggingface)
Understanding of large-scale orchestration, deployment, scheduling (via Kubernetes or similar)

Job Responsibility

Ship new model architectures by integrating them into our inference engine
Collaborate closely across research, engineering and infrastructure to streamline and optimize model efficiency and deployments
Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
Automate, test and maintain our inference services to ensure maximum uptime and reliability
Optimize deployment workflows to scale across thousands of machines
Manage and optimize our inference workloads across different clusters & hardware providers
Build sophisticated scheduling systems to optimally leverage our expensive GPU resources while meeting internal SLOs
Build and maintain CI/CD pipelines for processing/optimizing model checkpoints, platform components, and SDKs for internal teams to integrate into our products/internal tooling

Fulltime

Principal ML Engineer, ML Platform Engineering

Xometry is seeking a Principal Machine Learning Engineer to join our core machin...

Location

United States , North Bethesda

Salary:

140000.00 - 182000.00 USD / Year

Cherry Ventures

Expiration Date

Until further notice

Requirements

At least 7 years of experience in machine learning engineering, software engineering, data science, or similar technical role
A bachelor’s degree is required, but an advanced degree (M.S. or PhD) in computer science, machine learning, AI, or a related field is preferred and may substitute for some years of experience
Demonstrated experience designing and deploying cloud infrastructure (AWS preferred) to support machine learning, and machine learning models, with considerations for scale, reliability and security
Deep understanding of the machine learning lifecycle and related infrastructure needs - feature stores, a/b testing, model registration, drift detection, automated retraining, etc
Strong technical expertise. You will need to either have or demonstrate the ability ability to quickly build technical expertise in the following: Software engineering principles, including parallel and distributed computing, version control, reproducibility, and continuous integration
Machine learning techniques and algorithms, with emphasis on their impact to infrastructure implementation Including large-scale language and vision models (Transformers, GPT, VLMs, LLMs), deep learning (PyTorch, Tensorflow)
Infrastructure as Code (IaC), especially Terraform
REST API design and implementation
Object oriented and functional programming in Python
Multimodal data processing (e.g., combining text, image, and 3D data)

Job Responsibility

Hands-On Technical Leadership: Adopt a 'lead by example' approach by actively coding and troubleshooting, as well as creating documentation and technical diagrams
Teaching & Mentorship: You will serve as a mentor and guide to engineers across the organization, teaching and mentoring them to grow their skills
Code Review: You will do code review and mentor others within the organization regarding best practices in ML Engineering
Operational Excellence: Guarantee the delivery of superior infrastructure and software that not only meets but exceeds customer expectations, while aligning with the strategic business timelines
Collaborative Strategy: Forge strong partnerships with product managers, data scientists, and company leadership to promote a culture of open communication and integrated team dynamics
Guide Innovation: Champion the adoption of cutting-edge technologies, methodologies, and practices to enhance problem-solving efficiency and effectiveness across the AI/ML organization.

What we offer

401(k) match
medical, dental and vision insurance
life and disability insurance
generous paid time off including vacation, sick leave, floating and fixed holidays, maternity and bonding leave
EAP, other wellbeing resources

Fulltime

Senior Machine Learning Engineer, Model Risk Management

Block lends, moves money, and screens for financial crime at enormous scale, and...

Location

United States , New York

Salary:

194500.00 - 343100.00 USD / Year

Cash App

Expiration Date

Until further notice

Requirements

A quantitative degree or equivalent experience, and senior-IC depth building or validating models in a high-stakes domain such as credit, fraud, or financial crime
Command of effective-challenge methodology: reproduction, conceptual-soundness review, benchmarking, stress testing, and outcomes analysis, with an eye for how a model holds up after launch and where its assumptions break
Deep applied ML and statistics across model families, from regression and tree ensembles to deep learning, with sound judgment about evaluation, calibration, and generalization
Experimentation and statistical rigor: holdout and experiment design, reasoning about uncertainty, and evaluating a model beyond aggregate accuracy
Solid software and data engineering: production-quality Python, SQL on large datasets, and reproducible, tested code
Fluency with modern AI: building with LLMs and agentic tools, and the judgment to know when their output can be trusted
Familiarity with model risk management frameworks and fair-lending standards, with the specifics learnable on the job
The communication to explain and defend your conclusions to model owners and senior stakeholders, and the independence to operate under ambiguity

Job Responsibility

Independently challenge model owners across lending, fraud, and AML: reproduce their results, set and defend the acceptance thresholds, and own the call on whether a model is sound
Hunt the silent errors that make metrics lie, and prove them out before they reach production
Choose evaluation that holds up under real conditions: rare events, shifting populations, and drift that only shows up after launch
Work hands-on in codebases you did not write, learning the data, configs, and conventions, and ship production code in the tooling you build to validate them
Build the agentic validation tooling the team depends on, orchestrating agents that run in parallel
Reason about ML systems end to end — how features, training, serving, monitoring, and scale fit together — to evaluate and challenge an owner's design
Tie explainability and fair-lending findings on consumer credit models back to the model and product decisions that follow
Help define how Block validates the systems at the frontier of production AI, setting standards where none exist yet

What we offer

Remote work
Medical insurance
Flexible time off
Retirement savings plans
Modern family planning

Fulltime

Senior ML Engineer

We are the global test and automation specialists, powering next-generation tech...

Location

United States , North Reading

Salary:

158600.00 - 253700.00 USD / Year

Teradyne

Expiration Date

Until further notice

Requirements

5+ years of experience in machine learning, applied AI, or related fields
Hands-on experience fine-tuning large language models
Experience with reinforcement learning (e.g., policy gradients, PPO, actor-critic methods)
Experience designing reward models or evaluation systems
Strong software engineering skills (Python, distributed systems familiarity)
Experience building production ML systems (MLOps, monitoring, deployment)
Ability to work cross-functionally with product, software, and hardware teams
Strong communication skills
comfortable engaging directly with customers & stakeholders
Computer vision skills in manufacturing inspection: defect detection, etc.

Job Responsibility

Own the technical direction and quality of all ML model development across Teradyne's AI initiatives, setting and enforcing engineering standards across the team
Lead end-to-end development of production ML systems: data ingestion and feature engineering, model architecture design, training pipelines, evaluation frameworks, and deployment
Design and implement novel ML approaches tailored to Teradyne's unique data domain including time-series parametric test data (STDF/TEMS), wafer map analysis, etc.
Drive applied research and model innovation, explore and evaluate new architectures, algorithms, and training methodologies, and translate promising approaches into production systems
Develop and maintain rigorous model evaluation frameworks, including validation methodologies, risk quantification, and production monitoring strategies
Lead technical design reviews
serve as final arbiter of ML architecture and modeling decisions for the team
Build and maintain production ML systems with a strong focus on reliability, scalability, and performance in Teradyne's ATE and manufacturing environments
Partner directly with customers and application engineers to understand real-world debug workflows and translate them into ML solutions
Mentor and develop junior ML engineers

What we offer

medical
dental
vision
Flexible Spending Accounts
retirement savings plans
life and disability insurance
paid vacation & holidays
tuition assistance programs

Fulltime

Senior Data Scientist / ML Engineer

We have a dream: to change industries through the power of digital technology. W...

Location

Bulgaria;Colombia;Croatia;Poland;Portugal;Spain;Ukraine

Salary:

Not provided

Intellias

Expiration Date

Until further notice

Requirements

Education: Bachelor’s or Master’s degree in Computer Science, Statistics, Mathematics, Data Science, or a closely related quantitative field
Experience: 5+ years of professional experience in machine learning engineering, AI development, or a closely related role
Machine Learning & Statistics: Solid understanding of classical ML algorithms (e.g., tree-based models, SVMs, clustering, ensemble methods), feature engineering, model evaluation metrics, and statistical methods (hypothesis testing, regression analysis, probability distributions)
LLM Expertise: Demonstrated project experience with large language models, including prompt engineering and prompt management strategies, LLM application development (end-to-end), fine-tuning of large language models, retrieval-augmented generation (RAG) pipeline design and implementation, practical experience with vector stores such as ChromaDB, pgvector, and PostgreSQL
AI Agents: Hands-on experience building AI agents and multi-agent systems using frameworks such as LangChain, LangGraph, CrewAI, or similar orchestration frameworks
Programming: Proficiency in Python with a strong emphasis on writing clean, maintainable, production-quality code. Familiarity with software engineering best practices (testing, code review, documentation)
Cloud: Practical experience with Google Cloud Platform (GCP) services for ML workloads (e.g., Vertex AI, Cloud Run, GCS, BigQuery, Compute Engine)
DevOps & MLOps: Docker: Proficiency in containerization — building, managing, and deploying Docker images and containers. GitLab: Proficient GitLab skills for version control, merge request workflows, and repository management
API Development: Experience with FastAPI, including request validation, async handling, and integration with ML model serving
Soft Skills: Excellent communication skills, Strong work ethic and high personal accountability, Ownership mentality — takes full responsibility for deliverables and outcomes, Proactive, self-starting approach to identifying problems and driving project success without waiting for direction

Job Responsibility

Drive/Participate the ideation, development, and execution of POCs and AI related project
Develop and implement machine learning models, algorithms, and data-driven solutions to address complex business problems
Collaborate cross-functionally with engineering, product management, and other relevant teams to integrate data-driven functionalities into our products

Fulltime

Senior / Lead Machine Learning Engineer, Serving

Inworld is a product-oriented research lab of top AI researchers and engineers, ...

Location

Germany

Salary:

Not provided

Inworld AI

Expiration Date

Until further notice

Requirements

Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM
Model Acceleration. Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs
Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections
Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups
Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production
Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems
Professional fluency in English (written and spoken) is required, as you will be collaborating daily with our US-based leadership and engineering teams

Staff / Principal Machine Learning Engineer, Serving

A year ago, reliably working agentic systems and sub-second multimodal inference...

Location

United States , Mountain View

Salary:

270000.00 - 500000.00 USD / Year

Inworld AI

Expiration Date

Until further notice

Requirements

Inference Optimization. Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM
Model Acceleration. Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding
High-Performance Systems. Proficiency in C++, CUDA, Rust, or highly optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs
Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections
Public work. Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups
Full-cycle ownership. You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production
Background. PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems

Job Responsibility

We hand you unclear problems and expect you to make them clear
We value engineers who say 'I don't know yet' and then design the benchmark or prototype that finds out
We treat performance, latency, and reliability as first-class product features, not a box to check before launch
Impact comes before everything else, though we support sharing work and open-source contributions that move the field forward
Your work should be visible

What we offer

bonus
equity
benefits
relocation assistance

Fulltime

Select Country

ML Model Serving Engineer

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?