CrawlJobs Logo
Cerebras Systems Logo Cerebras Systems · IT - Software Development

Senior Inference ML Runtime Engineer

United States; Canada, Sunnyvale · Job Posted February 17, 2026
Apply Position
Job Link Share

Job Description

The Inference ML Engineering team at Cerebras Systems is dedicated to enabling our fast generative inference solution through simple APIs powered by a distributed runtime that runs on large clusters of our own hardware. Our mission is to empower enterprises, developers, and researchers to unlock the full potential of our platform, leveraging its performance, scalability, and flexibility. The team works closely with cross-functional groups, including compiler developers, cluster orchestrators, ML scientists, cloud architects, and product teams, to deliver high-impact solutions that redefine the boundaries of ML performance and usability.

Job Responsibility

  • Drive and provide technical guidance to a team of software engineers working on complex machine learning integration projects
  • Design and implement ML features (e.g., structured outputs, biased sampling, predicted outputs) that improve performance of generative AI models at inference time
  • Design and implement high-throughput, low-latency multimodal inference models that support delivery of image, audio, and video inputs and outputs
  • Maintain our scalable serving backend for handling many concurrent requests per minute
  • Scale our inference service by implementing detailed observability throughout the entire stack
  • Analyze and improve latency, throughput, memory usage, and compute efficiency on the service and the implementation of various features
  • Optimize software to accelerate generative LLM inference by achieving high throughput and low latency
  • Stay up-to-date with advancements in machine learning and deep learning, and apply state-of-the-art techniques to enhance our solutions
  • Evaluate trade-offs between different approaches, clearly articulate design choices, and develop detailed proposals for implementing new features
  • Uncover, scope, and prioritize significant areas of technical debt across the software stack to ensure continued high quality of the inference service
  • Build and maintain robust automated test suites to ensure software quality, performance, and reliability
  • Contribute to an agile team environment by delivering high-quality software and adhering to agile development practices
  • Lead cross-functional initiative across the company to deliver high-quality inference solutions

Requirements

  • Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Mathematics, or a related field
  • 8+ years of experience in large-scale software engineering, with a focus on deep learning or related domains
  • Proficiency in Python for building and maintaining scalable systems
  • Advanced proficiency in C++, with an emphasis on multi-threaded programming, performance optimization, and system-level development
  • Demonstrated experience driving cross-functional projects
  • Experience building and scaling large-scale inference systems for LLMs or multimodal models
  • Familiarity with LLM serving frameworks, such as vLLM, SGLang, and TensorRT-LLM
  • Solid understanding of software architectural patterns for large-scale, high-performance applications
  • Hands-on experience with ML frameworks, such as PyTorch, and a strong understanding of their underlying architectures
  • Strong problem-solving skills, with the ability to balance technical depth with practical implementation constraints
  • Exceptional communication and presentation skills, with the ability to work both independently and collaboratively across multidisciplinary teams

What we offer

  • Build a breakthrough AI platform beyond the constraints of the GPU
  • Publish and open source their cutting-edge AI research
  • Work on one of the fastest AI supercomputers in the world
  • Enjoy job stability with startup vitality
  • Our simple, non-corporate work culture that respects individual beliefs
Cerebras Systems - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Inference ML Runtime Engineer

8 matching positions

Senior ML Inference Engineer - Platform

The Model Deployment & Inference Solutions team in GM AV deploys machine learnin...
Location
Location
United States , Austin; Mountain View
Salary
Salary:
128700.00 - 261300.00 USD / Year
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BS, MS, or PhD in Computer Science or a related technical field
  • 3+ years of relevant industry experience
  • Strong fundamentals and excellent coding ability in Python
  • Experience building or operating production platform or infrastructure systems where reliability, observability, and extensibility matter
  • Experience with ML model deployment, inference integration, model optimization workflows, or model serving infrastructure, with at least one prior context where you owned the path from a trained model to a running inference workload
  • Experience using coding agents (Cursor, Claude Code, GitHub Copilot, or equivalent) as part of your engineering workflow
  • Experience designing clean, well-tested software with clear interfaces and good abstractions
  • Strong cross-team collaboration skills
Job Responsibility
Job Responsibility
  • Design, build, and operate the ML deployment platform that automates the path from trained model to on-vehicle inference
  • Drive cross-organization model deployments to the autonomous vehicle stack, partnering with model development teams to take high-value models from training to production on-vehicle
  • Build agentic tools that diagnose and fix deployment-blocking issues, automating workflows currently performed manually by engineers
  • Build the developer experience that ML model development teams use day to day: tooling, dashboards, automation, and observability
  • Drive shift-left validation that surfaces deployment risk (compile, runtime, parity, latency) early in the model development cycle
  • Build platform tools that integrate the work of our sister teams (kernels, compiler, reduced precision and parity) so their optimization wins land directly in the deployment workflow
  • Partner with the team's Performance pillar and model development teams across the AV organization
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • Health Savings Account
  • Flexible Spending Accounts
  • Retirement savings plan
  • Sickness and accident benefits
  • Life insurance
  • Paid vacation & holidays
  • Tuition assistance programs
  • Fulltime
Read More
Arrow Right

Senior ML Engineer- Distillation

A seasoned Senior ML Engineer who drives distillation of ML Models for high-perf...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
amd.com Logo
AMD
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6–10+ years in ML engineering or applied research
  • 3+ years focused on model distillation/compression at production scale
  • Strong proficiency in PyTorch (preferred) or JAX/TF
  • Ability to implement custom training loops, distributed training, and mixed precision
  • Demonstrated experience shipping distilled or compressed models to production with measurable gains in latency/memory and maintained quality
  • Deep understanding of knowledge distillation techniques: teacher–student frameworks, soft-labels, intermediate feature matching, contrastive distillation, task-specific loss shaping
  • Hands-on experience with quantization (static/dynamic, PTQ/QAT), pruning, and graph-level optimizations (operator fusion)
  • GPU performance engineering: CUDA fundamentals, TensorRT/ONNX Runtime, kernel profiling (Nsight), memory/layout optimization
  • Solid grasp of computer graphics fundamentals: rendering pipeline, shaders, sampling, anti-aliasing, tone mapping, and perceptual metrics
  • Strong software engineering: Python/C++ proficiency, testing, code quality, version control, reproducible pipelines, containerization
Job Responsibility
Job Responsibility
  • Distillation and compression: KD variants, hint/fitnets, attention transfer, feature mimicking, low-rank/SVD, sparsity
  • Efficient architectures: MobileNet/EfficientNet, vision transformers optimization, lightweight diffusion/UNet variants, NeRF/instant-NGP distillation
  • Inference optimization: TensorRT, CUDA, cuDNN, ONNX, quantization-aware training, weight clustering, operator fusion
  • Metrics: SSIM, LPIPS, PSNR, FID/KID, latency/throughput profiling, memory/activation footprint analysis
  • Data and training: large-scale dataset curation, synthetic data generation, curriculum learning, augmentation strategies
  • MLOps: experiment tracking, CI/CD for models, model registries, reproducibility, telemetry
  • Integrate ML inference into production rendering pipelines: define model I/O, preprocessing/postprocessing, and make trade-offs for latency, throughput, and quality
  • Collaborate across teams (ML researchers, engine/platform, tooling, QA) to translate ML and product requirements into graphics-friendly implementations and integration plans
  • Mentor other engineers, conduct code reviews, and help define best practices for rendering, performance, and SDK delivery
Read More
Arrow Right

Senior ML Accelerator Engineer - GPU

About the Mission: GM’s vision of Zero Crashes, Zero Emissions, and Zero Congest...
Location
Location
United States , Austin
Salary
Salary:
128700.00 - 261300.00 USD / Year
gm.com Logo
General Motors
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 3+ years of relevant industry experience or equivalent experience
  • BS, MS or PhD in CS, or related technical field
  • Excellent GPU programming skills in CUDA, with a thorough understanding of parallel programming patterns and GPU architecture
  • Hands-on experience benchmarking, profiling, debugging and optimizing accelerator libraries and kernels to extract optimal performance using the NSight suite of tools or similar
  • Strong background in software architecture, library design, and design patterns
  • Strong C++ programming skills with the ability to feel comfortable in large codebases
  • Solid background in system performance, high performance computing and/or architecture-aware optimizations
  • Strong communication skills and the ability to work collaboratively within a team
  • Excellent analytical and problem-solving skills
Job Responsibility
Job Responsibility
  • Design, implement, benchmark, and iterate on CUDA-based kernels and custom operators to squeeze every last drop of performance out of on-vehicle inference workloads
  • Build and improve tooling and infrastructure that make it easier to profile, debug, and validate CUDA kernels and accelerator-backend code across the AV stack
  • Partner with AI Solutions, Compilers, and Architecture to translate model and system requirements into concrete kernel roadmaps, priorities, and project plans
  • Collaborate with cross-functional teams (compiler, performance tooling, runtime, deployment solutions) to deliver reusable, reliable, high-performance libraries into production
  • Maintain high technology standards, methodologies, processes, and guidelines for GPU kernel development and performance engineering through code review
  • Manage relationships with internal customers to ensure our kernels and libraries meet real-world needs
What we offer
What we offer
  • medical
  • dental
  • vision
  • Health Savings Account
  • Flexible Spending Accounts
  • retirement savings plan
  • sickness and accident benefits
  • life insurance
  • paid vacation & holidays
  • tuition assistance programs
  • Fulltime
Read More
Arrow Right

Senior Software Engineer(Ads)

Online Advertising is one of the fastest‑growing businesses on the Internet. Mic...
Location
Location
China , Beijing
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 4+ years' experience building high‑performance, large‑scale distributed systems or ML infrastructure
  • Expert‑level proficiency in C++, with strong understanding of data structures, algorithms, and system design
  • Experience building and optimizing performance‑critical production systems
  • Experience working in Ads, Search, Recommendation systems, or other large‑scale online serving systems
Job Responsibility
Job Responsibility
  • Design and build a unified inference platform for Ads, ensuring scalability, reliability, and efficiency
  • Optimize model inference via batching, quantization, scheduling, memory management, runtime optimization, and other performance improvements
  • Develop, optimize, and maintain performance‑critical components for high‑throughput, low‑latency production inference, including GPU‑accelerated paths when applicable
  • Collaborate with algorithm/model teams to co‑design serving‑aware model architectures and optimizations
  • Profile and improve end‑to‑end system performance: concurrency, memory footprint, throughput, and latency
  • Provide senior technical leadership across teams
  • elevate engineering best practices and influence long‑term technical strategy
  • Fulltime
Read More
Arrow Right

Senior AI Engineer - MSC AI Innovation

MSC AI Innovation is an AI-first team that incubates, builds, and accelerates so...
Location
Location
Israel , Tel Aviv, Herzliya
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years professional software development
  • 4+ years of software engineering experience in the AI space (e.g., building and shipping AI/ML or GenAI features in production)
  • Proven experience with building AI agents
  • Hands-on experience with evaluation methodologies and integrating quality standards/guardrails into delivery
  • Proficiency in Python and/or C#, with experience using REST APIs and SDKs
  • Deep understanding of AI system design, including ML fundamentals, Generative AI concepts, and cloud-native architectures
Job Responsibility
Job Responsibility
  • Design and build AI agents that plan, use tools/APIs, manage state/memory, and reliably complete multi-step workflows
  • Own AI features from design through production, including deployment, monitoring, and live‑site reliability, with an eval-first development lifecycle: define success criteria, build evaluation datasets and automated harnesses, and run human-in-the-loop reviews where needed
  • Develop and maintain prompt, retrieval, and memory strategies (system prompts, few-shot examples, tool schemas, retrieval context) with proper versioning and evaluation coverage
  • Debug AI behavior using prompt analysis, data inspection, and model/tool-call traces, and translate failure patterns into targeted improvements
  • Establish and track AI quality metrics (e.g., accuracy, groundedness, relevance, hallucination rate) and integrate them into CI/CD release gates
  • Optimize runtime performance and economics (token usage, inference cost, latency, caching, model selection/routing, batching) and implement monitoring and continuous improvement loops (online signals, drift detection, structured user feedback)
  • Partner with product, design, and domain stakeholders to define use cases, acceptance criteria, and rollout plans for AI features
  • Live site responsibility
  • Fulltime
Read More
Arrow Right

Senior AI Backend Engineer

As a core member of our AI Engineering team, you will collaborate with data scie...
Location
Location
Israel , Tel-Aviv
Salary
Salary:
Not provided
khealth.com Logo
K Health
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of backend engineering experience
  • Strong proficiency in more than one major programming language (such as Python, Java, Go, Rust, or Kotlin)
  • Solid understanding of AI systems architecture and experience working in environments involving AI agents, LLMs, or inference pipelines
  • Proven experience in building and scaling backend APIs, microservices, and background jobs
  • Strong experience with relational and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB, Redis), including schema design, data modeling, and query optimization
  • Experience with message brokers and distributed systems (e.g., Pub/Sub, Redis Streams, RabbitMQ)
  • Highly adaptable, a strong team player, with the ability to quickly learn and apply new technologies
Job Responsibility
Job Responsibility
  • Build and maintain reliable, scalable backend services to support AI agent execution and orchestration
  • Develop AI agent systems for complex operational workflows using LangChain, LangGraph, LiteLLM, and Langfuse
  • Orchestrate a hybrid model stack that includes OpenAI and Google Gemini alongside self-hosted and fine-tuned LLMs like Gemma and Llama
  • Build and maintain integrations with clinical systems (FHIR, EMR)
  • Drive observability and reliability using OpenTelemetry, Datadog, and Langfuse
  • Design APIs (GraphQL, REST), background workers, and event-driven systems that interface with AI inference engines and agent runtimes
  • Collaborate with Data Science, ML, and engineering teams to deploy AI features and improve the performance, scalability, and reliability of backend systems
  • Participate in code reviews, knowledge sharing, and mentoring to elevate the team’s technical capabilities
What we offer
What we offer
  • Competitive compensation packages based on industry benchmarks for function, level, and geographic location
Read More
Arrow Right

Senior Software Development Engineer in Test (SDET) - AI Cluster Networking and Security

In AI infrastructure organization, simplifying large hardware deployments with p...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
cerebras.net Logo
Cerebras Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or master's degree in engineering in computer science, electrical, AI, data science of related field
  • 10+ years of experience in testing one of areas like enterprise software, distributed systems, datacenter hardware and software
  • Experience working in large enterprise or cloud networking infrastructure, high speed switches, routers, firewalls
  • Experience in qualifying networking vendor platforms like Juniper, Arista or Cisco and network test equipment like Ixia/Spirent
  • Experience in Datacenter technology like BGP, ECN, PFC
  • Experience testing networking security, compliance and firewalls
  • Strong coding skills in one of the programming languages like python, golang or C/C++
  • Strong debugging skills to debug issues in large distributed systems, hardware, and software. Experience with debugging tools like gdb, strace, networking monitors
  • Strong understanding of operating systems internals like memory management, file system working, security basics and performance
  • Strong understanding of datacenter layout, device performance characteristics like PCIe, networking and storage
Job Responsibility
Job Responsibility
  • Innovate and execute tests on cutting edge AI infrastructure
  • Define optimized test strategies and methodologies
  • Be a quick learner, adapt to new technologies
  • Build a strong understanding of how to break these large distributed systems challenge into smaller components that can be unit tested
  • Automate first approach - Aim for 100% automated tests to test all cluster features in areas of high availability, failure scenarios, performance, stress and security
  • Champion cluster security, reliability for uptime of 99.9999% and ease of use with observability
  • Test all components of AI cluster including but not limited to cluster software involving kubernetes, prometheus and grafana. Cluster hardware components like ML wafer scale accelerators, CPU runtime nodes, High speed swarmx interconnect, High speed data transfer of weights through memoryx interconnect
  • Qualify cluster networking solutions which consists of high-speed switches, routers and optics from various vendors
  • Qualify cluster security features including OS security, network security, cloud compliance user access and security certifications
What we offer
What we offer
  • Build a breakthrough AI platform beyond the constraints of the GPU
  • Publish and open source their cutting-edge AI research
  • Work on one of the fastest AI supercomputers in the world
  • Enjoy job stability with startup vitality
  • Our simple, non-corporate work culture that respects individual beliefs
Read More
Arrow Right

Senior AI/ML Validation Engineer

We are seeking an experienced and versatile professional with expertise in valid...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amd.com Logo
AMD
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–12 years in QA/Test for systems software or platform engineering, with at least 4 years focused on GPU software, device drivers, or firmware validation
  • Demonstrable ownership of validation for AI/ML pipelines and serving stacks using PyTorch and at least one modern inference framework (e.g., vLLM), including accuracy baselining and performance regression detection
  • Proven expertise testing drivers and firmware with hands-on work in: PCIe fundamentals (link training, BARs, MSI/MSI-X), DMA engines, interrupt handling, and memory models
  • Failure modes: error injection, recovery paths, power/thermal events, and persistence across reboot/upgrade cycles
  • Deep proficiency in Linux (kernel/user space) and practical experience with Windows driver ecosystems
  • ability to: Read kernel logs and symbols, trace with ftrace/perf/ETW, and perform cross-layer debugging
  • Build custom kernels/modules and analyze crash dumps (kdump, WinDbg)
  • Strong programming for test automation: Python for framework and orchestration (pytest or equivalent), robust mocking/fixtures, and data-driven test generation
  • C/C++ for low-level test harnesses, protocol exercisers, and performance micro-benchmarks
  • Bash/PowerShell for environment setup, CI scripting, and reproducibility
Job Responsibility
Job Responsibility
  • Own end-to-end test strategy for AI/ML workflows (PyTorch, vLLM), GPU runtimes, drivers, and firmware across kernel and user space
  • Develop scalable automation frameworks spanning unit, integration, HIL (hardware-in-the-loop), system, and end-to-end tests
  • Implement and maintain CI quality gates (GitHub Actions/Workflows, Jenkins), including automated build, test execution, artifact management, reporting, and flake reduction
  • Design and execute performance, stress, reliability, soak, and long-haul tests targeting GPU compute, memory, I/O, and serving throughput/latency
  • Validate cross-platform compatibility (Linux/Windows), covering driver interfaces, kernel interactions, firmware behavior, and runtime stability
  • Create reproducible environments with containers/orchestration
  • instrument telemetry and observability for data-driven QA
  • Apply agentic AI techniques to accelerate test generation, triage, and root cause analysis
  • integrate intelligent diagnostics into pipelines
  • Develop rigorous test cases for low-level features (PCIe, DMA, interrupts, memory management), error handling, recovery, and fault injection
What we offer
What we offer
  • AMD benefits at a glance
Read More
Arrow Right