Performance Engineer - Inference Job at Cerebras Systems (Toronto)

Senior LLM Inference Performance Engineer

WHAT YOU DO AT AMD CHANGES EVERYTHING At AMD, our mission is to build great prod...

Location

Finland , Helsinki

Salary:

Not provided

AMD

Expiration Date

Until further notice

Requirements

Seasoned in deploying LLMs and other AI model types in production using frameworks like vLLM, SGLang, or similar tooling
Deep knowledge about LLM serving and performance metric evaluation
Comfortable with Python software development and bash scripting
Experience with Docker, Kubernetes and Helm
Desire and ability to continuously learn in a fast-changing environment
Initiative, pragmatic problem solving, and great collaboration skills
Bachelor's or master's degree in computer science, computer engineering, electrical engineering, or an equivalent field

Job Responsibility

LLM and AI Performance: Measure, analyze, and optimize LLM and AI service performance across metrics like latency and throughput for various training and inference use cases
Design and implement methodologies for measuring model performance, and automating optimization strategies to identify optimal configurations
Stay on top of current advances in AI, models, APIs, and open-source ecosystems, and translate them into scalable solutions
LLM and AI Tooling: Design and develop tooling to measure and analyze the performance of AI model deployments and the effect of different configurations and infrastructure, standalone and Kubernetes clusters
Develop and maintain tooling for interacting with different ecosystem functions to improve developer and user experience
Develop and maintain internal tooling to support LLM and AI performance tuning at scale

Fulltime

LLM Inference Performance & Evals Engineer

Join the inference model team dedicated to bring up the state-of-the-art models,...

Location

Canada , Toronto

Salary:

Not provided

Cerebras Systems

Expiration Date

Until further notice

Requirements

3+ years building high-performance ML or systems software
Solid grounding in Transformer math—attention scaling, KV-cache, quantisation—or clear evidence you learn this material rapidly
Comfort navigating the full AI toolchain: Python modeling code, compiler IRs, performance profiling, etc.
Strong debugging skills across performance, numerical accuracy, and runtime integration
Prior experience in modeling, compilers or crafting benchmarks or performance studies
not just black-box QA tests
Strong passion to leverage AI agents or workflow orchestration tools to boost personal productivity

Job Responsibility

Prototype and benchmark cutting-edge ideas: new attentions, MoE, speculative decoding, and many more innovations as they emerge
Develop agent-driven automation that designs experiments, schedules runs, triages regressions, and drafts pull-requests
Work closely with compiler, runtime, and silicon teams: unique opportunity to experience the full stack of software/hardware innovation
Keep pace with the latest open- and closed-source models
run them first on wafer scale to expose new optimization opportunities

What we offer

Build a breakthrough AI platform beyond the constraints of the GPU
Publish and open source their cutting-edge AI research
Work on one of the fastest AI supercomputers in the world
Enjoy job stability with startup vitality
Our simple, non-corporate work culture that respects individual beliefs

Software Engineer II and Senior Software Engineer - Performance

The Artificial Intelligence Performance team at Microsoft develops AI software t...

Location

United States , Mountain View

Salary:

100600.00 - 199000.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Python OR equivalent experience
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter

Job Responsibility

Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs
Measure, benchmark performance on Nvidia/AMD GPUs and first party Microsoft silicon
Optimize and monitor performance of LLMs and build SW tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level to improve customer experience and reduce the footprint of the computing fleet
Enable fast time to market of LLMs/models and their deployments at scale by building SW tools that afford velocity in porting models on new Nvidia and AMD GPUs
Design, implement, and test functions or components for our AI/DNN/LLM frameworks and tools
Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems
Communicate and collaborate with our partners both internal and external
Embody Microsoft's Culture and Values

Fulltime

New

Lead Machine Learning Inference Engineer, Advertising

Roku is changing how the world watches TV. Roku is the #1 TV streaming platform ...

Location

United States , Austin

Salary:

Not provided

Roku

Expiration Date

Until further notice

Requirements

M.S. or above in CS, ECE, or a related field
10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role
Strong programming skills in high-performance languages
Deep understanding of inference frameworks and ML system deployment
Proven experience optimizing performance for large-scale machine learning systems, including a deep knowledge of SOTA model optimizations, hardware-software co-design, GPU acceleration, and HPC techniques
Excellent communication and collaboration skills
Experience leading teams working on high-throughput, low-latency ML serving systems
Experience collaborating with and leading global, cross-functional teams
Contributions to open-source ML or systems projects

Job Responsibility

Lead the design and development of a SOTA Inference platform
Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services
Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance
Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

What we offer

Global access to mental health and financial wellness support and resources
Healthcare (medical, dental, and vision) where applicable
Life, accident, disability, commuter, and retirement options (401(k)/pension) where applicable
Time off in accordance with local leave policies

Fulltime

Sr. Lead AI Engineer (Inference Optimization, FM hosting, AI Platform)

At Capital One, we are creating responsible and reliable AI systems, changing ba...

Location

United States , San Jose, California; San Francisco, California; New York, New York; Cambridge, Massachusetts; McLean, Virginia

Salary:

229900.00 - 286200.00 USD / Year

Capital One

Expiration Date

Until further notice

Requirements

Bachelor's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 6 years of experience developing AI and ML algorithms or technologies, or a Master's degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or related fields plus at least 4 years of experience developing AI and ML algorithms or technologies
At least 6 years of experience programming with Python, Go, Scala, or Java

Job Responsibility

Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that change how our associates work and how our customers interact with Capital One
Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability, etc.
Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more
Invent and introduce state-of-the-art LLM optimization techniques to improve the performance — scalability, cost, latency, throughput — of large scale production AI systems
Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One

What we offer

Cash bonus(es)
Long term incentives (LTI)
Comprehensive, competitive, and inclusive set of health, financial and other benefits that support your total well-being

Fulltime

Senior ML Inference Engineer - Platform

The Model Deployment & Inference Solutions team in GM AV deploys machine learnin...

Location

United States , Austin; Mountain View

Salary:

128700.00 - 261300.00 USD / Year

General Motors

Expiration Date

Until further notice

Requirements

BS, MS, or PhD in Computer Science or a related technical field
3+ years of relevant industry experience
Strong fundamentals and excellent coding ability in Python
Experience building or operating production platform or infrastructure systems where reliability, observability, and extensibility matter
Experience with ML model deployment, inference integration, model optimization workflows, or model serving infrastructure, with at least one prior context where you owned the path from a trained model to a running inference workload
Experience using coding agents (Cursor, Claude Code, GitHub Copilot, or equivalent) as part of your engineering workflow
Experience designing clean, well-tested software with clear interfaces and good abstractions
Strong cross-team collaboration skills

Job Responsibility

Design, build, and operate the ML deployment platform that automates the path from trained model to on-vehicle inference
Drive cross-organization model deployments to the autonomous vehicle stack, partnering with model development teams to take high-value models from training to production on-vehicle
Build agentic tools that diagnose and fix deployment-blocking issues, automating workflows currently performed manually by engineers
Build the developer experience that ML model development teams use day to day: tooling, dashboards, automation, and observability
Drive shift-left validation that surfaces deployment risk (compile, runtime, parity, latency) early in the model development cycle
Build platform tools that integrate the work of our sister teams (kernels, compiler, reduced precision and parity) so their optimization wins land directly in the deployment workflow
Partner with the team's Performance pillar and model development teams across the AV organization

What we offer

Medical
Dental
Vision
Health Savings Account
Flexible Spending Accounts
Retirement savings plan
Sickness and accident benefits
Life insurance
Paid vacation & holidays
Tuition assistance programs

Fulltime

AI Product Performance Engineer

WHAT YOU DO AT AMD CHANGES EVERYTHING. At AMD, our mission is to build great pro...

Location

China , Shenzhen

Salary:

Not provided

AMD

Expiration Date

Until further notice

Requirements

deep knowledge of Data Center AI workloads such as LLM, Generative AI, Recommendation, NLP, Video Analytics, and/or transformer
hands-on experiences with various AI models, end-to-end pipeline, industry framework / SDKs and solutions
GPU Architecture Mastery
Kernel Programming Expertise: Strong proficiency in C++ and parallel computing, with extensive hands-on experience in NVIDIA CUDA or AMD HIP kernel programming
Performance Engineering: Demonstrated ability to debug and profile complex GPU workloads
Systems Knowledge: Familiarity with asynchronous execution, stream management, and host-device memory transfers
Python DSLs & Triton: Experience implementing kernels using OpenAI Triton or other Python-based DSLs
Inference Engine Experience: Hands-on experience integrating custom kernels into large-scale inference frameworks such as vLLM, SGLang, or TensorRT-LLM
Deep Learning Frameworks: Familiarity with writing custom extensions or operators for PyTorch (C++/CUDA extensions)
Hardware Agnosticism: Experience porting kernels between NVIDIA and AMD architectures or working with cross-platform HPC libraries

Job Responsibility

High-Performance Kernel Development: Design, implement, and optimize high-performance GPU kernels for AI/ML workloads to maximize hardware utilization
Performance Optimization: Analyze and optimize kernel execution for latency and throughput, addressing bottlenecks in memory bandwidth, instruction latency, and thread divergence
Workload Analysis: Evaluate the end-to-end performance impact of individual kernels on full-stack AI models, ensuring that micro-optimizations translate to application-level speedups
Profiling & Tuning: Utilize advanced GPU profiling tools (e.g., ROCm Profiler, Pytorch Profiler) to identify performance cliffs, stall pipelines, and memory hierarchy inefficiencies
Architecture Adaptation: Tailor implementation strategies to leverage specific features of modern GPU architectures (e.g., Matrix Cores, HBM characteristics)
Framework Integration: Collaborate with software stack teams to expose optimized kernels within high-level frameworks and inference engines

What we offer

AMD benefits at a glance

Fulltime

AI/HPC System Performance Engineer

Meta is building some of the world's largest AI and high-performance computing i...

Location

United States , Menlo Park

Salary:

154000.00 - 217000.00 USD / Year

Select Country

Performance Engineer - Inference

Job Description

Job Responsibility

Requirements

What we offer

Looking for more opportunities?