CrawlJobs Logo

Head of Inference Kernels

United States, San Jose 200000.00 - 300000.00 USD / Year · Job Posted February 18, 2026
Apply Position
Job Link Share

Job Description

As a core member of the team, you will play a pivotal role in leading a high-performing team to build a suite of optimized kernels and implement highly optimized inference stacks for a variety of state-of-the-art transformer models (e.g., Llama-3, Llama-4, Deepseek-R1, Qwen-3, Stable Diffusion-3 etc.). You will be responsible for managing and scaling a high-performance team to pioneer novel model mapping strategies, while co-designing inference time algorithms (e.g., speculative and parallel decoding, prefill-decode disaggregation etc.).

Job Responsibility

  • Architect Best-in-Class Inference Performance on Sohu: Deliver continuous batching throughput exceeding B200 by ≥10x on priority workloads
  • Develop Best-in-Performance Inference Mega Kernels: Develop complex, fused kernels that increase chip utilization and reduce inference latency, and validate these optimizations through benchmarking and regression-tested in production pipelines
  • Architect Model Mapping Strategies: Develop system level optimizations using a mix of techniques such tensor parallelism and expert parallelism for optimal performance
  • Hardware-Software Co-design of Inference-time Algorithmic Innovation: Develop and deploy production-ready inference-time algorithmic improvements (e.g., speculative decoding, prefill-decode disaggregation, KV cache offloading)
  • Build Scalable Team and Roadmap: Grow and retain a team of high-performing inference optimization engineers
  • Cross-Functional Performance Alignment: Ensure inference stack and performance goals are aligned with the software infrastructure teams, GTM and hardware teams for future generations of our hardware

Requirements

  • Experience in designing and optimizing GPU kernels for deep learning on GPUs using CUDA, and assembly (ASM)
  • Experience with low-level programming to maximize performance for AI operations, leveraging tools like Compute Kernel (CK), CUTLASS, and Triton for multi-GPU and multi-platform performance
  • Deep fluency with transformer inference architecture, optimization levers, and full-stack systems (e.g., vLLM, custom runtimes)
  • History of delivering tangible perf wins on GPU hardware or custom AI accelerators
  • Solid understanding of roofline models of compute throughput, memory bandwidth and interconnect performance
  • Experienced in running large-scale workloads on heterogeneous compute clusters, optimizing for efficiency and scalability of AI workloads
  • Scopes projects crisply, sets aggressive but realistic milestones, and drives technical decision-making across the team
  • Anticipates blockers and shifts resources proactively

Nice to have

  • Experience with implementation of state-of-the-art reasoning and chain-of-thought models at production scale
  • Experience with implementation of newer AI compute operations on hardware (e.g., flash attention, long-context attention variants and alternatives)
  • Analyzed and implemented strategies such as KV-cache offloading for efficient compute resource management
  • Familiarity with linear algebra (e.g. matrix decomposition, alternatives bases for vector spaces, matrix rank and its implications)
  • Managed lean, high-performing engineering teams and drove execution on timelines with high quality outcomes

What we offer

  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • significant equity package

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Head of Inference Kernels

8 matching positions

Engineering Manager - Inference

We are looking for an Inference Engineering Manager to lead our AI Inference tea...
Location
Location
United States , San Francisco
Salary
Salary:
300000.00 - 385000.00 USD / Year
perplexity.ai Logo
Perplexity
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Job Responsibility
Job Responsibility
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
What we offer
What we offer
  • Equity
  • Health
  • Dental
  • Vision
  • Retirement
  • Fitness
  • Commuter and dependent care accounts
  • Fulltime
Read More
Arrow Right

Inside Sales Supervisor

Reporting to Regional Management, the Supervisor ensures the daily management of...
Location
Location
Canada , Laval
Salary
Salary:
65000.00 - 75000.00 CAD / Year
https://www.randstad.com Logo
Randstad
Expiration Date
July 23, 2026
Flip Icon
Requirements
Requirements
  • Solid experience in personnel management (essential)
  • Technical knowledge in automation, pneumatics, or industrial electronics
  • Perfect bilingualism (French and English), both spoken and written
  • Proficiency with computer tools (knowledge of a CRM or ERP is an asset)
  • Diplomatic leadership, great listening skills, and agility in the face of change
Job Responsibility
Job Responsibility
  • Directly supervise the internal team (coaching, training, performance reviews)
  • Manage Key Performance Indicators (KPIs) and analyze daily margin reports
  • Take charge of sensitive client accounts to ensure their stabilization
  • Actively participate in the continuous improvement of operational processes
  • Collaborate closely with the external sales team
  • Provide direct operational support during high-traffic periods
What we offer
What we offer
  • Competitive salary between $65,000 and $75,000 depending on experience
  • Comprehensive group insurance (100% medication coverage)
  • Group RRSP with employer contribution up to 2%
  • 5 sick/family days per year
  • Vacation accessible in the first year (no accrual period needed)
  • Stimulating work environment and free coffee on-site
  • Fulltime
Read More
Arrow Right

Senior Inside Sales Representative

Location
Location
Canada , Laval
Salary
Salary:
55000.00 - 65000.00 CAD / Year
https://www.randstad.com Logo
Randstad
Expiration Date
July 23, 2026
Flip Icon
Requirements
Requirements
  • Minimum of 5 to 10 years of experience in technical commercial sales or B2B
  • Excellent knowledge of plumbing products (a major asset)
  • Bilingualism needed (many customers and suppliers are outside Quebec)
  • Outgoing, energetic personality with great interpersonal skills
  • Basic computer proficiency (Excel, ERP systems)
Job Responsibility
Job Responsibility
  • Assurer l'ouverture du magasin et le service au comptoir de 6h00 à 15h00
  • Préparer des soumissions techniques et traiter les commandes entrantes avec précision
  • Agir comme point d'escalade pour les problématiques clients complexes
  • Partager ses connaissances techniques et coacher les membres plus juniors de l'équipe
  • Identifier les opportunités de ventes croisées (upsell/cross-sell) pour répondre aux besoins réels des clients
  • Maintenir des relations de confiance à long terme avec une clientèle d'entrepreneurs exigeante
What we offer
What we offer
  • Profit-sharing bonus (annual program)
  • Comprehensive benefits package and pension plan with employer contributions
  • 3 to 4 weeks of paid vacation (including the construction holiday and the holiday season)
  • Stable daytime schedule: Monday to Friday, 6:00 AM to 3:00 PM (40 hours/week)
  • On-site position in Laval with free parking—say goodbye to downtown traffic
  • Access to a full gym on-site and a corporate culture focused on well-being
  • Fulltime
Read More
Arrow Right

Site Manager

Location
Location
Malaysia , Johor Bahru
Salary
Salary:
Not provided
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven ability to lead, manage, and motivate site-based engineering teams
  • Highly motivated, proactive, and dynamic with a strong work ethic
  • Ability to work under pressure and manage a high workload in fast-paced environments
  • Strong working knowledge of Microsoft Office Suite (Word, Excel, Project, PowerPoint, Outlook)
  • Excellent communication skills, both written and verbal
  • Confident in client-facing interactions, with the ability to understand and manage client requirements
  • Demonstrable experience within the IT industry, specifically in data centre installations
  • Solid understanding of structured cabling installation practices and standards
  • Strong organisational skills with the ability to plan, coordinate, and manage teams on site
Job Responsibility
Job Responsibility
  • Oversee a team of supervisors and engineers to ensure all cable installations are installed to specification, on time and meet Onnec Group and Manufacturers quality standards
  • Ensure all current Health & Safety requirements are met, carrying out Risk Assessments, drafting method statements and ensuring all H&S procedures are followed
  • Ensure stock is managed in a cost effective manner and that there are sufficient materials available at all times to ensure efficient installation
  • Order materials as required
  • Compile progress reports as required and / or provide information in a logical format to the Project Manager
  • Carry out quality checks on all works carried out by the cabling engineers and ensure all required Quality Standards are met
  • Read and understand all site drawings, update 'as and when' when work is completed
  • Attend and document client meetings and ensure all required actions are carried out in a timely manner
  • Compile process and produce documentation as required to meet company and client standards
  • Draw up patching schedules and ensure timely implementation of any patching requirements
  • Fulltime
Read More
Arrow Right

Associate Mental Health Therapist

Our Associate Program is designed for therapists who are committed to providing ...
Location
Location
United States
Salary
Salary:
31.00 USD / Hour
foresightmentalhealth.com Logo
Foresight Mental Health
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • California residents who hold an active Associate California Clinical Social Workers (ACSW), Associate Marriage and Family Therapists (AMFT), or Associate Professional Clinical Counselors (APCC) license
  • Associate therapists who have at least 750 clinical hours left to complete
  • At least 6 months of experience, especially providing outpatient mental health services to families and children
  • Legally authorized to work in the United States
  • Willing to undergo a background check
  • Ability to commit to schedule requirements
  • Willing to provide services to children, adolescents, couples, and families
Job Responsibility
Job Responsibility
  • Provide attuned psychotherapy across the lifespan, including children, adolescents, adults, couples, and families
  • Manage your caseload, including schedule maintenance, charge slips, high balances, and consistent client attendance
  • Collaborate with an interdisciplinary care team, including psychiatrists and mental health nurse practitioners
  • Get hands-on experience developing assessments, diagnoses, treatment plans, and progress notes
What we offer
What we offer
  • Comprehensive Benefits for full-time employees: Medical, dental, vision insurance
  • 4 weeks PTO
  • paid holidays
  • 401(k) match, HSA/FSA, company-paid life insurance, and more
  • Ongoing professional development, clinical supervision, consultation groups, and a dedicated support team
  • Free in-house Continuing Education
  • 2 units of supervision provided by an independently licensed therapist
  • Real human support through live trainings, in-house IT, and operations teams
  • Fulltime
Read More
Arrow Right

Payroll/ HR Clerk

We are looking for a detail-oriented Payroll/ HR Clerk to support payroll and be...
Location
Location
United States , Newark
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3 years of experience supporting payroll, benefits, or human resources administration in an office environment
  • Working knowledge of payroll review processes and the ability to identify discrepancies in employee data
  • Proficiency in Microsoft Excel, including sorting, filtering, and managing spreadsheets
  • Ability to manage multiple administrative tasks efficiently while meeting deadlines
Job Responsibility
Job Responsibility
  • Review payroll information for completeness and accuracy before processing to help ensure timely and correct employee payment records
  • Support benefits administration tasks by maintaining employee files, updating records, and assisting with related documentation
  • Coordinate online training assignments and monitor completion status to keep required learning records current
  • Facilitate candidate screening activities by tracking submissions, following up on outstanding items, and organizing results
  • Use Excel to sort, filter, and manage spreadsheets that support payroll, benefits, and personnel reporting needs
  • Perform clerical duties such as filing, scanning, data entry, and record maintenance with a high level of accuracy
  • Assist with HR system administration, including tasks connected to ADP Workforce Now, as part of ongoing department support
  • Maintain organized documentation and respond to routine administrative requests from internal stakeholders in a timely manner
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Read More
Arrow Right

Asbestos Surveyor

Location
Location
United Kingdom , Isle of Wight
Salary
Salary:
Not provided
mitie.com Logo
Mitie Cleaning & Hygiene Services
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • BOHS P402 qualified (or equivalent)
  • Strong knowledge of asbestos surveying across domestic, commercial and/or industrial sectors
  • Full UK driving licence
  • Committed to delivering high standards while maintaining a healthy work‑life balance
Job Responsibility
Job Responsibility
  • Conduct asbestos sampling, reinspections, Management, Refurbishment and Demolition surveys on a wide range of domestic, commercial and industrial properties in accordance with HSG264 and company procedures
  • Collect site data and transport asbestos samples to UKAS‑accredited laboratories approved by the company
  • Complete asbestos survey reports and drawings using company software
  • Build and maintain client relationships, providing specialist advice and technical support where required
  • Liaise with the Asbestos Surveying Quality team, Contracts team, Project Managers, Supervisors and clients as required, while setting a personal example and promoting the company in a professional manner at all times
What we offer
What we offer
  • Minimal overnight stays and localised work wherever possible
  • Clear career progression with structured pathways into Senior Surveyor, Project Management and Auditing roles
  • Lots of progression and development opportunities
  • Real Living Wage Employer
  • Virtual GP
  • Financial wellbeing assistance through Salary Finance scheme
  • Flexible lifestyle benefits platform (Choices)
  • Purchase up to five extra days’ holiday each year
  • Critical illness insurance
  • Dental treatment
  • Fulltime
Read More
Arrow Right

Water Technician - Level 2

At Marlowe Environmental Services - Part of a Mitie Company, our people are our ...
Location
Location
United Kingdom , Horley
Salary
Salary:
Not provided
mitie.com Logo
Mitie Cleaning & Hygiene Services
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Water Hygiene Level 2 qualification (or equivalent)
  • Awareness of ACOP L8, HSG274 and water safety guidance
  • Full UK driving licence
Job Responsibility
Job Responsibility
  • Carry out water hygiene and water treatment tasks in line with industry standards, legislation and Mitie procedures
  • Complete PPM and remedial works including temperature monitoring, sampling and testing
  • Maintain accurate records and reports for compliance and auditing purposes
  • Identify and report risks, defects or non‑compliances, escalating where appropriate
  • Work collaboratively with colleagues and customers to deliver a high‑quality, professional service
What we offer
What we offer
  • Company Van and Equipment
  • Real Living Wage Employer
  • Virtual GP
  • financial wellbeing assistance through Salary Finance
  • flexible lifestyle benefits platform
  • Cycle to Work Scheme
  • Employee Shares Schemes
  • Salary Finance Options
  • Holiday
  • Fulltime
Read More
Arrow Right