CrawlJobs Logo

Head of Inference Kernels

etched.com Logo

Etched

Location Icon

Location:
United States , San Jose

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

200000.00 - 300000.00 USD / Year

Job Description:

As a core member of the team, you will play a pivotal role in leading a high-performing team to build a suite of optimized kernels and implement highly optimized inference stacks for a variety of state-of-the-art transformer models (e.g., Llama-3, Llama-4, Deepseek-R1, Qwen-3, Stable Diffusion-3 etc.). You will be responsible for managing and scaling a high-performance team to pioneer novel model mapping strategies, while co-designing inference time algorithms (e.g., speculative and parallel decoding, prefill-decode disaggregation etc.).

Job Responsibility:

  • Architect Best-in-Class Inference Performance on Sohu: Deliver continuous batching throughput exceeding B200 by ≥10x on priority workloads
  • Develop Best-in-Performance Inference Mega Kernels: Develop complex, fused kernels that increase chip utilization and reduce inference latency, and validate these optimizations through benchmarking and regression-tested in production pipelines
  • Architect Model Mapping Strategies: Develop system level optimizations using a mix of techniques such tensor parallelism and expert parallelism for optimal performance
  • Hardware-Software Co-design of Inference-time Algorithmic Innovation: Develop and deploy production-ready inference-time algorithmic improvements (e.g., speculative decoding, prefill-decode disaggregation, KV cache offloading)
  • Build Scalable Team and Roadmap: Grow and retain a team of high-performing inference optimization engineers
  • Cross-Functional Performance Alignment: Ensure inference stack and performance goals are aligned with the software infrastructure teams, GTM and hardware teams for future generations of our hardware

Requirements:

  • Experience in designing and optimizing GPU kernels for deep learning on GPUs using CUDA, and assembly (ASM)
  • Experience with low-level programming to maximize performance for AI operations, leveraging tools like Compute Kernel (CK), CUTLASS, and Triton for multi-GPU and multi-platform performance
  • Deep fluency with transformer inference architecture, optimization levers, and full-stack systems (e.g., vLLM, custom runtimes)
  • History of delivering tangible perf wins on GPU hardware or custom AI accelerators
  • Solid understanding of roofline models of compute throughput, memory bandwidth and interconnect performance
  • Experienced in running large-scale workloads on heterogeneous compute clusters, optimizing for efficiency and scalability of AI workloads
  • Scopes projects crisply, sets aggressive but realistic milestones, and drives technical decision-making across the team
  • Anticipates blockers and shifts resources proactively

Nice to have:

  • Experience with implementation of state-of-the-art reasoning and chain-of-thought models at production scale
  • Experience with implementation of newer AI compute operations on hardware (e.g., flash attention, long-context attention variants and alternatives)
  • Analyzed and implemented strategies such as KV-cache offloading for efficient compute resource management
  • Familiarity with linear algebra (e.g. matrix decomposition, alternatives bases for vector spaces, matrix rank and its implications)
  • Managed lean, high-performing engineering teams and drove execution on timelines with high quality outcomes
What we offer:
  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • significant equity package

Additional Information:

Job Posted:
February 18, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Head of Inference Kernels

Engineering Manager - Inference

We are looking for an Inference Engineering Manager to lead our AI Inference tea...
Location
Location
United States , San Francisco
Salary
Salary:
300000.00 - 385000.00 USD / Year
perplexity.ai Logo
Perplexity
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Job Responsibility
Job Responsibility
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
What we offer
What we offer
  • Equity
  • Health
  • Dental
  • Vision
  • Retirement
  • Fitness
  • Commuter and dependent care accounts
  • Fulltime
Read More
Arrow Right

Restaurant Coordinator

What you will do as a Restaurant Coordinator
Location
Location
United States , Bethlehem
Salary
Salary:
15.00 - 18.00 USD / Hour
atriaseniorliving.com Logo
Atria Senior Living
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High School Diploma or General Education Degree (GED)
  • Minimum of 2 years’ dining room or restaurant waitstaff experience
  • Working knowledge of kitchen operations and food safety standards
  • Strong organizational and time-management skills
  • Ability to resolve problems of dissatisfied customers and/or employees
  • Previous experience in banquets or special event planning
  • Basic computer skills – Microsoft Word and Excel
Job Responsibility
Job Responsibility
  • Serve all of our customers – residents, guests and family members – with top-level service
  • Assist your manager with scheduling waitstaff and keeping residents satisfied with our dining programs
  • Coach and train the new and current waitstaff, while fostering a strong sense of teamwork
  • Acquaint new residents with their dining questions and work with your manager to plan menus
  • Build positive and strong relationships with employees, co-workers and residents
What we offer
What we offer
  • Competitive Pay
  • Paid Holidays & PTO
  • Comprehensive Health Plans
  • Career Advancement Opportunities
  • Tuition Reimbursement
  • Employee Loyalty Rewards Program
  • Retirement Savings Plan with Employer Match
  • Rewards & Recognition
  • Health & Safety
  • Fulltime
Read More
Arrow Right

Memory Care Caregiver

Location
Location
United States , Levittown
Salary
Salary:
17.00 - 20.40 USD / Hour
atriaseniorliving.com Logo
Atria Senior Living
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • No experience needed. We will provide you with all the training you need!
Job Responsibility
Job Responsibility
  • Enhance the lives of older adults by assisting them with mobility needs and daily living activities, including housekeeping, bathing, grooming, and routine personal tasks
  • Work cooperatively with other talented team members to support and care for residents
  • Receive on-the-job training and have opportunities for career growth and advancement
  • Solve problems and exhibit ethical behavior
What we offer
What we offer
  • Competitive Pay
  • Paid Holidays & PTO
  • Comprehensive Health Plans
  • Career Advancement Opportunities
  • Tuition Reimbursement
  • Employee Loyalty Rewards Program
  • Retirement Savings Plan with Employer Match
  • Rewards & Recognition
  • Health & Safety
  • Parttime
Read More
Arrow Right

Caregiver

Location
Location
United States , Plainview
Salary
Salary:
17.00 - 20.40 USD / Hour
atriaseniorliving.com Logo
Atria Senior Living
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • No experience needed. We will provide you with all the training you need!
Job Responsibility
Job Responsibility
  • Enhance the lives of older adults by assisting them with mobility needs and daily living activities, including housekeeping, bathing, grooming, and routine personal tasks
  • Work cooperatively with other talented team members to support and care for residents
  • Receive on-the-job training and have opportunities for career growth and advancement
  • Solve problems and exhibit ethical behavior
What we offer
What we offer
  • Competitive Pay
  • Paid Holidays & PTO
  • Comprehensive Health Plans
  • Career Advancement Opportunities
  • Tuition Reimbursement
  • Employee Loyalty Rewards Program
  • Retirement Savings Plan with Employer Match
  • Rewards & Recognition
  • Health & Safety
  • Fulltime
Read More
Arrow Right

Housekeeper

In the role of Housekeeper, you will be responsible for maintaining the Communit...
Location
Location
United States , Simi Valley
Salary
Salary:
17.00 - 20.40 USD / Hour
atriaseniorliving.com Logo
Atria Senior Living
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Basic reading, writing, and mathematical skills
  • Communicate effectively in English with employees and residents
  • General knowledge of sanitation and safe food handling practices
  • Ability to retain orders (up to four at one time)
  • Strong organization and time management skills
  • Basic computer skills
Job Responsibility
Job Responsibility
  • Provide general housekeeping for both the Community as a whole and individual resident apartments
  • Provide deep cleaning of apartments as needed
  • Launder all linens by washing and drying in the laundry facilities provided
  • Follow Company sanitation and safety procedures
  • Greet residents and guests at each table with a smile to serve beverages, cereals, and/or soups/salads from the cart
  • Courteously and clearly review the menu of the meal, take resident orders, and serve meals and desserts
  • Return cart to the kitchen and assist with storage of perishable items
  • Bus and clear dishes from tables after meals and reset for the next meal
  • Complete side work – fill sugar holders, replenish creamers, clean coffeemaker, etc
  • May perform other duties as needed and/or assigned
What we offer
What we offer
  • Competitive Pay
  • Paid Holidays & PTO
  • Comprehensive Health Plans
  • Career Advancement Opportunities
  • Tuition Reimbursement
  • Employee Loyalty Rewards Program
  • Retirement Savings Plan with Employer Match
  • Rewards & Recognition
  • Health & Safety
  • Fulltime
Read More
Arrow Right

Receptionist

Location
Location
United States , Sacramento
Salary
Salary:
16.90 - 20.28 USD / Hour
atriaseniorliving.com Logo
Atria Senior Living
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or general education degree (GED)
  • or one to three years related experience and/or training
  • or equivalent combination of education and experience
  • Basic First Aid and CPR certification where required
  • Must successfully complete all Atria specified training programs
  • Able to read and comprehend simple instructions, short correspondence, and memos
  • Able to write simple correspondence
  • Able to effectively present information in one-on-one and small group situations to customers, clients, and other employees of the organization
  • Able to add and subtract two digit numbers and to multiply and divide with 10’s and 100’s
  • Able to apply common sense understanding to carry out detailed but uninvolved written or oral instructions
Job Responsibility
Job Responsibility
  • Answers incoming telephone calls in a cheerful and marketing oriented way, determines purpose of callers, and forwards calls to appropriate personnel or department
  • Retrieves messages from voice mail and forwards to appropriate personnel
  • Takes and delivers messages or transfers calls to voice mail when appropriate personnel are unavailable
  • Answers general questions about the community and provides callers with address, directions, and other information and refers other specific questions to appropriate staff
  • Welcomes on-site visitors, determines nature of business, and announces visitors to appropriate personnel
  • Coordinates guestroom reservations
  • Maintains current list of residents who are hospitalized
  • Updates appointment calendars, including transportation and beauty calendars, as directed
  • Receives, sorts, and routes mail, and maintains and routes publications
  • Distributes written communication, accepts and signs for deliveries (mail, prescriptions, etc.)
What we offer
What we offer
  • Competitive Pay
  • Paid Holidays & PTO
  • Comprehensive Health Plans
  • Career Advancement Opportunities
  • Tuition Reimbursement
  • Employee Loyalty Rewards Program
  • Retirement Savings Plan with Employer Match
  • Rewards & Recognition
  • Health & Safety
  • Parttime
Read More
Arrow Right

Customs Compliance and Transport Specialist

We are looking for a detail-oriented and proactive individual to join our Custom...
Location
Location
Canada , Saint-Jean-sur-Richelieu
Salary
Salary:
Not provided
bourassa.ca Logo
TRANSPORT BOURASSA INC.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Relevant experience in transportation, logistics, or customs
  • Bilingualism (essential – frequent interactions in English)
  • Strong ability to manage priorities and handle multiple tasks simultaneously
  • High level of autonomy, initiative, and resourcefulness
  • Analytical mindset with strong problem-solving skills
  • Assertive personality and ability to work in a structured environment
  • Proficiency in Microsoft Office Suite
Job Responsibility
Job Responsibility
  • Ensure compliance and validation of customs documentation
  • Enter and track shipments in internal systems
  • Coordinate information with brokers and stakeholders
  • Resolve customs clearance issues
  • Prepare and transmit required data to customs systems (ACE, etc.)
  • Contribute to the improvement of internal processes
What we offer
What we offer
  • Group and dental insurance (50% paid by the employer)
  • RRSP with employer contribution
  • 10 sick days + 3 personal days
  • 3 weeks of vacation after 2 years
  • 24/7 access to the gym
  • On-site subsidized daycare
  • Social activities and corporate events
  • Fulltime
Read More
Arrow Right

Transport dispatcher / operations coordinator

Transport Bourassa is a rapidly growing company currently seeking a Transport di...
Location
Location
Canada , Saint-Jean-sur-Richelieu
Salary
Salary:
Not provided
bourassa.ca Logo
TRANSPORT BOURASSA INC.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum of 2 years of experience in a similar role
  • Excellent command of both French and English (communication with Canadian and American clients)
  • Strong organizational skills, autonomy, and attention to detail
  • Ability to work under pressure and manage multiple priorities
  • Thrives in a fast-paced, dynamic environment
  • Excellent customer service skills
  • Strong geographical knowledge
Job Responsibility
Job Responsibility
  • Receive freight transportation requests by phone, email, or other means from clients, the pricing department, and sales representatives
  • Process the information received to find a carrier servicing the required region and ensure the shipment is completed promptly, according to pre-established client agreements and Transport Bourassa rate structures
  • Negotiate rates and trip conditions with partner carriers to obtain the best service at the best price
  • Closely monitor transportation movements handled by carriers
  • Research, identify, and develop business relationships with transportation companies capable of covering regions not served by our fleet to meet all client transportation needs
  • Maintain relationships with subcontractors and uphold rate agreements
  • Track all shipments to ensure all necessary documentation is obtained for billing and proof of delivery
  • Perform any other related duties as assigned
What we offer
What we offer
  • Group and dental insurance, with 50% of premiums paid by the employer
  • Subsidized daycare reserved for employees and located on site
  • Employer contributions to a retirement savings plan (RRSP)
  • 10 paid sick days and 3 paid personal days per year
  • 3 weeks of vacation after 2 years
  • On-site gym accessible 24/7
  • Dynamic social club with activities throughout the year
  • A Christmas party you definitely don’t want to miss!
  • Fulltime
Read More
Arrow Right