CrawlJobs Logo

Head of Inference Kernels

etched.com Logo

Etched

Location Icon

Location:
United States , San Jose

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

200000.00 - 300000.00 USD / Year

Job Description:

As a core member of the team, you will play a pivotal role in leading a high-performing team to build a suite of optimized kernels and implement highly optimized inference stacks for a variety of state-of-the-art transformer models (e.g., Llama-3, Llama-4, Deepseek-R1, Qwen-3, Stable Diffusion-3 etc.). You will be responsible for managing and scaling a high-performance team to pioneer novel model mapping strategies, while co-designing inference time algorithms (e.g., speculative and parallel decoding, prefill-decode disaggregation etc.).

Job Responsibility:

  • Architect Best-in-Class Inference Performance on Sohu: Deliver continuous batching throughput exceeding B200 by ≥10x on priority workloads
  • Develop Best-in-Performance Inference Mega Kernels: Develop complex, fused kernels that increase chip utilization and reduce inference latency, and validate these optimizations through benchmarking and regression-tested in production pipelines
  • Architect Model Mapping Strategies: Develop system level optimizations using a mix of techniques such tensor parallelism and expert parallelism for optimal performance
  • Hardware-Software Co-design of Inference-time Algorithmic Innovation: Develop and deploy production-ready inference-time algorithmic improvements (e.g., speculative decoding, prefill-decode disaggregation, KV cache offloading)
  • Build Scalable Team and Roadmap: Grow and retain a team of high-performing inference optimization engineers
  • Cross-Functional Performance Alignment: Ensure inference stack and performance goals are aligned with the software infrastructure teams, GTM and hardware teams for future generations of our hardware

Requirements:

  • Experience in designing and optimizing GPU kernels for deep learning on GPUs using CUDA, and assembly (ASM)
  • Experience with low-level programming to maximize performance for AI operations, leveraging tools like Compute Kernel (CK), CUTLASS, and Triton for multi-GPU and multi-platform performance
  • Deep fluency with transformer inference architecture, optimization levers, and full-stack systems (e.g., vLLM, custom runtimes)
  • History of delivering tangible perf wins on GPU hardware or custom AI accelerators
  • Solid understanding of roofline models of compute throughput, memory bandwidth and interconnect performance
  • Experienced in running large-scale workloads on heterogeneous compute clusters, optimizing for efficiency and scalability of AI workloads
  • Scopes projects crisply, sets aggressive but realistic milestones, and drives technical decision-making across the team
  • Anticipates blockers and shifts resources proactively

Nice to have:

  • Experience with implementation of state-of-the-art reasoning and chain-of-thought models at production scale
  • Experience with implementation of newer AI compute operations on hardware (e.g., flash attention, long-context attention variants and alternatives)
  • Analyzed and implemented strategies such as KV-cache offloading for efficient compute resource management
  • Familiarity with linear algebra (e.g. matrix decomposition, alternatives bases for vector spaces, matrix rank and its implications)
  • Managed lean, high-performing engineering teams and drove execution on timelines with high quality outcomes
What we offer:
  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • significant equity package

Additional Information:

Job Posted:
February 18, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:
PREMIUM
More languages and countries
+ Unlock 31694 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Head of Inference Kernels

Engineering Manager - Inference

We are looking for an Inference Engineering Manager to lead our AI Inference tea...
Location
Location
United States , San Francisco
Salary
Salary:
300000.00 - 385000.00 USD / Year
perplexity.ai Logo
Perplexity
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Job Responsibility
Job Responsibility
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
What we offer
What we offer
  • Equity
  • Health
  • Dental
  • Vision
  • Retirement
  • Fitness
  • Commuter and dependent care accounts
  • Fulltime
Read More
Arrow Right
New

Assistant Controller

We are looking for an Assistant Controller to help lead accounting operations fo...
Location
Location
United States of America , Las Vegas
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Accounting or Finance & CPA
  • 3+ years of accounting experience
  • Public accounting experience – BIG-4 strongly preferred
  • Good knowledge and understanding of GAAP, internal controls/ SOX and financial reporting
  • Good proficiency with MS Excel
  • Good analytical and problem solving skills
  • Good communication & presentation skills
Job Responsibility
Job Responsibility
  • Oversee daily accounting activities across payables, payroll, receivables, job cost accounting, and general ledger functions to promote accuracy, timeliness, and compliance with established standards
  • Partner with the Controller to build and manage annual budgets covering departmental operations, housing activity, capital assets, land development, and cash flow planning
  • Prepare and analyze financial projections, including revenue and profitability outlooks, while evaluating budget-to-actual results to identify trends and variances
  • Support the completion of monthly, quarterly, and annual financial reporting deliverables for both internal stakeholders and external parties
  • Coordinate the preparation and review of tax filings related to personal property, sales and use, and real estate obligations
  • Help develop, implement, and maintain accounting procedures and internal controls that align with corporate requirements and sound financial practices
  • Serve as a key contact for internal and external audits by providing documentation, answering inquiries, and assisting with follow-up actions tied to audit outcomes
  • Review and authorize invoices and other expenditures within approved company policies and delegated authority limits
  • Work closely with department leaders to ensure consistent cost coding and proper application of general ledger accounting across projects and operations
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right
New

Apprentice Skills Coach in Early Years

We’re looking for a permanent Apprentice Skills Coach (Assessor) in Early Years,...
Location
Location
United Kingdom , Ashton-Under-Lyne
Salary
Salary:
Not provided
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Substantial relevant occupational experience
  • Experience of dealing with the training needs of learners through a range of methodologies
  • Literacy and Numeracy to at least Level 2
  • Assessor qualifications (A1, TAQA, D32)
  • NVQ Level 3 in Plumbing
Job Responsibility
Job Responsibility
  • Deliver training, assessing and quality assuring work-based learning programmes, in line with the college's Apprenticeship Strategy
  • Working towards the growth of high quality, employer focused provision that maximises the opportunities for learners to achieve and progress into sustained employment
What we offer
What we offer
  • Greater Manchester Pension scheme with generous employer contribution
  • Costa Coffee on site
  • Free on site Gym at Beaufort Road
  • Onsite bakery products at Tameside One
  • The Restaurant at Tameside One
  • Retail discounts
  • Cycle to work scheme
  • Great holiday allowance of 33 days (excluding bank holidays or efficiency days at Christmas)
  • Cash health care plan
  • Savings club via the Manchester Credit Union
  • Parttime
Read More
Arrow Right
New

Bar Manager

Archive & Myth is a premium cocktail bar within The Hippodrome Casino, offering ...
Location
Location
United Kingdom , London
Salary
Salary:
42000.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience as a Bar Manager in a premium venue
  • Strong leadership and team management skills
  • Deep knowledge of cocktails, spirits, and bar operations
  • Excellent communication and organisational abilities
  • A passion for hospitality and guest service
  • Flexibility to work late nights and weekends
Job Responsibility
Job Responsibility
  • Lead and manage the team, ensuring high standards of service and professionalism
  • Oversee stock control, ordering, and supplier relationships
  • Maintain bar presentation, cleanliness, and compliance with health & safety regulations
  • Train and mentor staff in bar tending, service standards, and product knowledge
  • Monitor performance, manage rotas, and drive team engagement
  • Deliver exceptional guest experiences and resolve any service issues promptly
What we offer
What we offer
  • Inclusion in the hospitality tronc
  • 50% discount in the Hippodrome Casino's restaurants & bars
  • 30% off Magic Mike Live tickets for friends and family
  • Enhanced maternity & paternity pay
  • Mental health and wellbeing support through our Employee Assistance Programme
  • Access to discounts for many retailers, leisure, food and entertainment establishments
  • Season ticket loans
  • Salary exchange Cycle to Work & Tusker electric vehicle schemes
  • Excellent foreign exchange rates
  • Life assurance
  • Fulltime
Read More
Arrow Right
New

Email Marketing & Onsite Merchandising Manager

The Email Marketing & Onsite Merchandising Manager will lead the planning, execu...
Location
Location
United States , Seattle
Salary
Salary:
35.00 - 43.00 USD / Hour
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3–5 years of professional experience in email marketing, lifecycle marketing, digital marketing, or onsite merchandising
  • Experience managing customer-facing marketing campaigns in a fast-paced environment
  • Strong understanding of email marketing best practices, campaign optimization, segmentation, and performance analytics
  • Experience leveraging data and metrics to drive marketing decisions and improve campaign effectiveness
  • Proficiency with Excel and/or Tableau including data analysis, pivot tables, charts, and reporting
  • Familiarity with GenAI tools and applications for marketing content, campaign support, and copy development
  • Strong project management, organizational, and prioritization skills with the ability to manage multiple initiatives simultaneously
  • Excellent communication and cross-functional collaboration skills
  • Bachelor’s degree required
Job Responsibility
Job Responsibility
  • Own end-to-end execution of email marketing campaigns, including audience targeting, segmentation, creative coordination, deployment, reporting, and optimization
  • Plan and execute lifecycle marketing and onsite merchandising initiatives that drive customer engagement, registrations, and conversions
  • Analyze campaign performance metrics including open rates, click-through rates, conversions, traffic trends, and customer engagement insights to identify optimization opportunities
  • Develop reporting dashboards and provide weekly, monthly, and quarterly updates on campaign performance, project status, and marketing KPIs
  • Partner closely with creative, marketing operations, analytics, and cross-functional business teams to develop campaign strategies and technical requirements
  • Coordinate multiple concurrent marketing initiatives while proactively identifying and resolving project blockers
  • Utilize customer data and marketing insights to support data-driven decision-making and continuous campaign improvement
  • Contribute to testing strategies and optimization efforts to improve overall customer experience and campaign effectiveness
  • Support onsite merchandising and digital marketing efforts aligned with broader customer acquisition and retention initiatives
  • Communicate campaign results, recommendations, and performance insights clearly to stakeholders and leadership teams
What we offer
What we offer
  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right
New

Residential Care Lead

We’re looking for a dedicated Residential Care Lead who will ensure that our res...
Location
Location
United Kingdom , Wallingford
Salary
Salary:
18.50 GBP / Hour
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
May 28, 2026
Flip Icon
Requirements
Requirements
  • Level 4 Diploma in Lead Practitioner, or in social care
  • Strong communication, leadership and IT skills
  • Awareness and knowledge of the requirements of the care workers code of conduct
  • Applicants must already have the right to work in the UK
Job Responsibility
Job Responsibility
  • Lead and promote a high-quality, compassionate service, ensuring compliance with CQC, MCA, and safeguarding requirements
  • Work closely with the home leadership team to maintain a safe, happy, and well-run environment for residents
  • Oversee admissions, care planning, reviews, and medication management to agreed standards
  • Support residents’ personal care, wellbeing, independence, and engagement, liaising with external professionals as needed
  • Manage, support, and develop colleagues in your lead area, including rotas and safe staffing within budget
  • Complete audits, investigations, and administrative tasks in line with organisational procedures
What we offer
What we offer
  • Enhanced pay for family leave
  • Overtime and bank holiday enhancements to pay
  • Group pension plan with option to join the more generous scheme with a 9% employer contribution on a salary sacrifice basis
  • Occupational sick pay after probation
  • Wellbeing support through various initiatives including an employee assistance programme with free counselling and Mental Health First Aiders
  • Opportunity to earn rewards via Care Friends which you can cash in when you choose
  • Free Blue Light card – provides members with thousands of amazing retail discounts
  • Long service awards
  • Apprenticeships, continuous learning and development, professional development and study support opportunities
  • Subsidised meals while on duty
  • Fulltime
!
Read More
Arrow Right
New

Service Advisor

Here at Vertu Renault Leicester, we have a fantastic opportunity for a Service A...
Location
Location
United Kingdom , Leicester
Salary
Salary:
32000.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • previous customer facing service experience
  • customer focus
  • relationship maintenance
  • communication skills
  • ability to work in a fast-paced environment whilst meeting targets and deadlines
Job Responsibility
Job Responsibility
  • setting customer expectations and keeping in touch with them throughout the service or repair of their vehicle
  • up selling additional products
  • follow-up with the customer to ensure they are happy with the work carried out
What we offer
What we offer
  • 25 days holiday rising with length of service
  • bank holidays
  • access to online rewards platform giving cash back and discounts for multiple retailers
  • preferential service rates
  • colleague purchase scheme
  • share incentive scheme
  • pension
  • enhanced maternity & paternity
  • on-going training opportunities
  • career progression
  • Fulltime
Read More
Arrow Right
New

Bakery Team Leader

Outstanding to Work For as accredited by Best Companies. Here at the Cornish Bak...
Location
Location
United Kingdom , Portsmouth
Salary
Salary:
13.50 GBP / Hour
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Innovative
  • Passion for inspiring and nourishing a team
  • Ability to share passion with colleagues and customers
  • Eligible to live and work in the UK
Job Responsibility
Job Responsibility
  • Take the lead on shifts to support your Team in delivering outstanding service
  • Train and coach our team to Brand Standards
  • Make the World’s Best Coffee
  • Prepare Award Winning Food
  • Drive high standards at all times
  • Assist the Management Team with operating the bakery
  • Make the most of colleagues strengths and ensure that they are in the right place to maximise sales
What we offer
What we offer
  • 50% discount on food and drink
  • Pasty Perks – our bespoke reward system that offers discounts and savings on your favourite brands
  • Financial well-being benefit through Stream –save directly from wages into a high interest savings account or access wages before pay-day
  • Employer funded health and wellbeing services, with access to a 24/7 GP line, Employee Assistance Program and discounted gym memberships
  • Paid birthday day off
  • Member of the shareholder fund (service dependent)
  • Team social events with in-bakery incentives
  • Up to 28 days of holiday (including bank holidays), pro rata if necessary
Read More
Arrow Right