CrawlJobs Logo

Head of Inference Kernels

United States, San Jose 200000.00 - 300000.00 USD / Year · Job Posted February 18, 2026
Apply Position
Job Link Share

Job Description

As a core member of the team, you will play a pivotal role in leading a high-performing team to build a suite of optimized kernels and implement highly optimized inference stacks for a variety of state-of-the-art transformer models (e.g., Llama-3, Llama-4, Deepseek-R1, Qwen-3, Stable Diffusion-3 etc.). You will be responsible for managing and scaling a high-performance team to pioneer novel model mapping strategies, while co-designing inference time algorithms (e.g., speculative and parallel decoding, prefill-decode disaggregation etc.).

Job Responsibility

  • Architect Best-in-Class Inference Performance on Sohu: Deliver continuous batching throughput exceeding B200 by ≥10x on priority workloads
  • Develop Best-in-Performance Inference Mega Kernels: Develop complex, fused kernels that increase chip utilization and reduce inference latency, and validate these optimizations through benchmarking and regression-tested in production pipelines
  • Architect Model Mapping Strategies: Develop system level optimizations using a mix of techniques such tensor parallelism and expert parallelism for optimal performance
  • Hardware-Software Co-design of Inference-time Algorithmic Innovation: Develop and deploy production-ready inference-time algorithmic improvements (e.g., speculative decoding, prefill-decode disaggregation, KV cache offloading)
  • Build Scalable Team and Roadmap: Grow and retain a team of high-performing inference optimization engineers
  • Cross-Functional Performance Alignment: Ensure inference stack and performance goals are aligned with the software infrastructure teams, GTM and hardware teams for future generations of our hardware

Requirements

  • Experience in designing and optimizing GPU kernels for deep learning on GPUs using CUDA, and assembly (ASM)
  • Experience with low-level programming to maximize performance for AI operations, leveraging tools like Compute Kernel (CK), CUTLASS, and Triton for multi-GPU and multi-platform performance
  • Deep fluency with transformer inference architecture, optimization levers, and full-stack systems (e.g., vLLM, custom runtimes)
  • History of delivering tangible perf wins on GPU hardware or custom AI accelerators
  • Solid understanding of roofline models of compute throughput, memory bandwidth and interconnect performance
  • Experienced in running large-scale workloads on heterogeneous compute clusters, optimizing for efficiency and scalability of AI workloads
  • Scopes projects crisply, sets aggressive but realistic milestones, and drives technical decision-making across the team
  • Anticipates blockers and shifts resources proactively

Nice to have

  • Experience with implementation of state-of-the-art reasoning and chain-of-thought models at production scale
  • Experience with implementation of newer AI compute operations on hardware (e.g., flash attention, long-context attention variants and alternatives)
  • Analyzed and implemented strategies such as KV-cache offloading for efficient compute resource management
  • Familiarity with linear algebra (e.g. matrix decomposition, alternatives bases for vector spaces, matrix rank and its implications)
  • Managed lean, high-performing engineering teams and drove execution on timelines with high quality outcomes

What we offer

  • Medical, dental, and vision packages with generous premium coverage
  • $500 per month credit for waiving medical benefits
  • Housing subsidy of $2k per month for those living within walking distance of the office
  • Relocation support for those moving to San Jose (Santana Row)
  • Various wellness benefits covering fitness, mental health, and more
  • Daily lunch + dinner in our office
  • significant equity package

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Head of Inference Kernels

8 matching positions

Engineering Manager - Inference

We are looking for an Inference Engineering Manager to lead our AI Inference tea...
Location
Location
United States , San Francisco
Salary
Salary:
300000.00 - 385000.00 USD / Year
perplexity.ai Logo
Perplexity
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of engineering experience with 2+ years in a technical leadership or management role
  • Deep experience with ML systems and inference frameworks (PyTorch, TensorFlow, ONNX, TensorRT, vLLM)
  • Strong understanding of LLM architecture: Multi-Head Attention, Multi/Grouped-Query Attention, and common layers
  • Experience with inference optimizations: batching, quantization, kernel fusion, FlashAttention
  • Familiarity with GPU characteristics, roofline models, and performance analysis
  • Experience deploying reliable, distributed, real-time systems at scale
  • Track record of building and leading high-performing engineering teams
  • Experience with parallelism strategies: tensor parallelism, pipeline parallelism, expert parallelism
  • Strong technical communication and cross-functional collaboration skills
Job Responsibility
Job Responsibility
  • Lead and grow a high-performing team of AI inference engineers
  • Develop APIs for AI inference used by both internal and external customers
  • Architect and scale our inference infrastructure for reliability and efficiency
  • Benchmark and eliminate bottlenecks throughout our inference stack
  • Drive large sparse/MoE model inference at rack scale, including sharding strategies for massive models
  • Push the frontier with building inference systems to support sparse attention, disaggregated pre-fill/decoding serving, etc.
  • Improve the reliability and observability of our systems and lead incident response
  • Own technical decisions around batching, throughput, latency, and GPU utilization
  • Partner with ML research teams on model optimization and deployment
  • Recruit, mentor, and develop engineering talent
What we offer
What we offer
  • Equity
  • Health
  • Dental
  • Vision
  • Retirement
  • Fitness
  • Commuter and dependent care accounts
  • Fulltime
Read More
Arrow Right

Learning & Development Consultant

We are seeking a Learning & Development Consultant to help design, manage, and o...
Location
Location
United States , Fortville
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree preferred with 5-8 years of relevant experience
  • Equivalent combination of education and experience will be considered
  • Background in Learning & Development, training design, enablement, or organizational development
  • Strong judgment and analytical ability based on multiple information sources
  • Experience working with SMEs rather than functioning as the technical expert
  • Ability to take new perspectives on existing learning solutions
  • Advanced proficiency in Microsoft Office (Word, Excel, Outlook)
  • Strong organizational skills with an inquisitive, improvement-oriented mindset
  • Comfort with basic to moderately complex quantitative analysis (percentages, utilization, financial figures)
  • Ability to influence, build consensus, and work within broad guidelines
Job Responsibility
Job Responsibility
  • Design, develop, and manage learning programs across Learning Resources, Content, and Curriculum Development
  • Partner with subject matter experts to translate expertise into effective learning materials and exercises
  • Recommend appropriate learning delivery methods (virtual, blended, self-paced, instructor-led) to maximize knowledge transfer
  • Apply learning industry trends and best practices to continuously improve program quality
  • Collaborate with leadership to establish performance objectives and track progress
  • Build strong working relationships with business partners, managers, consultants, and trainers
  • Lead cross-functional alignment on training priorities and execution
  • Assess third-party training programs and audit trainer delivery effectiveness
  • Monitor course utilization and learning resource adoption to ensure organizational needs are met
  • Use data and judgment to recommend improvements and innovative solutions
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
Read More
Arrow Right

Accounts Receivable Specialist

Robert Half is partnering with a growing construction client in San Diego to ide...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of accounts receivable experience, preferably in construction
  • Experience with progress billing, SOV, and AIA billing strongly preferred
  • Strong attention to detail and ability to manage multiple projects
  • Proficiency in Excel and accounting systems
  • Strong communication skills, particularly for collections and internal coordination
Job Responsibility
Job Responsibility
  • Prepare and issue monthly progress billings based on Schedule of Values (SOV)
  • Generate invoices, including AIA billing, lien releases, and supporting documentation
  • Process change orders and extras to ensure accurate contract value and billing
  • Post and apply customer payments and maintain accurate AR records
  • Handle collections on progress billings and retention balances
  • Verify job setup and ensure alignment between contracts, billing, and accounting records
  • Provide preliminary notices and maintain insurance documentation
  • Support the accounting team with data entry and administrative tasks
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right

Front of House Assistant

We are looking for a Part Time Permanent Front of House Assistant to join the te...
Location
Location
United Kingdom , Wilmslow
Salary
Salary:
10.40 - 12.75 GBP / Hour
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Friendly, with an engaging personality and a genuine love for customer service
  • Strong customer service skills
  • Excellent communication skills
  • Great teamwork
  • Hardworking
  • Shift flexibility including weekends
  • The ability to work well and efficiently under pressure
Job Responsibility
Job Responsibility
  • Set up and maintain the front of house and counters each day to our company standards
  • Presenting menus, serving, and helping customers select food and beverage by offering suggestions
  • Providing Counter Service and preparing beverages
  • Deliver our Chef prepared meals to customers at their tables
  • Preparing restaurant tables with special attention to hygiene and cleanliness
  • Processing payment at electronic till points
  • Performing basic cleaning tasks as needed
  • Closing front of house areas each day to our company standards
  • May be required to assist with back of house duties
What we offer
What we offer
  • Competitive rates of pay
  • Healthy work/life balance with no evening work
  • Access to 100s of retail and lifestyle experiences and discounts
  • Health Cash Plans
  • Generous staff discounts
  • 6 weeks Annual Leave per year
  • Free Parking
  • Parttime
Read More
Arrow Right

Ai And Application Development Engineer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Core Java: Strong understanding of Java (JDK 8+, preferably Java 11/17), including multithreading, collections, garbage collection, and JVM internals
  • Frameworks: Extensive experience with Spring Framework (Spring Boot, Spring MVC, Spring Data JPA, Spring Security)
  • Middleware: Proven experience in designing and developing RESTful APIs and microservices
  • Database Technologies (Must): Relational Databases: Strong proficiency in SQL and experience with Oracle databases, including schema design, query optimization, and stored procedures
  • NoSQL Databases: Experience with MongoDB, including data modeling, querying, and performance tuning
  • CI/CD & DevOps: Hands-on experience with CI/CD tools and practices (e.g., Jenkins, GitLab CI, GitHub Actions, Maven/Gradle, Docker, Kubernetes)
  • Version Control: Proficiency with Git and standard branching strategies (e.g., Gitflow)
  • Testing: Experience with unit testing frameworks (JUnit, Mockito) and integration testing
  • Web Technologies (Beneficial): Familiarity with web services (SOAP/REST), XML, JSON
  • AI Tools & Methodologies (Must): Demonstrable exposure and practical experience with AI development tools such as Devin, GitHub Copilot, Claude, Anti Gravity, and Codex
Job Responsibility
Job Responsibility
  • Lead the design, development, and implementation of complex middleware applications using Java and Spring Boot: Utilize AI-powered code generation tools (e.g., Devin, Copilot, Codex) to accelerate development, automate boilerplate code, suggest optimal implementations, and enforce architectural patterns
  • Architect and optimize database interactions with Oracle, SQL, and MongoDB, ensuring high performance and data integrity: Employ AI to analyze database query performance, suggest advanced indexing strategies, optimize schema designs, and generate efficient SQL/NoSQL queries
  • Drive the adoption and continuous improvement of CI/CD pipelines to facilitate rapid and reliable software delivery: Integrate AI into CI/CD processes for intelligent test case generation, predictive failure analysis, automated code vulnerability scanning, and optimization of pipeline execution times based on historical data
  • Collaborate with cross-functional teams, including product management, QA, and operations, to define requirements, design solutions, and deliver high-quality software: Use AI-powered communication and summarization tools (e.g., Claude) to streamline requirement gathering, document analysis, and stakeholder communication
  • Mentor and provide technical guidance to junior and mid-level software engineers, fostering a culture of technical excellence and continuous learning: Leverage AI platforms for personalized learning paths, automated code feedback, and explanations of complex technical concepts
  • Actively research and experiment with AI technologies to identify opportunities for enhancing developer productivity, automating tasks, and improving software quality: Continuously explore emerging AI tools and techniques (such as Anti Gravity for complex problem-solving) and assess their applicability to our development ecosystem
  • Participate in code reviews, ensuring adherence to coding standards, best practices, and architectural guidelines: Utilize AI-powered code analysis tools to pre-scan code for potential bugs, security vulnerabilities, performance bottlenecks, and style deviations, allowing human reviewers to focus on higher-level logic and design
  • Troubleshoot and resolve complex technical issues, ensuring the stability and performance of production systems: Implement AI-driven anomaly detection in monitoring systems, leverage AI for rapid log analysis and root cause identification, and automate incident response workflows
  • Contribute to the strategic planning and technical roadmap for our middleware platforms: Employ AI to analyze industry trends, forecast technology evolution, assess the impact of new features, and prioritize roadmap initiatives based on data-driven insights
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, and model development: Utilize AI for data synthesis, predictive modeling for estimations, identification of potential IT risks, and accelerated model prototyping and validation
  • Fulltime
Read More
Arrow Right

Accounts Receivable Clerk

Our company is looking for a skilled and detail-oriented Accounts Receivable Cle...
Location
Location
United States , Saint Louis
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or equivalent required
  • relevant coursework or associate's degree in accounting preferred
  • Experience with accounting software and strong Microsoft Excel skills
  • Exceptional attention to detail and organizational abilities
  • Strong written and verbal communication skills
  • Ability to work both independently and as part of a team
Job Responsibility
Job Responsibility
  • Generate and send customer invoices
  • Process incoming payments and apply to accounts
  • Monitor accounts for overdue balances and follow up on collections
  • Reconcile accounts and resolve billing discrepancies
  • Maintain accurate records of accounts receivable transactions
  • Work closely with internal teams to ensure accurate and timely billing
  • Provide excellent customer service to clients regarding account inquiries
What we offer
What we offer
  • Medical insurance
  • Vision insurance
  • Dental insurance
  • Life insurance
  • Disability insurance
  • 401(k) plan
  • Free online training
Read More
Arrow Right

Registered Manager/Clinical Nurse Lead

We are seeking an experienced Registered Manager / Clinical Nurse Lead to provid...
Location
Location
United Kingdom , Altrincham
Salary
Salary:
60000.00 GBP / Year
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Registered Nurse with an active, clean NMC PIN
  • Willing and able to act as CQC Registered Manager
  • Proven experience in complex care and community-based services
  • Minimum 6 months’ experience in a community nursing setting
  • Experience delivering face-to-face classroom training and completing competency sign-offs
  • Experience with clinical risk assessments and care planning
  • Sound working knowledge of CQC regulations and inspection frameworks
  • Strong experience in safe medicines management
Job Responsibility
Job Responsibility
  • Collaborate with the Head of Complex Care to mobilise new and existing care packages safely and efficiently
  • Provide clinical leadership across services, ensuring delivery of best practice and high-quality care
  • Lead clinical risk assessments, care planning, and reviews to ensure safe, person-centred, outcome-focused care
  • Offer expert clinical guidance and decision-making support to staff and senior colleagues
  • Act as the CQC Registered Manager, ensuring full regulatory compliance and maintaining a minimum “Good” rating
  • Maintain effective governance systems and drive continuous improvement in clinical quality and compliance
  • Oversee safe and effective medicines management in line with legislation and best practice
  • Lead audits, inspections, and implementation of action plans relating to clinical standards
  • Deliver and support clinical training, competency assessments, and staff development through coaching and supervision
  • Provide clinical oversight across complex care needs including airway management (tracheostomy/ventilation), spinal injuries, bowel and diabetes management, clinical observations, MND, and epilepsy care
What we offer
What we offer
  • Salary £60,000
  • 25 days annual leave plus bank holidays
  • Employee Assistance Programme offering counselling and other support services
  • Employee Recognition Awards
  • Company pension scheme
  • Company events
  • Fulltime
Read More
Arrow Right

Banamex Apps Support C11

Location
Location
Mexico , Álvaro Obregón, Ciudad De Mexico
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2-5 years of experience in a technology planning role
  • Bachelor’s/University degree or equivalent experience
  • Conocimientos acerca de la plataforma Unisys
  • ODT
  • OPCON
  • SightLine
  • Conocimientos Procesos Online (System COMS)
  • Conocimientos del entorno Banamex
  • Liberación de cambios a Producción
  • Ambientes de Pruebas
Job Responsibility
Job Responsibility
  • Tracks business unit performance against the IT business plan and service level agreements
  • identifies gaps
  • Contributes to and / or creates internal communications
  • Works on mergers & acquisitions activities. Creates and prepares reports, metrics and presentations
  • Utilizes good understanding of procedures and concepts in own area and basic knowledge of other technical areas
  • Applies good understanding of how the team and area integrate with others in accomplishing objectives
  • Provides informal guidance to new or lower level employees
  • Makes evaluative judgments based on analysis of factual information
  • resolves problems by identifying and selecting solutions through the application of acquired technical experience and guided by precedents
  • Directly impacts the business by ensuring the quality of work provided by self and others
  • Fulltime
Read More
Arrow Right