CrawlJobs Logo

Engineering Manager, Inference Platform

United States; Canada, Sunnyvale · Job Posted February 17, 2026
Apply Position
Job Link Share

Job Description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. Our novel wafer-scale architecture provides the AI compute power of dozens of GPUs on a single chip, with the programming simplicity of a single device. This approach allows Cerebras to deliver industry-leading training and inference speeds and empowers machine learning users to effortlessly run large-scale ML applications, without the hassle of managing hundreds of GPUs or TPUs. Cerebras' current customers include top model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference. Thanks to the groundbreaking wafer-scale architecture, Cerebras Inference offers the fastest Generative AI inference solution in the world, over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.

Job Responsibility

  • Provide hands-on technical leadership, owning the technical vision and roadmap for the Cerebras Inference Platform, from internal scaling to on-prem customer solutions
  • Lead the end-to-end development of distributed inference systems, including request routing, autoscaling, and resource orchestration on Cerebras' unique hardware
  • Drive a culture of operational excellence, guaranteeing platform reliability (>99.9% uptime), performance, and efficiency
  • Lead, mentor, and grow a high-caliber team of engineers, fostering a culture of technical excellence and rapid execution
  • Productize the platform into an enterprise-ready, on-prem solution, collaborating closely with product, ops, and customer teams to ensure successful deployments

Requirements

  • 6+ years in high-scale software engineering
  • 3+ years leading distributed systems or ML infra teams
  • strong coding and review skills
  • Proven track record scaling LLM inference: optimizing latency (<100ms P99), throughput, batching, memory/IO efficiency and resources utilization
  • Expertise in distributed inference/training for modern LLMs
  • understanding of AI/ML ecosystems, including public clouds (AWS/GCP/Azure)
  • Hands-on with model-serving frameworks (e.g. vLLM, TensorRT-LLM, Triton or similar) and ML stacks (PyTorch, Hugging Face, SageMaker)
  • Deep experience with orchestration (Kubernetes/EKS, Slurm), large clusters, and low-latency networking
  • Strong background in monitoring and reliability engineering (Prometheus/Grafana, incident response, post-mortems)
  • Demonstrated ability to recruit and retain high-performing teams, mentor engineers, and partner cross-functionally to deliver customer-facing products

Nice to have

  • Experience with on-prem/private cloud deployments
  • Background in edge or streaming inference, multi-region systems, or security/privacy in AI
  • Customer-facing experience with enterprise deployments

What we offer

  • Build a breakthrough AI platform beyond the constraints of the GPU
  • Publish and open source their cutting-edge AI research
  • Work on one of the fastest AI supercomputers in the world
  • Enjoy job stability with startup vitality
  • Our simple, non-corporate work culture that respects individual beliefs

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Engineering Manager, Inference Platform

8 matching positions

Principal Software Engineering Manager, Simulation Platform - CoreAI

The AI Frameworks team at Microsoft develops AI software that enables running AI...
Location
Location
United States , Multiple Locations
Salary
Salary:
163000.00 - 296400.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, or related technical discipline
  • 5+ years of experience building/managing team of software engineers
  • 5+ years of experience managing a software project
  • 10+ years of experience in computer architecture and/or embedded systems/software
  • 10+ years of software development experience
  • 10+ years of experience with C++ based object-oriented programming and design
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • 1+ years’ experience with Python
  • A deep technical background and solid foundation in computer architecture, system/hardware simulation, embedded software development and/or firmware
  • Experience designing and/or managing large C++ OOP, scalable, multi-threaded and multi-process software
Job Responsibility
Job Responsibility
  • Managing a team of software engineers
  • Managing development of AI chip simulator, which involves writing requirements, scoping and planning solutions, estimating and assigning work, scheduling and tracking deliverables, integration and releases to partner team, documentation
  • Technical contribution to design, code quality reviews, and capable to step in with hands-on code development when necessary (C++ and Python)
  • Collaborate broadly across multiple disciplines and with various partner teams from hardware designers to AI models developers
  • Fulltime
Read More
Arrow Right

Senior Engineering Manager, ML Platform

We’re looking for hands-on builders–intellectually curious, deeply technical lea...
Location
Location
United States , San Francisco, CA; Los Angeles, CA; New York, NY; Seattle, WA
Salary
Salary:
255000.00 - 345000.00 USD / Year
whatnot.com Logo
Whatnot
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of engineering management experience developing production machine learning systems at consumer-scale loads
  • Bachelor’s degree in Computer Science, Statistics, Applied Mathematics or a related technical field, or equivalent work experience
  • 5+ years of hands-on software engineering experience building and maintaining production systems for consumer-scale loads
  • 1+ years of professional experience developing software in Python
  • Ability to work autonomously and drive initiatives across multiple product areas and communicate findings with leadership and product teams
  • Experience with operational, search, and key-value databases such as PostgreSQL, DynamoDB, Elasticsearch, Redis
  • Experience working with with ML-specific tools and frameworks such as MLFlow, LitServe, TorchServe, Triton
  • Firm grasp of visualization tools for monitoring and logging e.g. DataDog, Grafana
  • Familiarity with cloud computing platforms and managed services such as AWS Sagemaker, Lambda, Kinesis, S3, EC2, EKS/ECS, Apache Kafka, Flink
  • Professionalism around collaborating in a remote working environment and well tested, reproducible work
Job Responsibility
Job Responsibility
  • Own the infrastructure powering AI and ML models across critical business surfaces–supporting growth, recommendations, trust and safety, fraud, seller tooling, and more
  • Guide the prototyping, deployment, and productionization of novel ML architectures that directly shape user experience and marketplace dynamics
  • Help design and scale inference infrastructure capable of serving large models with low latency and high throughput
  • Oversee and evolve real-time feature pipelines that feed both our online and offline stores, ensuring single-second feedback from behavioral signals, high reliability, and model training fidelity
  • Drive feature platform improvements and expand scope to cover non-ML use cases such as fraud rules where point-in-time backtesting is also critical
  • Lead the development of distributed training and inference pipelines leveraging GPUs and both model and data parallelism
  • Optimize system performance by managing resource utilization and developing intelligent feature caching strategies
  • Empower scientists to iterate faster by building abstractions, APIs, and developer tools that simplify the development of near-realtime features and model iteration
  • Roll out ever-better ergonomics around model training and deployment
  • Stretch beyond your comfort zone to take on new technical challenges as we scale AI across Whatnot’s ecosystem
What we offer
What we offer
  • Generous Holiday and Time off Policy
  • Health Insurance options including Medical, Dental, Vision
  • Work From Home Support
  • Home office setup allowance
  • Monthly allowance for cell phone and internet
  • Care benefits
  • Monthly allowance for wellness
  • Annual allowance towards Childcare
  • Lifetime benefit for family planning, such as adoption or fertility expenses
  • Retirement
  • Fulltime
Read More
Arrow Right

Engineering Manager, GenAI Platform

Our generative AI-powered products are bringing joy back to the practice of medi...
Location
Location
United States , San Francisco
Salary
Salary:
220000.00 - 270000.00 USD / Year
abridge.com Logo
Abridge
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years as a software engineer
  • 2+ years managing software engineering teams
  • Experience in building and leading teams in the AI Infrastructure domain - LLM workflows, agentic systems, retrieval and evaluation
  • Comfortable giving constructive feedback on technical designs and code reviews
  • Skilled in building secure, compliant systems in major cloud platforms (GCP preferred, AWS experience welcome)
  • Familiar with containers, databases, and modern frontend frameworks
  • Skilled at hiring and mentorship, with a track record of helping engineers grow their skills and careers
  • Excited about being hands-on in a fast-moving, productive, and supportive environment
  • Has thrived in a fast-growing startup, knows how to operate in that environment
Job Responsibility
Job Responsibility
  • Recruit and mentor engineers with backgrounds in LLM-driven workflows and Agentic systems
  • Provide regular feedback
  • create opportunities for career growth
  • and foster a culture of collaboration and excellence
  • Work closely with ML/AI Research, Inference, Data, and Product teams to plan, execute, and support multiple projects simultaneously
  • responsible for the engineering process in the team and the output of the team
  • Help set business context for the team and apply your knowledge of business and software engineering to guide architectural discussions
  • anticipate and resolve technical challenges
  • and advocate for technology projects
  • Set a high standard for your team including software quality
What we offer
What we offer
  • Generous Time Off: 14 paid holidays, flexible PTO for salaried employees, and accrued time off for hourly employees
  • Comprehensive Health Plans: Medical, Dental, and Vision coverage for all full-time employees and their families
  • Generous HSA Contribution: If you choose a High Deductible Health Plan, Abridge makes monthly contributions to your HSA
  • Paid Parental Leave: Generous paid parental leave for all full-time employees
  • Family Forming Benefits: Resources and financial support to help you build your family
  • 401(k) Matching: Contribution matching to help invest in your future
  • Personal Device Allowance: Tax free funds for personal device usage
  • Pre-tax Benefits: Access to Flexible Spending Accounts (FSA) and Commuter Benefits
  • Lifestyle Wallet: Monthly contributions for fitness, professional development, coworking, and more
  • Mental Health Support: Dedicated access to therapy and coaching to help you reach your goals
  • Fulltime
Read More
Arrow Right

Technical Program Manager, Platform

As a Technical Program Manager for the Platform team, you will partner with engi...
Location
Location
United States , San Francisco, CA; New York, NY
Salary
Salary:
211200.00 - 264000.00 USD / Year
scale.com Logo
Scale
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience as a Technical Program Manager, Product Manager, or Software Engineer, with a proven track record of having built and shipped technical products or platforms from scratch (e.g., internal cloud infrastructure, developer APIs, distributed systems, or ML platforms)
  • Platform Domain Expertise: 3+ years of dedicated experience managing programs focused directly on core engineering infrastructure, cloud-native ecosystems (AWS/GCP), container orchestration (Kubernetes), or distributed systems
  • AI/ML Infrastructure Literacy: Foundational understanding of the infrastructure required for the Generative AI lifecycle, including high-throughput data pipelines, GPU/CPU cluster utilization, or model training/evaluation setups
  • Masterful Communication: Proven track record of presenting to and influencing executive-level stakeholders, with the ability to translate complex technical/architectural challenges into clear business impacts
  • Execution Excellence: Advanced proficiency with iterative development methodologies and modern project management tooling (Linear, Jira, etc.) applied to foundational infrastructure environments.
Job Responsibility
Job Responsibility
  • Lifecycle & Platform Delivery: Lead strategic planning and high-velocity execution for SGP core capabilities (orchestration layers, model serving, APIs). Manage features from technical scoping and architecture design through production launch
  • Cross-Functional GenAI Alignment: Drive execution and manage complex technical dependencies across systems engineering, Core ML, Research, and Product teams to deliver unified SGP capabilities with architectural consistency
  • Technical Translation & Requirements: Translate complex infrastructure metrics (LLM inference optimization, GPU utilization, compute orchestration) into actionable roadmaps. Map demands like multi-tenancy, data privacy, and isolation into platform features
  • Risk & Dependency Mitigation: Proactively identify, track, and mitigate technical risks unique to massive-scale GenAI infrastructure and global SGP deployments, maintaining momentum despite fast-evolving AI frameworks
  • Developer Velocity & Operational Excellence: Establish lightweight agile processes that empower engineers to ship fast without breaking core systems. Define and enforce clear SLOs and performance benchmarks to guarantee production-grade reliability for clients
  • Metrics-Driven Adoption: Track and report on SGP adoption metrics, system reliability, delivery forecasts, and engineering bottlenecks directly to executive leadership to ensure the platform scales responsibly.
What we offer
What we offer
  • comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • commuter stipend
  • Fulltime
Read More
Arrow Right

Principal Product Manager/Architect - Foundry Inference Platform (CoreAI)

We are seeking a Principal Product Manager/Architect to define and guide the tec...
Location
Location
United States , Redmond
Salary
Salary:
163000.00 - 296400.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree AND 10+ years experience in product/service/program management or software development OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check
Job Responsibility
Job Responsibility
  • 1. Product Reliability: Own the product direction for Microsoft Foundry inference, with a primary mandate to make the platform the most reliable enterprise inferencing service available. This includes defining architectural standards for global serving, multi-region resiliency, automated failover, and platform-managed disaster recovery
  • Drive architectural alignment across global routing, capacity pooling, observability, and control plane abstractions to ensure consistent availability, predictable recovery behavior, and simplified customer operations at scale
  • Partner with engineering, infrastructure, and security leaders to ensure reliability targets, SLAs, SLOs and recovery objectives are designed into the platform by default
  • 2. GPU Fleet Efficiency & Capacity: Set the product direction for GPU fleet efficiency and capacity management, guiding platform-level design decisions that maximize utilization, minimize fragmentation, and accelerate timetomonetization of new hardware and models
  • This includes shaping the architecture for global capacity pooling, intelligent scheduling, fungibility across workloads, automated demand forecasting, and softwaredefined allocation
  • The Product Manager/Architect is expected to influence architectural investments across inference utilization, model serving, and hardware/system performance
  • 3. Strategic Customer & Innovation Engagement: Act as a senior technical advisor and architect for Foundry’s most innovative and strategic customers
  • Engage directly with customers on deep technical challenges, including largescale model migrations, reliabilitysensitive production deployments, and advanced serving architectures
  • Support competitive and strategic initiatives by articulating Foundry’s architectural advantages, turning bespoke requests into scalable features
  • 4. Cross-Company Technical Leadership: Serve as a unifying architectural voice across product management, engineering, infrastructure, and partner teams
  • Fulltime
Read More
Arrow Right

Principal Software Engineering Manager - AI Frameworks

As a Principal Software Engineering Manager - AI Frameworks on the team, you wil...
Location
Location
United States , Redmond
Salary
Salary:
139900.00 - 304200.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Master’s Degree in Computer Science or related technical field AND 10+ years of software engineering experience, including 6+ years in engineering management, OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management, or equivalent experience
  • Strong technical foundation in software engineering principles, computer architecture, GPU architecture, and hardware acceleration for neural networks, with the ability to guide teams working in these areas
  • Experience leading teams responsible for end-to-end performance analysis and optimization of LLMs, AI systems, or HPC workloads, including use of GPU profiling and performance analysis tools
  • Demonstrated ability to lead cross-team initiatives, align stakeholders, and translate research or platform capabilities into scalable, production-ready solutions
  • Proven people leadership skills, including hiring, coaching, performance management, and career development, with a track record of building high-performing, inclusive teams
  • Exposure to AI / ML infrastructure, including DNN or LLM training and/or inference systems, and experience with at least one modern deep learning framework (e.g., PyTorch, TensorFlow, ONNX Runtime)
  • Familiarity with GPU software stacks and acceleration technologies such as CUDA, ROCm, Triton, or equivalent, sufficient to guide technical direction and evaluate tradeoffs
Job Responsibility
Job Responsibility
  • Lead and develop a team of engineers working across multiple layers of the AI software stack to enable large-scale training and inference
  • Set technical vision and execution strategy for model performance benchmarking, optimization, and deployment across GPUs and Microsoft hardware
  • Drive performance outcomes by prioritizing and overseeing efforts to benchmark, profile, debug, and optimize training and inference workloads
  • Own performance health by establishing mechanisms to monitor regressions, measure impact, and continuously improve time-to-deploy and hardware efficiency
  • Partner cross-functionally with research, product, infrastructure, and hardware teams to deliver scalable, production-ready AI performance improvements
  • Balance short-term delivery and long-term investments, ensuring the team’s work aligns with organizational goals, platform roadmaps, and Azure capex objectives
  • Build a strong engineering culture through coaching, feedback, hiring, and career development, enabling the team to operate with increasing autonomy and impact
  • Fulltime
Read More
Arrow Right

Principal Software Engineering Manager

Join the Microsoft Location Service Team and Lead the Future of Location-Based E...
Location
Location
Spain , Barcelona
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Demonstrated professional experience in software engineering, including designing, building, and operating large‑scale, complex, and data‑intensive systems
  • Demonstrated experience in an engineering management role, including leading, coaching, and scaling engineering teams responsible for delivering high‑quality, reliable software
  • Solid technical background in distributed systems, data platforms, or machine learning–enabled services, with experience participating in architecture reviews and technical decision‑making
  • Proven ability to lead multi‑team initiatives, operate effectively in ambiguous problem spaces, and influence outcomes across organizational boundaries
  • Demonstrated track record of developing engineers and leaders, and of fostering inclusive, high‑performing engineering cultures focused on quality, ownership, and continuous improvement
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Lead, coach, and grow a team of engineers and scientists, fostering a culture of technical excellence, accountability, and continuous learning
  • Set and communicate a clear technical vision and execution strategy for core location services and inference platforms, aligning team priorities with broader organizational goals
  • Enable teams to adopt AI‑assisted and agentic coding practices to improve engineering velocity, code quality, and long‑term maintainability
  • Drive delivery of large-scale, highly reliable systems, balancing innovation with operational excellence across availability, reliability, performance, and cost
  • Partner with product, applied science, data, and partner teams to translate ambiguous problem spaces into clear, high-impact solutions
  • Lead identification of cross-team dependencies and drive the creation and review of design documents for complex products, services, and platforms
  • Influence org-level architecture, quality standards, and investment decisions across Microsoft’s location ecosystem
  • Serve as a role model for Microsoft values by creating a psychologically safe and inclusive environment where diverse perspectives thrive
  • Fulltime
Read More
Arrow Right

Engineering Manager - Machine Learning Infrastructure

We build simple yet innovative consumer products and developer APIs that shape h...
Location
Location
United States , New York
Salary
Salary:
216000.00 - 367200.00 USD / Year
plaid.com Logo
Plaid
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8–10 years of experience in ML infrastructure, including direct hands-on expertise as an engineer, IC/TL
  • 2+ years of experience managing infrastructure or ML platform engineers
  • Proven experience delivering and operating ML or AI infrastructure at scale
  • Solid technical depth across ML/AI infrastructure domains (e.g., feature stores, pipelines, deployment, inference, observability)
  • Demonstrated ability to drive execution on complex technical projects with cross-team stakeholders
  • Strong communication and stakeholder management skills
Job Responsibility
Job Responsibility
  • Lead and support the ML Infra team, driving project execution and ensuring delivery on key commitments
  • Build and launch Plaid’s next-generation feature store to improve reliability and velocity of model development
  • Define and drive adoption of an ML Ops “golden path” for secure, scalable model training, deployment, and monitoring
  • Ensure operational excellence of ML pipelines, deployment tooling, and inference systems
  • Partner with ML product teams to understand requirements and deliver solutions that accelerate model development and iteration
  • Recruit, mentor, and develop engineers, fostering a collaborative and high-performing team culture
  • Fulltime
Read More
Arrow Right