Software Engineer: ML Infra Job at Generalist AI (San Mateo)

ML Infra Engineer (Data Systems)

As an ML Infra Engineer (Data Systems), you’ll build and operate the data infras...

Location

United States , San Francisco

Salary:

Not provided

Physical Intelligence

Expiration Date

Until further notice

Requirements

Strong software engineering fundamentals
Experience building distributed systems or large-scale data pipelines
Comfort reasoning about performance, memory, I/O, and storage efficiency
Familiarity with batch and/or streaming processing systems
Experience with object storage systems and data format tradeoffs
Ownership mindset: design, build, operate, and iterate on systems end-to-end
Enjoy working closely with researchers and unblocking fast-moving projects

Job Responsibility

Data Ingestion & Processing: Design and build high-throughput pipelines that validate, transform, and featurize raw multimodal data
Batch & Streaming Systems: Operate large-scale batch and streaming workflows over massive datasets
Storage Systems: Design object storage layouts, metadata systems, and efficient access patterns
choose file formats with performance and scalability in mind
Data Lifecycle Management: Build systems for backfills, dataset rebuilds, garbage collection, and large-scale transformations
Training-Time Performance: Optimize dataloaders, sharding, prefetching, caching, and throughput to reduce time from data arrival → model training
Metadata & Indexing: Build scalable metadata stores for datasets, annotations, and training artifacts
Data Movement: Move hundreds of terabytes to petabytes efficiently across clusters and environments
Operational Correctness: Implement observability, validation, and guardrails to prevent silent data regressions
Cross-Functional Collaboration: Work closely with cross-functional teams of researchers, engineers and roboticists to translate evolving data needs into robust systems

Fulltime

ML Infra Engineer

In this role you will help scale and optimize our training systems and core mode...

Location

United States , San Francisco

Salary:

Not provided

Physical Intelligence

Expiration Date

Until further notice

Requirements

Strong software engineering fundamentals and experience building ML training infrastructure or internal platforms
Hands-on large-scale training experience in JAX (preferred), PyTorch
Familiarity with distributed training, multi-host setups, data loaders, and evaluation pipelines
Experience managing training workloads on cloud platforms (e.g., SLURM, Kubernetes, GCP TPU/GKE, AWS)
Ability to debug and optimize performance bottlenecks across the training stack
Strong cross-functional communication and ownership mindset

Job Responsibility

Own training/inference infrastructure: Design, implement, and maintain systems for large-scale model training, including scheduling, job management, checkpointing, and metrics/logging
Scale distributed training: Work with researchers to scale JAX-based training across TPU and GPU clusters with minimal friction
Optimize performance: Profile and improve memory usage, device utilization, throughput, and distributed synchronization
Enable rapid iteration: Build abstractions for launching, monitoring, debugging, and reproducing experiments
Manage compute resources: Ensure efficient allocation and utilization of cloud-based GPU/TPU compute while controlling cost
Partner with researchers: Translate research needs into infra capabilities and guide best practices for training at scale
Contribute to core training code: Evolve JAX model and training code to support new architectures, modalities, and evaluation metrics

Fulltime

Software Engineer, Infra PyTorch (PhD)

This role is about developing the core PyTorch 2.0 technologies, innovating and ...

Location

United States , Menlo Park

Salary:

181000.00 USD / Year ▼

Sr Software Engineer - Matching ML Platform

Uber is looking for a Software Engineer to join our Matching ML Platform team. T...

Location

United States , Seattle, Washington; San Francisco, California

Salary:

202000.00 - 224000.00 USD / Year

Uber

Expiration Date

Until further notice

Requirements

5+ years experience working on the full software life cycle including gathering requirements, project planning, solution design, coding/implementation, testing, rollout/deployment and best practices as an individual contributor
Experience with ML in production systems
Experience coding using general purpose programming language (eg. C/C++, Java, Python, Go, C#)
Fast and passionate learner
Strong collaboration, documentation and communication skills

Job Responsibility

Build and scale a low-latency platform powering millions of real-time match decisions per second
Identify opportunities to improve various ML system's performance and health
Design modular systems that accelerate product innovation without rework
Optimize for fairness, efficiency, and marketplace health at global scale
Collaborate across product, infra, and ML teams to deliver business-critical impact

What we offer

Bonus program
Equity award & other types of compensation
401(k) plan
Various benefits

Fulltime

Staff Software Engineer - AI/ML Infra

GEICO AI platform and Infrastructure team is seeking an exceptional Senior ML Pl...

Location

United States , Chevy Chase; New York City; Palo Alto

Salary:

115000.00 USD / Year

Geico

Expiration Date

Until further notice

Requirements

Bachelor’s degree in computer science, Engineering, or related technical field (or equivalent experience)
8+ years of software engineering experience with focus on infrastructure, platform engineering, or MLOps
3+ years of hands-on experience with machine learning infrastructure and deployment at scale
2+ years of experience working with Large Language Models and transformer architectures
Proficient in Python
strong skills in Go, Rust, or Java preferred
Proven experience working with open source LLMs (Llama 2/3, Qwen, Mistral, Gemma, Code Llama, etc.)
Proficient in Kubernetes including custom operators, helm charts, and GPU scheduling
Deep expertise in Azure services (AKS, Azure ML, Container Registry, Storage, Networking)
Experience implementing and operating feature stores (Chronon, Feast, Tecton, Azure ML Feature Store, or custom solutions)

Job Responsibility

Design and implement scalable infrastructure for training, fine-tuning, and serving open source LLMs (Llama, Mistral, Gemma, etc.)
Architect and manage Kubernetes clusters for ML workloads, including GPU scheduling, autoscaling, and resource optimization
Design, implement, and maintain feature stores for ML model training and inference pipelines
Build and optimize LLM inference systems using frameworks like vLLM, TensorRT-LLM, and custom serving solutions
Ensure 99.9%+ uptime for ML platforms through robust monitoring, alerting, and incident response procedures
Design and implement ML platforms using DataRobot, Azure Machine Learning, Azure Kubernetes Service (AKS), and Azure Container Instances
Develop and maintain infrastructure using Terraform, ARM templates, and Azure DevOps
Implement cost-effective solutions for GPU compute, storage, and networking across Azure regions
Ensure ML platforms meet enterprise security standards and regulatory compliance requirements
Evaluate and potentially implement hybrid cloud solutions with AWS/GCP as backup or specialized use cases

What we offer

Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being
Financial benefits including market-competitive compensation
a 401K savings plan vested from day one that offers a 6% match
performance and recognition-based incentives
and tuition assistance
Access to additional benefits like mental healthcare as well as fertility and adoption assistance
Supports flexibility- We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year

Fulltime

Staff Software Engineer - AI/ML Infra

GEICO AI platform and Infrastructure team is seeking an exceptional Senior ML Pl...

Location

United States , Palo Alto

Salary:

90000.00 USD / Year

Geico

Expiration Date

Until further notice

Requirements

Bachelor’s degree in computer science, Engineering, or related technical field (or equivalent experience)
8+ years of software engineering experience with focus on infrastructure, platform engineering, or MLOps
3+ years of hands-on experience with machine learning infrastructure and deployment at scale
2+ years of experience working with Large Language Models and transformer architectures
Proficient in Python
strong skills in Go, Rust, or Java preferred
Proven experience working with open source LLMs (Llama 2/3, Qwen, Mistral, Gemma, Code Llama, etc.)
Proficient in Kubernetes including custom operators, helm charts, and GPU scheduling
Deep expertise in Azure services (AKS, Azure ML, Container Registry, Storage, Networking)
Experience implementing and operating feature stores (Chronon, Feast, Tecton, Azure ML Feature Store, or custom solutions)

Job Responsibility

Design and implement scalable infrastructure for training, fine-tuning, and serving open source LLMs (Llama, Mistral, Gemma, etc.)
Architect and manage Kubernetes clusters for ML workloads, including GPU scheduling, autoscaling, and resource optimization
Design, implement, and maintain feature stores for ML model training and inference pipelines
Build and optimize LLM inference systems using frameworks like vLLM, TensorRT-LLM, and custom serving solutions
Ensure 99.9%+ uptime for ML platforms through robust monitoring, alerting, and incident response procedures
Design and implement ML platforms using DataRobot, Azure Machine Learning, Azure Kubernetes Service (AKS), and Azure Container Instances
Develop and maintain infrastructure using Terraform, ARM templates, and Azure DevOps
Implement cost-effective solutions for GPU compute, storage, and networking across Azure regions
Ensure ML platforms meet enterprise security standards and regulatory compliance requirements
Evaluate and potentially implement hybrid cloud solutions with AWS/GCP as backup or specialized use cases

What we offer

Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being
Financial benefits including market-competitive compensation
a 401K savings plan vested from day one that offers a 6% match
performance and recognition-based incentives
and tuition assistance
Access to additional benefits like mental healthcare as well as fertility and adoption assistance
Supports flexibility- We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year

Fulltime

Senior Software Engineer - Matching ML Platform

Uber is looking for a Software Engineer to join our Matching ML Platform team. T...

Location

United States , Seattle; San Francisco; Sunnyvale

Salary:

202000.00 - 224000.00 USD / Year

Uber

Expiration Date

Until further notice

Requirements

5+ years experience working on the full software life cycle including gathering requirements, project planning, solution design, coding/implementation, testing, rollout/deployment and best practices as an individual contributor
Experience with ML in production systems
Experience coding using general purpose programming language (eg. C/C++, Java, Python, Go, C#)
Fast and passionate learner
Strong collaboration, documentation and communication skills

Job Responsibility

Build and scale a low-latency platform powering millions of real-time match decisions per second
Identify opportunities to improve various ML system's performance and health
Design modular systems that accelerate product innovation without rework
Optimize for fairness, efficiency, and marketplace health at global scale
Collaborate across product, infra, and ML teams to deliver business-critical impact

What we offer

Eligible to participate in Uber's bonus program
May be offered an equity award & other types of comp
All full-time employees are eligible to participate in a 401(k) plan
Eligible for various benefits

Fulltime

Software Engineer, Systems ML - SW/HW Co-design

Meta is seeking an AI Software Engineer to join our Research & Development teams...

Location

United States , Sunnyvale

Salary:

181000.00 USD / Year ▼

Select Country

Software Engineer: ML Infra

Job Description

Job Responsibility

Requirements

What we offer

Looking for more opportunities?