Ai infrastructure engineer, model serving platform Job at Scale (San Francisco)

Senior ML Platform Engineer, AI Platform

We are seeking a skilled and passionate ML Platform Engineer to join our team an...

Location

Singapore , Singapore

Salary:

Not provided

Airwallex

Expiration Date

Until further notice

Requirements

5+ years in backend software development
at least 2+ years focus on AI/ML Platform or MLOps infrastructure
deep expertise in MLOps practices, including automated deployment pipelines, model optimization, and production lifecycle management
proven experience designing and implementing low-latency model serving solutions
proficiency in Python
skill in writing high-quality, maintainable code
experience in design and development of large-scale distributed, high concurrency, low-latency inference, high availability systems
excellent communication and mentoring abilities
a relevant degree in Computer Science, Mathematics or related fields

Job Responsibility

Platform Development: Design, build, and maintain the end-to-end MLOps platform using Kubernetes and Cloud Services
Infrastructure as Code (IaC): Use Terraform or similar tools to manage, provision, and scale all ML-related infrastructure securely and efficiently
Pipeline Automation: Implement and optimize CI/CD/CT (Continuous Integration, Delivery, Training) pipelines to automate model training, testing, packaging, and deployment using tools like Argo and Kubeflow Pipelines
Serving Infrastructure: Build highly available, low-latency, and high-throughput model serving infrastructure
Observability: Implement robust monitoring, alerting, and logging solutions to track infrastructure health, model performance, and data/model drift
Tooling & Support: Evaluate, integrate, and support ML tools such as Feature Stores and distributed model training pipelines
Security & Compliance: Ensure platform security, implement RBAC (Role-Based Access Control), and manage secrets for sensitive data and production environments
Collaboration: Work closely with Data Scientists and ML Engineers to understand their needs and provide technical guidance on best practices for scaling their models

Fulltime

AI Systems Engineer – AI Model (Training & Inference)

The AMD AI Group is looking for a Senior Software Development Engineer to own th...

Location

Canada , Markham

Salary:

106400.00 - 159600.00 CAD / Year

AMD

Expiration Date

Until further notice

Requirements

Industry experience shipping production AI/ML infrastructure, with hands-on work spanning both training and inference.
Bachelor’s or Master’s degree or Ph.D in Computer/Software Engineering, Computer Science, or related technical discipline

Job Responsibility

Enable and optimize large-scale model training (LLMs, VLMs, MoE architectures) on AMD Instinct GPU clusters, ensuring correctness, reproducibility, and competitive throughput.
Build and maintain training infrastructure: job orchestration, distributed checkpointing, data loading pipelines, and storage optimization for multi-thousand GPU clusters on Kubernetes.
Debug and resolve training-specific issues including gradient norm explosions, non-deterministic behavior across GPU generations, and compute-communication overlap in distributed training (FSDP, DeepSpeed, Megatron-LM).
Optimize RCCL collective communication patterns for training workloads, including all-reduce, all-gather, and reduce-scatter across multi-node topologies.
Develop monitoring, alerting, and compliance infrastructure to ensure training cluster health, data security, and SLA adherence at scale.
Design and build end-to-end validation and testing infrastructure using proxy workloads, synthetic benchmarks, and configurable workload generators to systematically validate platform readiness across AMD Instinct GPU generations.
Write and optimize high-performance GPU kernels (GEMM, attention, quantized matmul, GPTQ/AWQ) in HIP, Triton, and MLIR targeting AMD Instinct architectures, with demonstrated ability to outperform open-source baselines.
Drive end-to-end inference enablement on new AMD GPU silicon - be among the first to get frontier models running on each new Instinct generation, creating reproducible guides and reference implementations.
Optimize inference serving frameworks (vLLM, SGLang, TorchServe) for AMD GPUs: batching strategies, KV-cache management, speculative decoding, and continuous batching for production throughput/latency targets.
Develop novel approaches to inference acceleration, including bio-inspired algorithms, SLM-assisted batching, and custom scheduling strategies that exploit AMD hardware characteristics.

Fulltime

Senior Machine Learning Engineer, AI Platform

Location

United States; Canada

Salary:

128000.00 - 171000.00 CAD / Year

Mozilla

Expiration Date

Until further notice

Requirements

Bachelor’s degree with 4–6 years of relevant industry experience, or Master’s degree with significant hands-on experience building and operating production ML systems, or work experience equivalent
Strong experience developing in Python for machine learning systems, backend services, or distributed data processing
Proven experience deploying and operating ML workloads in cloud environments, including production-grade infrastructure
Solid understanding of model serving architectures, inference pipelines, and performance tradeoffs (latency, throughput, cost, scaling strategies)
Hands-on experience working with GPU-based workloads and accelerated computing in production settings
Experience designing CI/CD pipelines and development workflows that support reliable ML system deployment
Ability to independently scope and drive technical initiatives while balancing product and operational priorities
Strong problem-solving skills and the ability to debug performance and reliability issues in distributed systems
Clear and effective communication skills, with experience collaborating across engineering, product, and infrastructure teams

Job Responsibility

Design, build, and operate core AI platform components used to train, deploy, and serve machine learning models in production environments
Own model serving and inference workflows end-to-end, driving improvements in reliability, scalability, performance, and operational excellence
Lead efforts to optimize inference systems for throughput, latency, and cost efficiency across CPU and GPU workloads
Design and manage GPU-based inference and training workloads, including performance tuning, capacity planning, and resource utilization optimization
Own and improve critical parts of the model lifecycle, including packaging, versioning, testing strategies, validation, and deployment automation
Implement and evolve observability practices (metrics, logging, tracing, alerting) to improve visibility and operational resilience of ML services and pipelines
Partner closely with product, infrastructure, security, and data teams to design scalable platform capabilities that enable AI-powered features
Contribute to technical design discussions, propose architectural improvements, and mentor junior engineers through code reviews and knowledge sharing
Participate in and help improve operational processes, including incident response, on-call rotations, and post-incident reviews

What we offer

Generous performance-based bonus plans to all eligible employees
Rich medical, dental, and vision coverage
Generous retirement contributions with 100% immediate vesting (regardless of whether you contribute)
Quarterly all-company wellness days where everyone takes a pause together
Country specific holidays plus a day off for your birthday
One-time home office stipend
Annual professional development budget
Quarterly well-being stipend
Considerable paid parental leave
Employee referral bonus program

Fulltime

Senior Machine Learning Engineer, AI Platform

The AI Platform team is responsible for building the foundational infrastructure...

Location

United States; Canada

Salary:

139000.00 - 218000.00 USD / Year

Mozilla

Expiration Date

Until further notice

Requirements

Bachelor’s degree with 4–6 years of relevant industry experience, or Master’s degree with significant hands-on experience building and operating production ML systems, or work experience equivalent
Strong experience developing in Python for machine learning systems, backend services, or distributed data processing
Proven experience deploying and operating ML workloads in cloud environments, including production-grade infrastructure
Solid understanding of model serving architectures, inference pipelines, and performance tradeoffs (latency, throughput, cost, scaling strategies)
Hands-on experience working with GPU-based workloads and accelerated computing in production settings
Experience designing CI/CD pipelines and development workflows that support reliable ML system deployment
Ability to independently scope and drive technical initiatives while balancing product and operational priorities
Strong problem-solving skills and the ability to debug performance and reliability issues in distributed systems
Clear and effective communication skills, with experience collaborating across engineering, product, and infrastructure teams

Job Responsibility

Design, build, and operate core AI platform components used to train, deploy, and serve machine learning models in production environments
Own model serving and inference workflows end-to-end, driving improvements in reliability, scalability, performance, and operational excellence
Lead efforts to optimize inference systems for throughput, latency, and cost efficiency across CPU and GPU workloads
Design and manage GPU-based inference and training workloads, including performance tuning, capacity planning, and resource utilization optimization
Own and improve critical parts of the model lifecycle, including packaging, versioning, testing strategies, validation, and deployment automation
Implement and evolve observability practices (metrics, logging, tracing, alerting) to improve visibility and operational resilience of ML services and pipelines
Partner closely with product, infrastructure, security, and data teams to design scalable platform capabilities that enable AI-powered features
Contribute to technical design discussions, propose architectural improvements, and mentor junior engineers through code reviews and knowledge sharing
Participate in and help improve operational processes, including incident response, on-call rotations, and post-incident reviews

What we offer

Generous performance-based bonus plans
Rich medical, dental, and vision coverage
Generous retirement contributions with 100% immediate vesting
Quarterly all-company wellness days
Country specific holidays plus a day off for your birthday
One-time home office stipend
Annual professional development budget
Quarterly well-being stipend
Considerable paid parental leave
Employee referral bonus program

Fulltime

Vice President - Technology (Data & AI Infrastructure Engineer)

Our client's technology team is responsible for creating and continuously improv...

Location

United States , New York

Salary:

175000.00 - 215000.00 USD / Year

Renner Brown

Expiration Date

Until further notice

Requirements

Bachelor's degree in computer science, Computer Engineering, or related field (Master's degree is a plus)
8+ years in infrastructure engineering, cloud platform engineering, or data engineering
Demonstrated experience building shared platforms or developer services in an enterprise environment
Azure expertise: Azure AI Foundry, Azure Data Factory, Azure Databricks, AKS, Azure API Management, Azure Key Vault, Azure Entra ID
Strong Python skills: backend services, REST APIs (FastAPI or Flask), and automation scripting
PowerShell for infrastructure tasks
Infrastructure-as-Code: Terraform and/or Bicep
container orchestration with Docker and Kubernetes
Experience integrating LLM APIs (Anthropic Claude, Azure OpenAI) in production including token cost management and observability
RAG pipeline experience: vector search (Azure AI Search or pgvector), document processing, and retrieval patterns

Job Responsibility

Design, build, and operate the firm's AI platform, enabling developers to build and deploy Python-based AI applications
Implement and manage Azure AI Foundry environments: model deployments, AI hubs, project workspaces, and access controls
Integrate and operationalize third-party AI APIs (Anthropic Claude API, Azure OpenAI) with secure access patterns, API gateway controls, rate limiting, and cost monitoring
Build internal developer tooling and SDK scaffolding to accelerate AI application development across the firm
Build and maintain data pipelines using Azure Data Factory and Azure Databricks to serve AI application data needs
Implement vector search and document retrieval infrastructure (Azure AI Search) to support RAG-based applications
Manage structured and unstructured data stores including Azure Data Lake, Azure SQL, and Cosmos DB
Provision and maintain secure, scalable infrastructure on Azure (primary) and AWS using Infrastructure-as-Code (Terraform or Bicep)
Build and maintain CI/CD pipelines for AI application deployment via Azure DevOps or GitHub Actions
Manage containerized workloads using Docker and Kubernetes (AKS) for AI application hosting and API services

Fulltime

AI Platform Engineer

You are a platform or DevOps engineer with strong experience running complex sys...

Location

Netherlands , Amsterdam

Salary:

Not provided

Levy Professionals

Expiration Date

Until further notice

Requirements

Strong experience with Kubernetes in production and solid Linux systems knowledge
Hands-on experience with GPU infrastructure (e.g. NVIDIA A100/H100) and LLM/ML model serving
Experience with CI/CD tools (Azure DevOps, GitLab CI, Jenkins) and Python scripting
Familiarity with monitoring tools (Prometheus, Grafana) and infrastructure automation (Terraform, Ansible)
Experience in regulated environments or cost optimisation for high-performance workloads is a plus
We specifically need people who have worked with large language models and GPU-based inference at scale

Job Responsibility

Design, build, and maintain a scalable platform for serving LLM workloads in production
Deploy and manage containerised workloads on Kubernetes, including GPU-based infrastructure
Implement and optimise model serving solutions (e.g. vLLM, Triton, TGI)
Set up monitoring and observability using tools such as Prometheus and Grafana
Build and improve CI/CD pipelines and automate infrastructure using Python and Infrastructure as Code

AI Platform Engineer

You are a platform or DevOps engineer with strong experience running complex sys...

Location

Netherlands , Amsterdam

Salary:

Not provided

Levy Professionals

Expiration Date

Until further notice

Requirements

Strong experience with Kubernetes in production and solid Linux systems knowledge
Hands-on experience with GPU infrastructure (e.g. NVIDIA A100/H100) and LLM/ML model serving
Experience with CI/CD tools (Azure DevOps, GitLab CI, Jenkins) and Python scripting
Familiarity with monitoring tools (Prometheus, Grafana) and infrastructure automation (Terraform, Ansible)
Experience in regulated environments or cost optimisation for high-performance workloads is a plus
We specifically need people who have worked with large language models and GPU-based inference at scale

Job Responsibility

Design, build, and maintain a scalable platform for serving LLM workloads in production
Deploy and manage containerised workloads on Kubernetes, including GPU-based infrastructure
Implement and optimise model serving solutions (e.g. vLLM, Triton, TGI)
Set up monitoring and observability using tools such as Prometheus and Grafana
Build and improve CI/CD pipelines and automate infrastructure using Python and Infrastructure as Code

AI Platform Engineer

You are a platform or DevOps engineer with strong experience running complex sys...

Location

Netherlands , Amsterdam

Salary:

Not provided

Levy Professionals

Expiration Date

Until further notice

Requirements

Strong experience with Kubernetes in production and solid Linux systems knowledge
Hands-on experience with GPU infrastructure (e.g. NVIDIA A100/H100) and LLM/ML model serving
Experience with CI/CD tools (Azure DevOps, GitLab CI, Jenkins) and Python scripting
Familiarity with monitoring tools (Prometheus, Grafana) and infrastructure automation (Terraform, Ansible)
Experience in regulated environments or cost optimisation for high-performance workloads is a plus
We specifically need people who have worked with large language models and GPU-based inference at scale

Job Responsibility

Design, build, and maintain a scalable platform for serving LLM workloads in production
Deploy and manage containerised workloads on Kubernetes, including GPU-based infrastructure
Implement and optimise model serving solutions (e.g. vLLM, Triton, TGI)
Set up monitoring and observability using tools such as Prometheus and Grafana
Build and improve CI/CD pipelines and automate infrastructure using Python and Infrastructure as Code

Select Country

Ai infrastructure engineer, model serving platform

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?