Senior Backend Engineer, Inference Platform Job at Together AI (San Francisco)

Senior Software Engineer, Backend Platform

At Harvey, we’re transforming how legal and professional services operate — not ...

Location

United States , San Francisco

Salary:

200000.00 - 260000.00 USD / Year

Harvey

Expiration Date

Until further notice

Requirements

5+ years of software engineering experience (post-BS/MS), including building scalable backend systems or internal developer platforms
Proficiency in Python (or similar languages) and deep knowledge of backend development fundamentals (APIs, data stores, concurrency, distributed systems)
Hands-on experience with web frameworks and service architectures (e.g. Flask/FastAPI, Bounded context services, microservices) and an understanding of designing clean, versioned APIs
Familiarity with caching, messaging, and database technologies (Redis, Kafka, SQL/NoSQL databases, Vector databases, etc.) and how to use them effectively for high performance and reliability
A track record of writing high-quality, well-tested code and using tools (unit/integration testing, static typing, CI) to catch issues early and ensure reliability
Strong problem-solving skills and a passion for improving developer experience — you enjoy creating tools or frameworks that make other engineers more productive
Excellent collaboration and communication skills, with the ability to work across teams and incorporate feedback

Job Responsibility

Develop and maintain Harvey’s internal backend frameworks and libraries that provide common capabilities (API routing, service lifecycle management, caching and messaging primitives, error handling interfaces, etc.), so product teams don’t have to reinvent them
Create and improve APIs, service templates, and versioned interfaces that establish consistent patterns for building new services and features
Introduce and champion modern backend architecture patterns like asynchronous I/O (asyncio) and streaming data processing, continually evolving our platform for better performance and scalability
Design Harvey-specific abstractions and domain-specific frameworks—covering cross-cutting concerns (e.g., authorization, streaming) and areas like data governance and event processing—to provide product engineers with these capabilities out of the box
Embed reliability and observability into the platform by building in tracing, metrics, and automated tests (shift-left), ensuring services built on our foundation are robust and easy to monitor
Collaborate with Model Infrastructure team to tackle challenges unique to GenAI-native applications — such as supporting high-throughput model inference, managing streaming and long-running API interactions, and designing abstractions for retrieval, context handling, and prompt lifecycle
Collaborate with the Developer Experience and Infrastructure teams (who own CI/CD pipelines, build tools, and release infrastructure) to integrate our platform components seamlessly into the deployment and monitoring ecosystem
Work closely with product engineering teams to gather feedback, evangelize best practices, and make the “paved road” approach a reality — providing strong defaults and clear documentation so teams can move fast with confidence

What we offer

Offers Equity
Offers Bonus
Comprehensive health, dental and vision coverage
retirement benefits (401k match up to 4%)
flexible PTO

Fulltime

Senior ML Infrastructure Engineer, Inference Platform

About the Team: The ML Inference Platform is part of the AV ML Infrastructure or...

Location

United States , Austin, Texas; Mountain View, California; Sunnyvale, California

Salary:

155420.00 USD / Year

General Motors

Expiration Date

Until further notice

Requirements

5+ years of industry experience, with focus on machine learning systems or high performance backend services
Expertise in either Python, C++ or other relevant coding languages
Expertise in ML inference, model serving frameworks (triton, rayserve, vLLM etc)
Strong communication skills and a proven ability to drive cross-functional initiatives
Ability to thrive in a dynamic, multi-tasking environment with ever-evolving priorities

Job Responsibility

Design and implement core platform backend software components
Collaborate with ML engineers and researchers to understand critical workflows, parse them to platform requirements, and deliver incremental value
Lead technical decision-making on model serving strategies, orchestration, caching, model versioning, and auto-scaling mechanisms for highly optimized use of accelerators
Drive the development of monitoring, observability, and metrics to ensure reliability, performance, and resource optimization of inference services
Proactively research and integrate state-of-the-art model serving frameworks, hardware accelerators, and distributed computing techniques
Lead technical initiatives across GM’s ML ecosystem
Raise the engineering bar through technical leadership, establishing best practices
Contribute to open source projects
represent GM in relevant communities

What we offer

medical
dental
vision
Health Savings Account
Flexible Spending Accounts
retirement savings plan
sickness and accident benefits
life insurance
paid vacation & holidays
tuition assistance programs

Fulltime

Senior ML Platform Engineer, AI Platform

We are seeking a skilled and passionate ML Platform Engineer to join our team an...

Location

Singapore , Singapore

Salary:

Not provided

Airwallex

Expiration Date

Until further notice

Requirements

5+ years in backend software development
at least 2+ years focus on AI/ML Platform or MLOps infrastructure
deep expertise in MLOps practices, including automated deployment pipelines, model optimization, and production lifecycle management
proven experience designing and implementing low-latency model serving solutions
proficiency in Python
skill in writing high-quality, maintainable code
experience in design and development of large-scale distributed, high concurrency, low-latency inference, high availability systems
excellent communication and mentoring abilities
a relevant degree in Computer Science, Mathematics or related fields

Job Responsibility

Platform Development: Design, build, and maintain the end-to-end MLOps platform using Kubernetes and Cloud Services
Infrastructure as Code (IaC): Use Terraform or similar tools to manage, provision, and scale all ML-related infrastructure securely and efficiently
Pipeline Automation: Implement and optimize CI/CD/CT (Continuous Integration, Delivery, Training) pipelines to automate model training, testing, packaging, and deployment using tools like Argo and Kubeflow Pipelines
Serving Infrastructure: Build highly available, low-latency, and high-throughput model serving infrastructure
Observability: Implement robust monitoring, alerting, and logging solutions to track infrastructure health, model performance, and data/model drift
Tooling & Support: Evaluate, integrate, and support ML tools such as Feature Stores and distributed model training pipelines
Security & Compliance: Ensure platform security, implement RBAC (Role-Based Access Control), and manage secrets for sensitive data and production environments
Collaboration: Work closely with Data Scientists and ML Engineers to understand their needs and provide technical guidance on best practices for scaling their models

Fulltime

Senior Backend Engineer

My client is a well funded early stage startup building a high scale social plat...

Location

United States , New York

Salary:

150000.00 - 300000.00 USD / Year

Apollo Solutions

Expiration Date

Until further notice

Requirements

Strong expertise in TypeScript, Node.js, and NestJS, with the ability to design and build production grade backend systems
Proven experience implementing secure authentication systems using OAuth, FIDO2 or WebAuthn, and JWT
Hands on experience working with Prisma ORM, PostgreSQL or Aurora, Redis, and DynamoDB across high-scale environments
Strong understanding of distributed systems, including event driven architecture, pub sub patterns, and message brokers such as Kafka or AWS MSK
Experience building real time systems
Deep familiarity with AWS and cloud infrastructure, including services such as Aurora Global Database, RDS Proxy, ElastiCache or Valkey, EKS, and Terraform
Experience working with observability tooling such as LGTM stack and implementing modern CI/ CD pipelines
Proven track record of building external facing systems where performance, concurrency, and reliability are critical

Job Responsibility

Design, build, and deploy backend services using TypeScript and NestJS with a focus on performance, scalability, and maintainability
Architect and implement high traffic APIs, authentication systems, and distributed services enabling real-time and AI-enhanced product features
Model and optimize data systems across relational databases, caches, and event streaming platforms to support both operational workloads and AI-driven insights
Build resilient, event-driven systems using pub sub patterns and long-lived connections that handle large scale concurrent usage
Work closely with AWS infrastructure to improve deployment pipelines, scalability, and overall system reliability
Support backend systems that enable AI and machine learning workflows, including data pipelines and real-time inference services
Collaborate with a small, high-performing engineering team to make strong technical decisions and execute quickly with high ownership

Fulltime

Senior Software Engineer - Backend

Security represents the most critical priorities for our customers in a world aw...

Location

United States , Redmond

Salary:

119800.00 - 234700.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
OR equivalent experience.
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.

Job Responsibility

Design, build, and own foundational platform services for data access platform, model execution, machine learning inference, and agent orchestration that are reused across the Microsoft Defender ecosystem
Lead the implementation of medium‑to‑large, scoped features and components, making sound architectural decisions that balance scalability, reliability, security, and long‑term maintainability
Collaborate closely with research, security, and product engineering teams to translate research needs into robust, reusable platform abstractions, rather than one‑off solutions.
Drive end‑to‑end ownership of services, including design, development, testing, deployment, monitoring, and live‑site operations.
Identify and address systemic platform gaps by consolidating shared capabilities and reducing fragmented or bespoke infrastructure across teams.
Contribute to platform standards and best practices for API design, service reliability, observability, and operational excellence
Participate actively in design and code reviews, raising the overall engineering quality and bar for platform development
Mentor and support other engineers through technical guidance, knowledge sharing, and example‑setting, without direct people management responsibility.
Proactively identify opportunities to improve performance, cost efficiency, security, and compliance across the fabric.

Fulltime

Senior Software Engineer – AI/ML Platform

As a Senior Software Engineer on Socure’s AI Platform team, you’ll design and bu...

Location

United States

Salary:

180000.00 - 200000.00 USD / Year

Socure

Expiration Date

Until further notice

Requirements

4+ years of experience as a software engineer, with at least 2 years focused on low latency and highly available backend systems
Bachelor’s or Master’s degree in Computer Science, Data Science, AI, Machine Learning, or a related field with a strong academic record
Strong fundamentals in data structures, algorithms, and distributed computing principles
Strong analytical and problem-solving skills, with a passion for AI and machine learning
Strong programming skills in Python
Hands-on experience with model systems including low latency model serving, registry, and pipeline orchestration(preferably SageMaker)
Solid understanding of MLOps best practices, including model versioning, testing, deployment, and reproducibility
Experience building and maintaining CI/CD pipelines for ML workflows
Experience with ML frameworks such as TensorFlow, PyTorch, or Scikit-learn
Experience with database technologies (SQL, NoSQL, or data warehouses like Snowflake or Redshift)

Job Responsibility

Build and maintain scalable systems and infrastructure for deploying and serving ML models
Design low-latency, fault-tolerant model inference systems using Amazon SageMaker
Implement safe deployment strategies like blue/green deployments and rollbacks
Create and manage CI/CD pipelines for ML workflows
Monitor model performance and system health using AWS observability tools (e.g., CloudWatch)
Develop internal tools and APIs to help ML teams deploy and monitor models easily
Collaborate with ML engineers, data scientists, and DevOps to productionize new models
Participate in code reviews, system design, and platform roadmap discussions
Continuously improve deployment reliability, speed, and usability of the ML platform

What we offer

Offers Equity
Offers Bonus

Fulltime

Senior Software Engineer (Typescript / Backend) - AI/ML

We are looking for a Senior Software Engineer to drive the development of AI/ML-...

Location

United States

Salary:

131000.00 - 185000.00 USD / Year

ClickHouse

Expiration Date

Until further notice

Requirements

5+ years of software engineering experience in production environments
Exposure to working directly with AI/ML technologies
Strong frontend skills with TypeScript/JavaScript and React
Backend development experience in TypeScript or Python, with a focus on API design and service architecture
You have a high level of ownership and can drive features from concept to production with minimal supervision
You thrive in collaborative environments and can effectively communicate technical concepts to diverse stakeholders

Job Responsibility

Feature Development: Design and implement AI-powered features across the full stack, from backend inference services to intuitive frontend interfaces within the ClickHouse Cloud platform
API Architecture: Create robust, scalable APIs that connect ClickHouse's database capabilities with modern AI/ML inference systems and external/internal AI services
UI/UX Implementation: Build responsive, intuitive user interfaces that make complex AI functionalities accessible and valuable to users of all technical backgrounds
Ecosystem Integrations: Implement and maintain integrations with the broader AI/ML ecosystem and standards, ensuring that ClickHouse as a technology works seamlessly with popular frameworks and tools
Technical Integration: Integrate models into production systems with proper monitoring, versioning, observability, and evaluation

What we offer

Flexible work environment - ClickHouse is a globally distributed company and remote-friendly. We currently operate in 20 countries
Healthcare - Employer contributions towards your healthcare
Equity in the company - Every new team member who joins our company receives stock options
Time off - Flexible time off in the US, generous entitlement in other countries
A $500 Home office setup if you’re a remote employee
Global Gatherings – We believe in the power of in-person connection and offer opportunities to engage with colleagues at company-wide offsites

Fulltime

Senior Machine Learning Engineer, AI Platform

The AI Platform team is responsible for building the foundational infrastructure...

Location

United States; Canada

Salary:

139000.00 - 218000.00 USD / Year

Mozilla

Expiration Date

Until further notice

Requirements

Bachelor’s degree with 4–6 years of relevant industry experience, or Master’s degree with significant hands-on experience building and operating production ML systems, or work experience equivalent
Strong experience developing in Python for machine learning systems, backend services, or distributed data processing
Proven experience deploying and operating ML workloads in cloud environments, including production-grade infrastructure
Solid understanding of model serving architectures, inference pipelines, and performance tradeoffs (latency, throughput, cost, scaling strategies)
Hands-on experience working with GPU-based workloads and accelerated computing in production settings
Experience designing CI/CD pipelines and development workflows that support reliable ML system deployment
Ability to independently scope and drive technical initiatives while balancing product and operational priorities
Strong problem-solving skills and the ability to debug performance and reliability issues in distributed systems
Clear and effective communication skills, with experience collaborating across engineering, product, and infrastructure teams

Job Responsibility

Design, build, and operate core AI platform components used to train, deploy, and serve machine learning models in production environments
Own model serving and inference workflows end-to-end, driving improvements in reliability, scalability, performance, and operational excellence
Lead efforts to optimize inference systems for throughput, latency, and cost efficiency across CPU and GPU workloads
Design and manage GPU-based inference and training workloads, including performance tuning, capacity planning, and resource utilization optimization
Own and improve critical parts of the model lifecycle, including packaging, versioning, testing strategies, validation, and deployment automation
Implement and evolve observability practices (metrics, logging, tracing, alerting) to improve visibility and operational resilience of ML services and pipelines
Partner closely with product, infrastructure, security, and data teams to design scalable platform capabilities that enable AI-powered features
Contribute to technical design discussions, propose architectural improvements, and mentor junior engineers through code reviews and knowledge sharing
Participate in and help improve operational processes, including incident response, on-call rotations, and post-incident reviews

What we offer

Generous performance-based bonus plans
Rich medical, dental, and vision coverage
Generous retirement contributions with 100% immediate vesting
Quarterly all-company wellness days
Country specific holidays plus a day off for your birthday
One-time home office stipend
Annual professional development budget
Quarterly well-being stipend
Considerable paid parental leave
Employee referral bonus program

Fulltime

Select Country

Senior Backend Engineer, Inference Platform

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?

Senior Backend Engineer, Inference Platform

Senior Software Engineer, Backend Platform

Senior ML Infrastructure Engineer, Inference Platform

Senior ML Platform Engineer, AI Platform

Senior Backend Engineer

Senior Software Engineer - Backend

Senior Software Engineer – AI/ML Platform

Senior Software Engineer (Typescript / Backend) - AI/ML

Senior Machine Learning Engineer, AI Platform

Our AI answers in your language