CrawlJobs Logo

Senior LLM Inference Performance Engineer

Finland, Helsinki · Job Posted May 29, 2026
Apply Position
Job Link Share

Job Description

WHAT YOU DO AT AMD CHANGES EVERYTHING At AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you'll discover the real differentiator is our culture. We push the limits of innovation to solve the world's most important challenges—striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond. Together, we advance your career. THE ROLE: The focus of this role is on the performance analysis and optimization of production grade AI services; in particular, in the AMD Inference Microservice (AIM) ecosystem. You will be part of a diverse and ambitious team responsible for ensuring reliable performance of various AI microservices on diverse hardware configurations. You will work with state-of-the-art AI tooling and models on cutting edge AI infrastructure. This role requires both deep understanding of LLMs as well as hands-on knowledge of AI tooling like inference servers.

Job Responsibility

  • LLM and AI Performance: Measure, analyze, and optimize LLM and AI service performance across metrics like latency and throughput for various training and inference use cases
  • Design and implement methodologies for measuring model performance, and automating optimization strategies to identify optimal configurations
  • Stay on top of current advances in AI, models, APIs, and open-source ecosystems, and translate them into scalable solutions
  • LLM and AI Tooling: Design and develop tooling to measure and analyze the performance of AI model deployments and the effect of different configurations and infrastructure, standalone and Kubernetes clusters
  • Develop and maintain tooling for interacting with different ecosystem functions to improve developer and user experience
  • Develop and maintain internal tooling to support LLM and AI performance tuning at scale

Requirements

  • Seasoned in deploying LLMs and other AI model types in production using frameworks like vLLM, SGLang, or similar tooling
  • Deep knowledge about LLM serving and performance metric evaluation
  • Comfortable with Python software development and bash scripting
  • Experience with Docker, Kubernetes and Helm
  • Desire and ability to continuously learn in a fast-changing environment
  • Initiative, pragmatic problem solving, and great collaboration skills
  • Bachelor's or master's degree in computer science, computer engineering, electrical engineering, or an equivalent field

Nice to have

  • Experience with multi-objective hyper-parameter optimization
  • Knowledge of GPU architecture, kernel development, and debugging (C/C++/CUDA)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior LLM Inference Performance Engineer

8 matching positions

Software Engineer II and Senior Software Engineer - Performance

The Artificial Intelligence Performance team at Microsoft develops AI software t...
Location
Location
United States , Mountain View
Salary
Salary:
100600.00 - 199000.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs
  • Measure, benchmark performance on Nvidia/AMD GPUs and first party Microsoft silicon
  • Optimize and monitor performance of LLMs and build SW tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level to improve customer experience and reduce the footprint of the computing fleet
  • Enable fast time to market of LLMs/models and their deployments at scale by building SW tools that afford velocity in porting models on new Nvidia and AMD GPUs
  • Design, implement, and test functions or components for our AI/DNN/LLM frameworks and tools
  • Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems
  • Communicate and collaborate with our partners both internal and external
  • Embody Microsoft's Culture and Values
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Performance

The Artificial Intelligence Performance team at Microsoft develops AI software t...
Location
Location
United States , Mountain View
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs
  • Measure, benchmark performance on Nvidia/AMD GPUs and first party Microsoft silicon
  • Optimize and monitor performance of LLMs and build SW tooling to enable insights into performance opportunities ranging from the model level to the systems and silicon level to improve customer experience and reduce the footprint of the computing fleet
  • Enable fast time to market of LLMs/models and their deployments at scale by building SW tools that afford velocity in porting models on new Nvidia and AMD GPUs
  • Design, implement, and test functions or components for our AI/DNN/LLM frameworks and tools
  • Speeding up/reducing complexity of key components/pipelines to improve performance and/or efficiency of our systems
  • Communicate and collaborate with our partners both internal and external
  • Embody Microsoft's Culture and Values
  • Fulltime
Read More
Arrow Right

Senior Research Engineer, LLM Evaluation and Behavioral Analysis

Together AI is building the fastest, most capable open-source-aligned LLMs and i...
Location
Location
United States , San Francisco
Salary
Salary:
220000.00 - 270000.00 USD / Year
together.ai Logo
Together AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong engineering skills with Python, evaluation tooling, and distributed workflows
  • Experience working with LLMs or transformer-based models, particularly in model evaluation, testing, or red-teaming
  • Ability to reason clearly about qualitative behavior, edge cases, and model failure patterns
  • Experience designing experiments, building datasets, and interpreting noisy behavioral signals
  • Understanding of function calling and structured output formats
  • Familiarity with GPU or distributed compute environments
  • Hands-on experience evaluating function-calling models, agentic systems, or tool-augmented LLM pipelines
  • Experience with multi-turn or multi-step reasoning tasks
  • Familiarity with inference systems, distributed infrastructure, or post-training workflows
  • Passion for discovering subtle behaviors, surprising model gaps, or edge-case failures
Job Responsibility
Job Responsibility
  • Build and iterate on evaluation frameworks that measure model performance across instruction following, function calling, long-context reasoning, multi-turn dialog, safety, and agentic behaviors
  • Develop specialized evaluation suites for: Function calling — argument correctness, schema adherence, tool selection, multi-function planning, and error recovery
  • Agentic workflows — task decomposition, multi-step planning, self-correction, and autonomous tool-use sequences
  • Tool-augmented interactions — search, retrieval, code execution, API-driven actions
  • Create CI/CD automated pipelines for A/B comparisons, regression detection, behavioral drift monitoring, and adversarial probing
  • Design and curate high-quality evaluation datasets, especially nuanced or challenging cases across domains
  • Collaborate with researchers and engineers to diagnose failures, triage regressions, and guide data selection, shaping strategies, objective design, and system improvements
  • Work with engineering teams to build dashboards, reports, and internal tools that help visualize behavior changes across releases
  • Operate in a fast-paced, high-impact environment with deep technical ownership and close partnership with world-class model researchers and infra engineers
What we offer
What we offer
  • competitive compensation
  • startup equity
  • health insurance
  • other benefits
  • Fulltime
Read More
Arrow Right

Senior LLM & Agentic AI Engineer

Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8-11 years of software engineering experience
  • 2+ years of hands-on experience building GenAI or LLM-powered applications
  • Strong experience with GPT, Claude, Gemini, or similar LLMs
  • Strong experience with LangChain, LangGraph, CrewAI, or AI Agent frameworks
  • Strong experience with Retrieval-Augmented Generation (RAG)
  • Strong experience with Vector databases
  • Strong experience with Prompt Engineering
  • Strong programming skills in Java and/or Python
  • Experience building REST APIs and microservices
  • Familiarity with cloud platforms and modern software development practices
Job Responsibility
Job Responsibility
  • Design and deploy enterprise-grade GenAI applications using GPT, Claude, Gemini, or equivalent LLMs
  • Build Agentic AI solutions using frameworks such as LangChain, LangGraph, CrewAI, or similar
  • Develop multi-agent systems to automate customer journeys and business workflows
  • Implement advanced prompt engineering, evaluation frameworks, and guardrails to improve AI response quality
  • Design and optimize Retrieval-Augmented Generation (RAG) architectures
  • Integrate vector databases and enterprise knowledge repositories
  • Improve retrieval accuracy, response relevance, and grounding of AI-generated content
  • Build scalable backend services and APIs using Java and/or Python
  • Develop cloud-native microservices and integrate AI capabilities into customer-facing applications
  • Collaborate with architects, product managers, and business stakeholders to deliver high-impact solutions
  • Fulltime
Read More
Arrow Right

Senior Inference ML Runtime Engineer

The Inference ML Engineering team at Cerebras Systems is dedicated to enabling o...
Location
Location
United States; Canada , Sunnyvale; Toronto
Salary
Salary:
Not provided
cerebras.net Logo
Cerebras Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Mathematics, or a related field
  • 8+ years of experience in large-scale software engineering, with a focus on deep learning or related domains
  • Proficiency in Python for building and maintaining scalable systems
  • Advanced proficiency in C++, with an emphasis on multi-threaded programming, performance optimization, and system-level development
  • Demonstrated experience driving cross-functional projects
  • Experience building and scaling large-scale inference systems for LLMs or multimodal models
  • Familiarity with LLM serving frameworks, such as vLLM, SGLang, and TensorRT-LLM
  • Solid understanding of software architectural patterns for large-scale, high-performance applications
  • Hands-on experience with ML frameworks, such as PyTorch, and a strong understanding of their underlying architectures
  • Strong problem-solving skills, with the ability to balance technical depth with practical implementation constraints
Job Responsibility
Job Responsibility
  • Drive and provide technical guidance to a team of software engineers working on complex machine learning integration projects
  • Design and implement ML features (e.g., structured outputs, biased sampling, predicted outputs) that improve performance of generative AI models at inference time
  • Design and implement high-throughput, low-latency multimodal inference models that support delivery of image, audio, and video inputs and outputs
  • Maintain our scalable serving backend for handling many concurrent requests per minute
  • Scale our inference service by implementing detailed observability throughout the entire stack
  • Analyze and improve latency, throughput, memory usage, and compute efficiency on the service and the implementation of various features
  • Optimize software to accelerate generative LLM inference by achieving high throughput and low latency
  • Stay up-to-date with advancements in machine learning and deep learning, and apply state-of-the-art techniques to enhance our solutions
  • Evaluate trade-offs between different approaches, clearly articulate design choices, and develop detailed proposals for implementing new features
  • Uncover, scope, and prioritize significant areas of technical debt across the software stack to ensure continued high quality of the inference service
What we offer
What we offer
  • Build a breakthrough AI platform beyond the constraints of the GPU
  • Publish and open source their cutting-edge AI research
  • Work on one of the fastest AI supercomputers in the world
  • Enjoy job stability with startup vitality
  • Our simple, non-corporate work culture that respects individual beliefs
Read More
Arrow Right

Senior AI Engineer (Agentic AI / LLM Engineering)

We’re partnering with a rapidly growing, innovation-focused organization that is...
Location
Location
United States
Salary
Salary:
Not provided
zeektek.com Logo
Zeektek
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in software engineering, data engineering, or AI/ML engineering
  • Hands-on experience building applications using LLMs (e.g., GPT, open-source models)
  • Experience with agentic AI frameworks (e.g., LangChain, AutoGen, CrewAI, or similar)
  • Strong programming skills in Python
  • Experience working with large datasets and distributed data platforms
  • Familiarity with Databricks or similar modern data platforms
  • Experience building production-grade AI systems (not just POCs)
  • Strong understanding of prompt engineering and LLM optimization techniques
Job Responsibility
Job Responsibility
  • Design and build agentic AI systems leveraging large language models (LLMs)
  • Develop scalable AI solutions using modern data and AI platforms (Databricks preferred)
  • Translate business problems into production-ready AI workflows and applications
  • Collaborate with product and architecture teams to define AI use cases and technical approaches
  • Implement and optimize: Prompt engineering strategies
  • Token usage and cost efficiency
  • Model performance and response quality
  • Work with large-scale datasets to support training, fine-tuning, and inference workflows
  • Contribute to the development of AI engineering standards, frameworks, and best practices
  • Partner with data science teams to integrate models into production environments
What we offer
What we offer
  • Weekly Direct Deposit
  • 401K Matching
  • Competitive medical, dental and vision insurance
  • Consistent communication throughout your project
  • ZeekTek Referral Program
Read More
Arrow Right

Senior Ai Engineer

Radancy is the global leader in talent acquisition software, helping enterprises...
Location
Location
Austria , Vienna
Salary
Salary:
75000.00 - 80000.00 EUR / Year
radancy.com Logo
Radancy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years as an AI Engineer or equivalent experience
  • Strong background in Software Engineering 5+ years of experience or equivalent
  • Experience with distributed event streaming platforms (Apache Kafka)
  • Experience building backend services in a strongly typed language (Rust, Scala, Go, or similar)
  • Knowledge of AI / ML fundamentals and model optimization techniques
  • Knowledge of NLP, LLMs, RAG, and embedding models
  • Experience with cloud infrastructure (AWS) and CI/CD pipelines for deploying and maintaining backend services
  • Experience working on highly concurrent backend systems with durable state and fault tolerance
  • Knowledge of MLOps, LLM evaluation, and model lifecycle management
  • Understanding of responsible AI, security, and compliance in production systems including AWS guardrails and response style tuning
Job Responsibility
Job Responsibility
  • Design and develop the foundations of our AI Architecture
  • Define data collection and modeling frameworks
  • Implement, maintain and improve classification pipelines and inference for AI Models
  • Improve application performance using quantitative metrics and business KPIs
  • Designing scalable, automated data pipelines using tools like Apache Kafka
  • Explore and implement solutions to classification problems in cooperation with Data Scientists
  • Working collaboratively with other AI Engineering teams in cross product efforts
  • Fulltime
Read More
Arrow Right

Senior Golang Engineer - AI Product & Platforms

We are Citi's Application, Platform and Engineering team, a start-up with the ex...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Golang expertise essential
  • Deep-dive Golang engineering expertise from building high-performance, large-scale production systems
  • Production system builder – proven track record of architecting and building large-scale, high-availability production applications and business-facing platforms from the ground up using Go
  • Advanced Go expertise – deep proficiency in Go's concurrency model (goroutines, channels, select), memory management, profiling, and performance tuning for latency-sensitive systems
  • Microservices and API design – extensive experience designing, building, and maintaining RESTful and gRPC APIs with a focus on clean contracts, versioning, and backward compatibility in high-traffic production systems
  • HashiCorp Vault and secrets management – experience integrating with Vault for dynamic credentials, secrets engines, and enterprise-scale secrets management within Go services
  • Enterprise authentication & authorization – designing and implementing OAuth, JWT, RBAC, and complex identity systems with fine-grained access controls in business-critical applications
  • Cloud-native and Kubernetes expertise – building, deploying, and operating containerized Go applications in Kubernetes, leveraging service meshes, Helm charts, and cloud-native patterns at enterprise scale
  • AI/ML platform engineering – experience building backend infrastructure and APIs that serve AI/ML models, manage inference pipelines, and support LLM-powered applications at scale
  • Observability and reliability engineering – implementing comprehensive logging, metrics, distributed tracing, and alerting to ensure system health and rapid incident resolution
Job Responsibility
Job Responsibility
  • Build AI-powered products from 0-1 – Engineer production-grade, business-facing AI platforms with clean, performant, and maintainable Go code from day one
  • Design and build high-performance backend services – Architect and implement low-latency, high-throughput microservices in Go that operate reliably at planetary scale
  • Design and build developer tools and frameworks – Create reusable libraries, SDKs, and internal tooling in Go that accelerate development across fast-paced engineering teams
  • Tackle complex distributed systems challenges – Design solutions for concurrency, fault tolerance, data consistency, and service orchestration across large-scale distributed environments
  • Champion engineering excellence – Drive best practices in code quality, testing strategies, CI/CD pipelines, and observability to maintain velocity without sacrificing reliability
  • Mentor and elevate the team – Guide other engineers on Go idioms, system design patterns, performance optimization, and building software that scales
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Fulltime
Read More
Arrow Right