CrawlJobs Logo

Senior Software Engineer, Go - LLM Team

assemblyai.com Logo

AssemblyAI

Location Icon

Location:
United States

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

180000.00 - 240000.00 USD / Year

Job Description:

We're seeking an exceptional Senior Software Engineer to join our LLM team. This role is focused on building and maintaining our LLM gateway service—a unified API platform that connects customers to multiple LLM providers. You'll work on high-impact projects that directly solve customer problems, improve their AI and agentic workflows, and ensure reliable access to the best models for their use cases.

Job Responsibility:

  • Partner closely with the customer success team to understand customer use cases, challenges, and integration needs
  • Translate customer pain points surfaced by the CS team into technical solutions and product improvements
  • Build features and tooling that directly address customer needs and improve their workflows
  • Provide technical guidance and expertise to the customer success team to help them support customers effectively
  • Own and deliver complete features and integrations within our LLM gateway service
  • Build and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock etc.)
  • Write clean, maintainable, well-tested code following best practices
  • Design and implement scalable, fault-tolerant solutions with appropriate abstractions
  • Proactively identify and address technical debt, reliability issues, and code quality concerns
  • Participate in on-call rotation to ensure service reliability and rapid incident response
  • Conduct thorough code reviews focused on maintainability, testing, reliability, and architectural concerns
  • Ensure proper test coverage across unit, integration, and end-to-end testing levels
  • Improve code maintainability and extensibility through targeted refactoring
  • Contribute to runbooks, incident postmortems, customer-facing documentation, and knowledge sharing
  • Identify bottlenecks and propose solutions to improve team velocity and system reliability
  • Work closely with product, customer success, and engineering teams to deliver solutions
  • Collaborate with external API providers to troubleshoot integration issues and improve reliability
  • Translate ambiguous customer requirements into well-scoped technical work
  • Proactively communicate status, risks, and blockers
  • Mentor junior engineers on coding standards, reliability practices, and customer-centric thinking
  • Participate in architectural discussions and contribute to technical decisions

Requirements:

  • 5+ years of professional software engineering experience
  • Strong proficiency in Go with demonstrated ability to write production-quality code
  • Hands-on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering concepts
  • Experience building or working with API gateways, proxies, and rate limiting systems
  • Deep experience with prompt engineering, chain-of-thought, few-shot learning, and other LLM optimization techniques
  • Experience building applications with llms or ai agents with cascading architectures
  • Solid understanding of software engineering fundamentals: Data structures and algorithms, System design and architectural patterns, Testing strategies (unit, integration, end-to-end), Code review practices and technical collaboration
  • Experience with: RESTful APIs and distributed systems concepts, Building or integrating with third-party APIs, SQL and database design, Version control (Git) and CI/CD pipelines, Containerization (Docker) and cloud infrastructure
  • Strong communication skills and ability to collaborate effectively with cross-functional teams
  • Track record of delivering high-quality, reliable software in a fast-paced, team environment
  • Ability to thrive in a startup environment with changing priorities and rapid iteration

Nice to have:

  • Experience with AWS services (ECS, Lambda, ElastiCache/Redis, RDS, S3, CloudWatch, API Gateway)
  • Experience with monitoring and observability tools (Datadog, CloudWatch)
  • Experience with request routing, load balancing, and traffic management
  • Experience with circuit breakers, retry logic, fallback strategies, and fault tolerance patterns
  • Understanding of API design, versioning, and backward compatibility
  • Experience working closely with customer success or support teams
What we offer:
  • Competitive equity grants
  • 100% employer-paid benefits
  • Flexibility of being fully remote
  • 401k match up to 4% for US-based full time team members

Additional Information:

Job Posted:
February 17, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Senior Software Engineer, Go - LLM Team

Senior Software Engineer, Observability

The Observability team at Airtable ensures that engineers have the tools they ne...
Location
Location
United States , San Francisco; New York; Seattle
Salary
Salary:
196000.00 - 270000.00 USD / Year
airtable.com Logo
Airtable
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of software engineering experience
  • 3+ years focused on observability or infrastructure at scale
  • Demonstrated success implementing and running production-grade logging, metrics, or tracing systems
  • Proficiency in distributed systems concepts, data streaming pipelines, and container orchestration (Kubernetes)
  • Deep hands-on knowledge of tools such as Prometheus, Grafana, Datadog, OpenTelemetry, ELK Stack, Loki, or ClickHouse
  • Comfort with at least one programming language (e.g., Go, Python, Java) to build and maintain observability tooling
  • Experience mentoring engineers and collaborating across multiple teams
  • Strong communication skills
  • Eagerness to own high-impact initiatives
  • Proven ability to balance short-term fixes with long-term strategic vision
Job Responsibility
Job Responsibility
  • Architect and scale core observability systems
  • Lead the design and evolution of logging, metrics, and tracing pipelines
  • Evaluate and integrate new technologies (e.g., OpenTelemetry, ClickHouse, ELK stack)
  • Guide and mentor a growing team of infrastructure engineers
  • Define and uphold coding standards and operational excellence
  • Partner with Deploy Infrastructure, Service Orchestration, and Product teams
  • Align infrastructure decisions with business goals
  • Own end-to-end reliability for observability tools and establish SLAs, SLOs, and error budgets
  • Optimize performance and cost of large-scale data pipelines
  • Shape the observability roadmap
What we offer
What we offer
  • Opportunity to receive benefits
  • Restricted stock units
  • May include incentive compensation
  • Comprehensive benefit offerings
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Build AI Tools

This role sits within the newly formed GenAI Security team, which is responsible...
Location
Location
United Kingdom , Belfast
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Highly motivated self-starter with excellent interpersonal and problem-solving skills
  • Bachelor’s degree or equivalent work experience
  • Good oral and written communication skills
  • Significant relevant industry work experience
  • Experience of the full lifecycle of design, implementation and running of enterprise software solutions involving cross functional team collaboration
  • Expertise in a major programming language such as Python and/or Go, and associated tooling (Git, Maven, IDEs, Jenkins, Bitbucket etc)
  • Expertise in designing and implementing secure APIs and libraries
  • Experience in Generative AI, LLM frameworks, LLM prompt engineering and/or adversarial testing is a bonus
  • Experience with Cyber engineering and Operations, which could include DevSecOps or MLSecOps
  • Experience contributing to the architecture and design (architecture, design patterns, reliability, scaling) of new and current systems
Job Responsibility
Job Responsibility
  • Designing, developing, optimizing, and enhancing a GenAI prompt security platform to protect firm AI/LLM-based applications from adversarial attacks and prompt injections
  • Building and automating a security testing framework to validate protection mechanisms for various LLM use cases
  • Owning solutions that are expected to operate and perform at scale across the organisation
  • Collaboration with multiple stakeholders and partners across Engineering and Operations as well as partner teams within the wider Citi organisation, across different time zones
What we offer
What we offer
  • 27 days annual leave (plus bank holidays)
  • A discretional annual performance related bonus
  • Private Medical Care & Life Insurance
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right

Senior Gen AI Engineer

At Citi Innovation Labs, we’re building a variety of generative AI applications ...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven Track Record: Demonstrated success in designing and deploying high-impact applications
  • Experience of leading development efforts from inception to production
  • Strategic and Analytical Mindset: Ability to approach challenges from a holistic perspective, balancing technical requirements with business goals
  • Architecture and Design Expertise: Extensive experience in system design, architectural principles, and microservices architecture
  • Ability to design fault tolerant, high-availability systems
  • Expertise with LLMs: Hands-on experience with Large Language Models and a strong interest in AI
  • Multi-Language Mastery: Extensive experience in at least two server-side languages (Python, Kotlin, Java, or Go) and the ability to switch languages use the best tool for the job
  • Inquisitive Mindset: Eager to learn, experiment, and stay updated with emerging technologies
  • Cloud Native & Kubernetes: Proven experience in developing and deploying microservice architecture applications using cloud-native tools, such as Kubernetes
  • Education: Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent industry experience
Job Responsibility
Job Responsibility
  • Lead and Design Robust Systems: Take ownership of architecture and design of applications, services and libraries, ensuring scalability, resilience, and performance at an Enterprise level
  • Set Technical Direction: Guide technical standards, best practices, and architectural frameworks to shape the development of generative AI solutions across teams
  • Mentor and Grow: Provide mentorship and coaching to less experienced developers, helping to build a high-performance team
  • Write Excellent Code: We care deeply about the craft of code—whether it’s enhancing user experience, optimizing performance, or enhancing an API design
  • Collaborate Globally: Work closely with global teammates to form a high-performing software development team
  • Drive Innovation: Identify and experiment with emerging technologies, driving the application of AI to solve challenging business problems
  • Engage with Stakeholders: Collaborate with data scientists, product owners, and project stakeholders to ensure effective delivery
  • Fulltime
Read More
Arrow Right

Principal Software Engineer, AI Developer Tools

At Docker, we make app development easier so developers can focus on what matter...
Location
Location
United States , Seattle
Salary
Salary:
232000.00 - 319000.00 USD / Year
docker.com Logo
Docker
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years software engineering experience with 3+ years in Staff or Principal Engineer roles
  • Deep expertise in AI/ML technologies with hands-on production experience building LLM-powered applications, AI agents, or AI-assisted developer tools
  • Strong understanding of LLM APIs (OpenAI, Anthropic, etc.), prompt engineering, agent orchestration frameworks, and practical applications of AI in software development workflows
  • Proven track record of architecting and building highly scalable distributed systems and developer-facing platforms
  • Production experience with modern cloud-native infrastructure including Kubernetes, GitOps deployment patterns, observability systems, and CI/CD pipelines
  • Proficiency in Go (preferred), Rust, Java, or Python with strong software engineering fundamentals
  • Experience designing developer tools, platform engineering systems, or internal tools that enable other teams
  • Exceptional product and platform mindset considering business outcomes, developer experience, and technical trade-offs
  • Strong communication skills with ability to influence technical and non-technical stakeholders across the organization
  • Track record of technical mentorship and elevating engineering teams' capabilities
Job Responsibility
Job Responsibility
  • Define the long-term technical vision and architecture for AI-powered developer tools and the self-service platform that enables teams to build their own AI agents
  • Establish architectural patterns, technical standards, and best practices for LLM integration, AI agent development, and production AI systems serving developers
  • Lead technical strategy for platform capabilities including deployment frameworks (ArgoCD/GitOps), observability integration (Grafana), security controls, and operational tooling for AI developer tools
  • Design highly available, scalable infrastructure for hosting AI agents and developer tools with predictable performance and intelligent resource management
  • Drive technical decisions on AI technology choices, LLM provider strategies, prompt engineering approaches, and agent orchestration frameworks
  • Partner with Senior Manager and product leadership to align technical architecture with business objectives and productization opportunities
  • Architect and build production-ready AI agents for developer productivity including code review assistants, test generators, deployment diagnostics, and incident response automation
  • Design and implement the self-service platform infrastructure that reduces time-to-production for new AI tools from weeks to days
  • Build systems that accelerate adoption of AI-native development tools (Claude Code, Cursor, Warp) across Docker's engineering organization
  • Establish reliability, security, and performance standards for AI systems including SLOs, monitoring, incident response, and cost management
What we offer
What we offer
  • Freedom & flexibility
  • fit your work around your life
  • Designated quarterly Whaleness Days plus end of year Whaleness break
  • Home office setup
  • we want you comfortable while you work
  • 16 weeks of paid Parental leave
  • Technology stipend equivalent to $100 net/month
  • PTO plan that encourages you to take time to do the things you enjoy
  • Training stipend for conferences, courses and classes
  • Equity
  • Fulltime
Read More
Arrow Right

Software Engineer III, AI Developer Tools

At Docker, we make app development easier so developers can focus on what matter...
Location
Location
United States , Seattle
Salary
Salary:
157000.00 - 222750.00 USD / Year
docker.com Logo
Docker
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years building production-grade backend systems or developer-facing tools with strong software engineering fundamentals
  • Hands-on production experience with AI/ML technologies including practical experience with LLM APIs (OpenAI, Anthropic, etc.), prompt engineering, and AI agent development
  • Proficiency in Go (preferred), Rust, Java, or Python with strong software engineering fundamentals
  • Experience designing and building distributed systems, microservices, or platform infrastructure
  • Strong understanding of cloud-native systems (AWS, GCP, or Azure), APIs, and data stores
  • Solid grasp of CI/CD, automated testing, code review practices, and modern development workflows
  • Demonstrated ability to work independently on day-to-day work with general guidance on new projects
  • Product-minded approach to building developer tools with focus on user experience and measurable outcomes
  • Excellent communication skills in remote, asynchronous environments with ability to document technical decisions clearly
  • Ability to build effective working relationships across multiple teams
Job Responsibility
Job Responsibility
  • Build AI-Powered Developer Tools: Design, implement, and ship production-ready AI agents and tools that accelerate developer productivity such as code review and refactoring assistants, automated test generators, local environment setup tools, deployment pipeline diagnostic agents, and agents that simplify on-call tasks when handling incidents
  • Implement LLM Integrations: Build robust, production-grade integrations with LLM APIs (OpenAI, Anthropic, etc.) such as prompt engineering, response parsing, error handling, rate limiting, cost management, and performance optimization
  • Develop Agent Orchestration Systems: Create agent frameworks and orchestration systems that enable complex multi-step workflows, tool calling, context management, and agent-to-agent communication
  • Contribute to Platform Infrastructure: Build self-service platform capabilities that enable teams across Docker to rapidly deploy and operate their own AI developer tools such as deployment pipelines, observability integration, security controls, and operational tooling
  • Drive Adoption of AI-Native Development: Build tools and programs that accelerate adoption of AI developer tools such as Claude Code, Cursor, and Warp across Docker's engineering organization
  • Ensure Production Quality: Write well-tested code with strong test coverage (unit, integration, end-to-end)
  • establish monitoring, alerting, and operational excellence for AI systems
  • Collaborate Cross-Functionally: Partner with Principal Engineer and Senior Engineers on architecture, work with product and design teams on features and UX, and collaborate with platform teams (Infrastructure, Security, Data) on integrations
  • build effective partnerships across multiple teams
  • Act as Technical Resource: Help teammates solve problems and share knowledge through code reviews and technical discussions
What we offer
What we offer
  • Freedom & flexibility
  • fit your work around your life
  • Designated quarterly Whaleness Days plus end of year Whaleness break
  • Home office setup
  • we want you comfortable while you work
  • 16 weeks of paid Parental leave
  • Technology stipend equivalent to $100 net/month
  • PTO plan that encourages you to take time to do the things you enjoy
  • Training stipend for conferences, courses and classes
  • Equity
  • Fulltime
Read More
Arrow Right

Senior Software Automation Engineer (AI & IT Systems)

The purpose of this role is to replace manual, process heavy workflows with scal...
Location
Location
Mexico
Salary
Salary:
Not provided
appen.com Logo
Appen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 plus years of experience in software engineering, automation, platform engineering, or related roles
  • Strong foundation in software engineering principles including code quality, testing, and maintainability
  • Proficiency in Python, JavaScript, Go, or similar languages used for automation and internal tooling
  • Hands on experience building and operating production systems or internal tools
  • Demonstrated experience applying AI or LLM based solutions in real world production environments
  • Background in SRE, DevOps, Platform Engineering, or IT automation with experience operating systems at scale
  • Experience using AI assisted development tools such as Cursor, GitHub Copilot, Claude Code, or similar tools
  • Solid understanding of IT systems, enterprise environments, and service management processes
  • Experience participating in on call rotations and supporting production or internal applications
  • Strong communication skills with the ability to influence teams across different technical backgrounds
Job Responsibility
Job Responsibility
  • Design software-based automation and AI driven systems that improve efficiency, reliability, and scalability of IT and internal platforms
  • Build internal tools, services, and integrations using APIs, event driven systems, and CI/CD practices
  • Automate operational workflows using AI and LLM technologies to support decision making and incident analysis
  • Improve system reliability by turning incident trends and operational pain points into durable automation
  • Reduce recurring production and internal application issues through structured engineering solutions
  • Strengthen engineering standards across IT automation through version control, testing, documentation, and observability
  • Influence teams to adopt software engineering approaches instead of manual or ticket-based models
  • Maintain internal applications through participation in a business hour on call rotation and use operational insights to drive continuous improvement
  • Deliver solutions that meet governance, privacy, and reliability standards in collaboration with Security, Compliance, Platform, and Business Systems teams
  • Fulltime
Read More
Arrow Right

Software Engineer II, AI Developer Tools

At Docker, we make app development easier so developers can focus on what matter...
Location
Location
United States , Seattle
Salary
Salary:
128000.00 - 181500.00 USD / Year
docker.com Logo
Docker
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years building backend systems, APIs, or developer-facing tools with strong software engineering fundamentals
  • Proficiency in Go (preferred), Rust, Java, or Python with understanding of data structures, algorithms, and design patterns
  • Basic understanding of AI/ML concepts with eagerness to learn about LLM APIs, prompt engineering, and AI agent development through hands-on work
  • Experience with cloud platforms (AWS, GCP, or Azure) and understanding of distributed systems or microservices
  • Familiarity with CI/CD pipelines, automated testing, version control (Git), and modern development workflows
  • Strong problem-solving skills with ability to work through technical challenges with guidance from senior engineers
  • Good communication skills in remote, asynchronous environments with ability to document technical decisions
  • Collaborative mindset with eagerness to learn from code reviews and feedback
  • Self-motivated with ability to work autonomously while knowing when to ask for help
  • Passion for developer tools and user experience
Job Responsibility
Job Responsibility
  • Build AI Developer Tool Features: Implement features for AI-powered developer tools such as code review assistants, test generators, deployment diagnostics, and on-call assistance tools
  • Implement LLM Integrations: Build integrations with LLM APIs (OpenAI, Anthropic, etc.) such as prompt engineering, response handling, error management, and performance optimization
  • Contribute to Platform Infrastructure: Help build self-service platform capabilities such as deployment pipelines, observability integration, security controls, and operational tooling that enable teams to rapidly deploy AI developer tools
  • Support AI-Native Development Adoption: Contribute to tools and programs that help teams adopt AI developer tools such as Claude Code, Cursor, and Warp across Docker's engineering organization
  • Write Quality Code: Develop well-tested code with unit and integration tests
  • follow team coding standards and participate actively in code reviews to learn best practices
  • Maintain Production Systems: Assist with monitoring, alerting, and troubleshooting production AI systems
  • participate in incident response and learn operational best practices
  • Collaborate and Learn: Work closely with Senior Engineers and Principal Engineer on technical designs
  • ask questions, seek feedback, and continuously improve your skills in AI/LLM technologies and platform engineering
What we offer
What we offer
  • Freedom & flexibility
  • fit your work around your life
  • Designated quarterly Whaleness Days plus end of year Whaleness break
  • Home office setup
  • we want you comfortable while you work
  • 16 weeks of paid Parental leave
  • Technology stipend equivalent to $100 net/month
  • PTO plan that encourages you to take time to do the things you enjoy
  • Training stipend for conferences, courses and classes
  • Equity
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, AI Developer Tools

At Docker, we make app development easier so developers can focus on what matter...
Location
Location
United States , Seattle
Salary
Salary:
184600.00 - 260700.00 USD / Year
docker.com Logo
Docker
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years building production-grade backend systems or developer-facing tools
  • Hands-on experience with AI/ML technologies such as practical production experience with LLM APIs (OpenAI, Anthropic, etc.), prompt engineering, or AI agent development
  • Proficiency in Go (preferred), Rust, Java, or Python with strong software engineering fundamentals
  • Experience designing and building distributed systems, microservices, or platform infrastructure
  • Strong understanding of cloud-native systems (AWS, GCP, or Azure), APIs, and data stores
  • Solid grasp of CI/CD, automated testing, code review practices, and modern development workflows
  • Product-minded approach to building developer tools with focus on user experience and measurable outcomes
  • Excellent communication skills in remote, asynchronous environments with ability to document technical decisions clearly
  • Ownership mentality with bias for action and iterative delivery
  • Comfortable working autonomously across distributed teams and navigating ambiguity
Job Responsibility
Job Responsibility
  • Build AI-Powered Developer Tools: Design, implement, and ship production-ready AI agents and tools that accelerate developer productivity such as code review and refactoring assistants, automated test generators, local environment setup tools, deployment pipeline diagnostic agents, and on-call assistance tools
  • Implement LLM Integrations: Build robust, production-grade integrations with LLM APIs (OpenAI, Anthropic, etc.) such as prompt engineering, response parsing, error handling, rate limiting, cost management, and performance optimization
  • Develop Agent Orchestration Systems: Create agent frameworks and orchestration systems that enable complex multi-step workflows, tool calling, context management, and agent-to-agent communication
  • Contribute to Platform Infrastructure: Build self-service platform capabilities that enable teams across Docker to rapidly deploy and operate their own AI developer tools such as deployment pipelines, observability integration, security controls, and operational tooling
  • Drive Adoption of AI-Native Development: Build tools and programs that accelerate adoption of AI developer tools such as Claude Code, Cursor, and Warp across Docker's engineering organization
  • Ensure Production Quality: Write well-tested code with strong test coverage (unit, integration, end-to-end)
  • establish monitoring, alerting, and operational excellence for AI systems
  • Collaborate Cross-Functionally: Partner with Principal Engineer on architecture, work with product and design teams on features and UX, and collaborate with platform teams (Infrastructure, Security, Data) on integrations
  • Participate in Operations: Take part in on-call rotation for AI developer tools
  • respond to incidents, debug production issues, and drive continuous improvement of system reliability
What we offer
What we offer
  • Freedom & flexibility
  • fit your work around your life
  • Designated quarterly Whaleness Days plus end of year Whaleness break
  • Home office setup
  • we want you comfortable while you work
  • 16 weeks of paid Parental leave
  • Technology stipend equivalent to $100 net/month
  • PTO plan that encourages you to take time to do the things you enjoy
  • Training stipend for conferences, courses and classes
  • Equity
  • Fulltime
Read More
Arrow Right