Principal AI Engineer Jobs

Apollo.io

Location:
United States

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

200000.00 - 300000.00 USD / Year

Save Job

Apply Position

Job Description:

We are looking for a Principal AI Engineer to lead the design and deployment of cutting-edge agentic systems, AI assistants, and LLM-powered features. This role is pivotal to Apollo’s AI roadmap, owning the architecture and productionization of intelligent systems that power the workflows of thousands of GTM teams globally. You’ll be a technical thought leader on the AI Engineering team and work alongside product, backend, and platform leaders to drive forward Apollo’s competitive edge in applied AI.

Job Responsibility:

Architect and lead the development of multi-agent systems capable of long-horizon planning, reasoning, and API orchestration
build reusable agentic components that integrate deeply into sales and marketing processes
own and evolve our in-house platform for scalable, low-latency, and cost-efficient LLM and agent deployments
lead design of interfaces powered by natural language understanding and retrieval-augmented generation (RAG)
build embedding-based, intent-aware search and personalization systems tuned to business user needs
drive innovation in personalized outreach generation using context-aware generation pipelines
tune inference pipelines, caching layers, and model selection logic for high-scale, cost-aware performance
define and drive robust offline and online testing methodologies (A/B, sandboxing, human evals) across agents and LLM flows
architect human-in-the-loop systems and telemetry to improve accuracy, UX, and explainability over time

Requirements:

10+ years of software engineering experience
at least 3 years in applied LLM or agentic AI systems (2023–present)
proven success in deploying LLM-powered products used by real users at scale
deep backend & systems engineering expertise with Python, distributed systems, and scalable APIs
familiarity with LangChain, LlamaIndex, or similar orchestration frameworks
experience with RAG pipelines, vector DBs, embedding models, and semantic search tuning
experience managing performance across cloud providers (e.g., AWS Bedrock, OpenAI, Anthropic, etc.)
demonstrated experience building multi-step agents, planning workflows, chaining reasoning steps, and integrating APIs with agent memory/state
comfort with advanced prompting strategies, few-shot and chain-of-thought reasoning, and embedding retrieval setups
strong understanding of AI system evaluation, human ratings, A/B experimentation, and feedback loop pipelines
experience designing safety-aware, reliable LLM systems in production environments
experience owning logging, monitoring, and observability for live AI systems
principal-level ownership
AI-native mentality
applied focus

What we offer:

equity
company bonus or sales commissions/bonuses
401(k) plan
at least 10 paid holidays per year
flex PTO
parental leave
employee assistance program
wellbeing benefits
global travel coverage
life/AD&D/STD/LTD insurance
FSA/HSA
medical benefits
dental benefits
vision benefits

Additional Information:

Job Posted:
December 06, 2025

Employment Type:

Fulltime

Work Type:

Remote work

View All Jobs In This Company

Job Link Share:

Principal AI Engineer