Principal Engineer, AI Model Lifecycle Job at Crusoe (San Francisco)

Principal Engineer, AI Model Lifecycle

Crusoe

Location:
United States , San Francisco

Category:
IT - Software Development

Contract Type:
Not provided

Salary:

260000.00 - 326000.00 USD / Year

Save Job

Apply Position

Job Description:

The Principal Software Engineer for the Model LifeCycle team will play a crucial role in building a comprehensive managed platform for the entire application development lifecycle, with a specific focus on leveraging Machine Learning models, including Large Language Models (LLMs). This role offers significant 0 → 1 ownership — you'll be designing and building core systems from first principles.

Job Responsibility:

Manage fine-tuning systems for large foundation models (SFT, PEFT, LoRA, adapters), including multi-node orchestration, checkpointing, failure recovery, and cost-efficient scaling
Implement and maintain end-to-end training pipelines for Large Language Models
Distillation and reinforcement learning pipelines (e.g., preference optimization, policy optimization, reward modeling)
Agent execution infrastructure
Dataset, model, and experiment management: versioning, lineage, evaluation, and reproducible fine-tuning at scale
Work closely with product, business, and platform teams to shape the core abstractions and APIs of the system
Influence long-term architectural decisions around training runtimes, scheduling, storage, and model lifecycle management
Contribute to and engage with the open-source LLM ecosystem

Requirements:

Advanced degree in Computer Science, Engineering, or a related field
10-15+ years of industry experience driving impactful projects in the AI Space
Proven track record of delivering early-stage projects under tight deadlines
Expertise in using cloud-based services, such as, elastic compute, object storage, virtual private networks, managed database, etc.
Experience in Generative AI (Large Language Models, Multimodal)
Deep experience with AI infrastructure, including training, inference

Nice to have:

Proficiency in Golang or Python for large-scale, production-level services
Contributions to open-source AI projects such as vLLM or similar frameworks
Performance optimizations on GPU systems and inference frameworks
Experience working with PyTorch
Experience with training and fine-tuning LLMs

What we offer:

Restricted Stock Units
Health insurance package options that include HDHP and PPO, vision, and dental for you and your dependents
Employer contributions to HSA accounts
Paid Parental Leave
Paid life insurance, short-term and long-term disability
Teladoc
401(k) with a 100% match up to 4% of salary
Generous paid time off and holiday schedule
Cell phone reimbursement
Tuition reimbursement
Subscription to the Calm app
MetLife Legal
Company paid commuter benefit
$300/month

Additional Information:

Job Posted:
February 21, 2026

Employment Type:

Fulltime

Work Type:

On-site work

Crusoe - All Job Offers

Job Link Share: