CrawlJobs Logo

Member of Technical Staff, Training Infra Engineer

· Job Posted February 20, 2026
Apply Position
Job Link Share

Job Description

Contribute in and provide strong support for model training pipelines, ship state of the art models to production, and bridge the gap between research and production. We have one of the highest ratio of compute to engineers in the world. We do not delineate strongly between engineering and research. Everyone will contribute to writing production code and supporting our research effort depending on individual interest and organizational needs. We have all the compute, data, and talent available for you to do your best work.

Job Responsibility

  • Design and write high-performant and scalable software for training
  • Improve our training setup from an infrastructure and codebase performance standpoint
  • Craft and implement tools to speed up our training cycles and improve the overall efficacy of our training infrastructure
  • Research, implement, and experiment with ideas on our supercompute and data infrastructure
  • Learn from and work with the best researchers in the field

Requirements

  • Extremely strong software engineering skills
  • Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR
  • Experience with distributed training infrastructures (Kubernetes, Slurm) and associated frameworks (Ray)
  • Experience using large-scale distributed training strategies
  • Hands on experience on training large model at scale and having contributed to the tooling and/or setup of the training infrastructure

Nice to have

paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)

What we offer

  • An open and inclusive culture and work environment
  • Work closely with a team on the cutting edge of AI research
  • Weekly lunch stipend, in-office lunches & snacks
  • Full health and dental benefits, including a separate budget to take care of your mental health
  • 100% Parental Leave top-up for up to 6 months
  • Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
  • Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
  • 6 weeks of vacation (30 working days!)

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Member of Technical Staff, Training Infra Engineer

8 matching positions

Member of Technical Staff - Data Infra - MAI Superintelligence Team

Help build the world’s most advanced multimodal dataset at Microsoft AI. We are ...
Location
Location
United States , Mountain View
Salary
Salary:
139900.00 - 274800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling or data engineering work
  • OR Master’s Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ year(s) experience in business analytics, data science, software development, or data engineering work
  • OR equivalent experience
  • Bachelor’s Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8+ years experience in business analytics, data science, software development, data modeling or data engineering work
  • OR Master’s Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years of business analytics, data science, software development, data modeling or data engineering work experience
  • OR equivalent experience
Job Responsibility
Job Responsibility
  • Design and develop data pipelines that ingest enormous amounts of multi-modal training data (text, audio, images, video)
  • Own and maintain critical data infrastructures, including spark, ray, vector databases, and others
  • Build and maintain cutting-edge infrastructure that can store and process the petabytes of data needed to power models
  • Partner with the pretraining and post-training teams to improve our data recipe by rigorous and careful experimentation
  • Embody our culture and values
  • Fulltime
Read More
Arrow Right

Member of Technical Staff - Data Infra - MAI Superintelligence Team

Help build the world’s most advanced multimodal dataset at Microsoft AI. We are ...
Location
Location
United States , Mountain View
Salary
Salary:
163000.00 - 296400.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR equivalent experience
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 12+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 15+ years experience in business analytics, data science, software development, data modeling, or data engineering
  • OR equivalent experience
  • 4+ years experience with data governance, data compliance and/or data security
  • Passionate about the role of data in large-scale AI model training
  • Thrive in a highly collaborative, fast-paced environment
  • Have a high degree of expertise and pay close attention to details
Job Responsibility
Job Responsibility
  • Design and develop data pipelines that ingest enormous amounts of multi-modal training data (text, audio, images, video)
  • Own and maintain critical data infrastructures, including spark, ray, vector databases, and others
  • Build and maintain cutting-edge infrastructure that can store and process the petabytes of data needed to power models
  • Partner with the pretraining and post-training teams to improve our data recipe by rigorous and careful experimentation
  • Embody our culture and values
  • Fulltime
Read More
Arrow Right

Member of Technical Staff, Multimodal Infrastructure

Microsoft AI is looking for a Member of Technical Staff, Multimodal Infrastructu...
Location
Location
United States , Mountain View
Salary
Salary:
139900.00 - 274800.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science, or related technical discipline AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Experience in multi-modal data processing: Strong proficiency in distributed data processing infra (resource utilization management, fault tolerance, ray & spark) and CPU/GPU batch processing optimizations
  • Experience with state-of-art model inference and serving frameworks
  • Experience with image/video/audio data processing
  • Experience with common data formats for efficient I/O
  • Experience in multi-modal pretraining and post-training: Strong proficiency in deep learning frameworks such as PyTorch, Megatron and Deepspeed
  • Knowledge of auto-regressive and diffusion transformer models
  • Experience with distributed training techniques such as data parallelism, model parallelism, and pipeline parallelism
  • Proven experiences in at least one of the following areas: image/video generation and editing
  • efficient architectures (e.g., MoE, window attention)
Job Responsibility
Job Responsibility
  • Design, develop and maintain large-scale multimodal data processing pipelines
  • Design, develop and maintain large-scale multimodal model pretraining and post-training frameworks
  • Design, develop and maintain large-scale multimodal model inference and serving frameworks
  • Work with research scientists and product engineers to solve infra-related problems
  • Find a path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively
  • Enjoy working in a fast-paced, design-driven, product development cycle
  • Embody our Culture and Values
  • Fulltime
Read More
Arrow Right

Staff Software Engineer - AI/ML Infra

GEICO AI platform and Infrastructure team is seeking an exceptional Senior ML Pl...
Location
Location
United States , Chevy Chase; New York City; Palo Alto
Salary
Salary:
115000.00 USD / Year
geico.com Logo
Geico
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer science, Engineering, or related technical field (or equivalent experience)
  • 8+ years of software engineering experience with focus on infrastructure, platform engineering, or MLOps
  • 3+ years of hands-on experience with machine learning infrastructure and deployment at scale
  • 2+ years of experience working with Large Language Models and transformer architectures
  • Proficient in Python
  • strong skills in Go, Rust, or Java preferred
  • Proven experience working with open source LLMs (Llama 2/3, Qwen, Mistral, Gemma, Code Llama, etc.)
  • Proficient in Kubernetes including custom operators, helm charts, and GPU scheduling
  • Deep expertise in Azure services (AKS, Azure ML, Container Registry, Storage, Networking)
  • Experience implementing and operating feature stores (Chronon, Feast, Tecton, Azure ML Feature Store, or custom solutions)
Job Responsibility
Job Responsibility
  • Design and implement scalable infrastructure for training, fine-tuning, and serving open source LLMs (Llama, Mistral, Gemma, etc.)
  • Architect and manage Kubernetes clusters for ML workloads, including GPU scheduling, autoscaling, and resource optimization
  • Design, implement, and maintain feature stores for ML model training and inference pipelines
  • Build and optimize LLM inference systems using frameworks like vLLM, TensorRT-LLM, and custom serving solutions
  • Ensure 99.9%+ uptime for ML platforms through robust monitoring, alerting, and incident response procedures
  • Design and implement ML platforms using DataRobot, Azure Machine Learning, Azure Kubernetes Service (AKS), and Azure Container Instances
  • Develop and maintain infrastructure using Terraform, ARM templates, and Azure DevOps
  • Implement cost-effective solutions for GPU compute, storage, and networking across Azure regions
  • Ensure ML platforms meet enterprise security standards and regulatory compliance requirements
  • Evaluate and potentially implement hybrid cloud solutions with AWS/GCP as backup or specialized use cases
What we offer
What we offer
  • Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being
  • Financial benefits including market-competitive compensation
  • a 401K savings plan vested from day one that offers a 6% match
  • performance and recognition-based incentives
  • and tuition assistance
  • Access to additional benefits like mental healthcare as well as fertility and adoption assistance
  • Supports flexibility- We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year
  • Fulltime
Read More
Arrow Right

Staff Software Engineer - AI/ML Infra

GEICO AI platform and Infrastructure team is seeking an exceptional Senior ML Pl...
Location
Location
United States , Palo Alto
Salary
Salary:
90000.00 USD / Year
geico.com Logo
Geico
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer science, Engineering, or related technical field (or equivalent experience)
  • 8+ years of software engineering experience with focus on infrastructure, platform engineering, or MLOps
  • 3+ years of hands-on experience with machine learning infrastructure and deployment at scale
  • 2+ years of experience working with Large Language Models and transformer architectures
  • Proficient in Python
  • strong skills in Go, Rust, or Java preferred
  • Proven experience working with open source LLMs (Llama 2/3, Qwen, Mistral, Gemma, Code Llama, etc.)
  • Proficient in Kubernetes including custom operators, helm charts, and GPU scheduling
  • Deep expertise in Azure services (AKS, Azure ML, Container Registry, Storage, Networking)
  • Experience implementing and operating feature stores (Chronon, Feast, Tecton, Azure ML Feature Store, or custom solutions)
Job Responsibility
Job Responsibility
  • Design and implement scalable infrastructure for training, fine-tuning, and serving open source LLMs (Llama, Mistral, Gemma, etc.)
  • Architect and manage Kubernetes clusters for ML workloads, including GPU scheduling, autoscaling, and resource optimization
  • Design, implement, and maintain feature stores for ML model training and inference pipelines
  • Build and optimize LLM inference systems using frameworks like vLLM, TensorRT-LLM, and custom serving solutions
  • Ensure 99.9%+ uptime for ML platforms through robust monitoring, alerting, and incident response procedures
  • Design and implement ML platforms using DataRobot, Azure Machine Learning, Azure Kubernetes Service (AKS), and Azure Container Instances
  • Develop and maintain infrastructure using Terraform, ARM templates, and Azure DevOps
  • Implement cost-effective solutions for GPU compute, storage, and networking across Azure regions
  • Ensure ML platforms meet enterprise security standards and regulatory compliance requirements
  • Evaluate and potentially implement hybrid cloud solutions with AWS/GCP as backup or specialized use cases
What we offer
What we offer
  • Comprehensive Total Rewards program that offers personalized coverage tailor-made for you and your family’s overall well-being
  • Financial benefits including market-competitive compensation
  • a 401K savings plan vested from day one that offers a 6% match
  • performance and recognition-based incentives
  • and tuition assistance
  • Access to additional benefits like mental healthcare as well as fertility and adoption assistance
  • Supports flexibility- We provide workplace flexibility as well as our GEICO Flex program, which offers the ability to work from anywhere in the US for up to four weeks per year
  • Fulltime
Read More
Arrow Right

Production Engineering Manager, Rotational Network Engineering (RNE) Program

This is unique role within Meta's Infrastructure organization. Meta’s Rotational...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of Networking, System Administration, Software Engineering or Product Development experience
  • Familiarity with source control, software development cycles and practices
  • Experience with launching and iterating on product, services, tools or technical frameworks
  • Experience managing an engineering team
  • B.S. in Engineering or equivalent experience
  • Analytical and troubleshooting skills
Job Responsibility
Job Responsibility
  • Build a plan for each team member with technical leads and mentors from Network Infrastructure, Backbone, and Datacenter Engineering
  • Establish and foster fruitful working relationships with various stakeholders and teams within Network Infra
  • Manage and grow multi-disciplinary recruiting plans across universities and industry
  • Develop and manage work plans from recruiting, to mentor and task selection to team assignment
  • Manage expectations of all interested parties: define clear program roadmap with key deliverables and milestone dates, maintain program information wiki pages, and identify and communicate risks and adjustments to the overall program to meet recruitment demands from Network Engineering teams
  • Understand the network product delivery cycle
  • Work closely with dedicated recruiting staff to expand the team, including sourcing candidates, interviewing candidates, participating in conferences/events, and on-boarding new employees
  • Influence Network Infrastructure teams for their buy in to the program, obtain agreement to provide mentors and projects and to consider rotational engineers as one of their hiring pipelines
  • Enable and unblock engineers through coaching, learning, and mentorship programs
  • Responsible for people management of a team of engineers, providing performance reviews, continual feedback, coaching and career growth for direct reports
Read More
Arrow Right

Accountant

We are seeking a detail-oriented Accountant to support daily accounting operatio...
Location
Location
United States , Miami
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Accounting, Finance, or related field
  • 2+ years of accounting experience
  • Knowledge of general accounting principles and financial reporting
  • Experience with general ledger, reconciliations, and close processes
  • Proficiency in Microsoft Excel and accounting software or ERP systems
  • Strong attention to detail, organization, and problem-solving skills
  • Ability to work independently and collaboratively in a team environment
  • Strong written and verbal communication skills
Job Responsibility
Job Responsibility
  • Prepare and record journal entries, accruals, and adjustments
  • Maintain and reconcile general ledger accounts
  • Perform monthly bank and account reconciliations
  • Assist with month-end, quarter-end, and year-end close
  • Prepare financial statements, reports, and supporting schedules
  • Analyze account activity and investigate discrepancies
  • Support accounts payable, accounts receivable, and payroll processes as needed
  • Help ensure compliance with company policies, internal controls, and accounting standards
  • Assist with audits by preparing documentation and responding to requests
  • Contribute to budgeting, forecasting, and variance analysis
What we offer
What we offer
  • medical insurance
  • vision insurance
  • dental insurance
  • life insurance
  • disability insurance
  • 401(k) plan
  • Fulltime
Read More
Arrow Right

Critical Environment Technician Manager

In alignment with our Microsoft values, we are committed to cultivating an inclu...
Location
Location
United States , Mount Pleasant
Salary
Salary:
75400.00 - 167900.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High School Diploma, GED, or equivalent AND 3+ years mission critical services work/applied learning experience (e.g., high availability assembly/manufacturing/critical infrastructure environments such as data centers, oil and gas refineries, hospitals, pharmaceutical, manufacturing, or related fields) OR equivalent experience
  • Ability to work shifts, including shift assignments during non-standard business hours that may include evening, nighttime, weekends, and/or holidays
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • High School Diploma, GED, or equivalent AND 6+ years mission critical services experience OR Associate's Degree or technical trade certification (e.g., military, trade school), or higher-equivalent education AND 5+ years mission-critical services experience OR equivalent experience
  • 1+ year(s) people management experience
  • 1+ year(s) experience in a specialized area (e.g., mechanical field, electrical field, controls field) or related field
Job Responsibility
Job Responsibility
  • People Management: Managers deliver success through empowerment and accountability by modeling, coaching, and caring. Model - Live our culture
  • Embody our values
  • Practice our leadership principles. Coach - Define team objectives and outcomes
  • Enable success across boundaries
  • Help the team adapt and learn. Care - Attract and retain great people
  • Know each individual’s capabilities and aspirations
  • Invest in the growth of others
  • Equipment and Systems Operations: Serve as an operations specialist one or more major area of operations (e.g., electrical, mechanical, controls, generators, and work on advanced tasks independently. Oversee and coach team with the inspection of critical environment-related facility equipment (e.g., controls, heating, ventilation, and air conditioning [HVAC], mechanical systems), building, and grounds regularly for unsafe or abnormal conditions to develop and analyze trends. Monitor performance of maintenance and operations utilizing telemetry, control systems, and other platforms and is able to identify all alarms. Utilize internal computerized maintenance management system (CMMS) to track all equipment assets and to complete work order requests for maintenance work and generate reporting to identify outstanding and ongoing work orders. Safely and quickly respond to and lead an onsite incident response team for all abnormal conditions that impact operations and coordinate with other critical facilities professionals to perform corrective repairs. Enhances, develops new, or follows preexisting emergency operating procedures (EOPs), methods of procedure (MOPs), and standard operating procedures (SOPs) in relation to incidents. Gathers necessary information and creates incident timelines/data, root-cause analyses, and/or action items following an abnormal condition
  • Equipment and Systems Maintenance: Guide, oversee, and perform various types of maintenance (e.g., planned, predictive, corrective) and repairs following methods of procedure (MOPs), and standard operating procedures (SOPs) for one or more disciplines and one or more types of equipment (e.g., electrical, mechanical, cooling systems) and escalate when appropriate. Serve as a subject matter expert for one type of equipment and oversee everyday tasks and troubleshooting within their area of expertise. Have a hands-on understanding of how equipment works within disciplines they have been trained and how to troubleshoot equipment, systems, subsystems, and components independently within their trained discipline(s). Provide and/or assign team to provide necessary escort to third-party contractors, sub contractors, vendors, and service providers on site for all severity leveled procedures. Coordinate and schedule supplier/vendor on-site activities and recognizes circumstances when to stop supplier work to address potential and/or identified concerns. Take part in getting third-party work underway (e.g., making sure systems are properly energized/deenergized), ensuring the work is started and completed in a safe manner in accordance with standard practices, procedures, federal/local legislation, and municipal codes. Advises junior colleagues on inspection and supervision issues. Provides consultation to lower-level colleagues in troubleshooting systems and problems
  • Critical Environment Culture: Understands, follows, ensures, and coaches team on safety and security requirements (e.g., job hazard assessments [JHAs], toolbox talks), and business processes and procedures to properly perform work in a safe, quality, and reliable manner in accordance with applicable federal, state, local, and Microsoft requirements. Proactively ensures safety and security requirements are followed and met for the work of themselves and others. Maintain safe working conditions and escalate immediately when unsafe working conditions are observed. Assesses and identifies appropriate resources and equipment necessary to fully support environmental health and safety (EH&S) objectives. Participates in required meetings, trainings, and necessary handoffs
What we offer
What we offer
  • Benefits and other compensation (details at https://careers.microsoft.com/us/en/us-corporate-pay)
  • Fulltime
Read More
Arrow Right