CrawlJobs Logo

Research Scientist / Engineer — Multimodal Agent

lumalabs.ai Logo

Luma AI

Location Icon

Location:
United States , Palo Alto

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

250000.00 - 450000.00 USD / Year

Job Description:

This is a rare and foundational opportunity to define the future of multimodal AI. You will be at the forefront of building and training large-scale multimodal models, directly impacting how users interact with pixels. This role offers the chance to bridge cutting-edge research with magical, shipped products, working end-to-end on novel problems with no existing playbook.

Job Responsibility:

  • Architect large-scale multimodal agentic models that use reasoning, planning, coding, and tool calling to achieve complex, multi-step multimodal work
  • Hillclimbing existing tasks and formulating new tasks through data
  • Design, implement, and run robust data pipelines for constructing, enriching, and filtering massive pixel datasets
  • Train large-scale multimodal models on massive datasets and GPU clusters
  • Define and build novel evaluation frameworks to measure multimodal agents

Requirements:

  • Strong foundation in machine learning, foundation models and agentic systems
  • Deep understanding of agentic systems and approaches in LLM/VLM reasoning, coding models, LLM/VLM tool calling
  • Hands-on experience with PyTorch and large-scale training (distributed, mixed precision, large datasets)

Nice to have:

  • Experience in the following around data, modeling, or evaluation: State-of-the-art foundation models in reasoning
  • State-of-the-art foundation models in coding
  • State-of-the-art foundation models in tool calling
  • State-of-the-art multimodal agents

Additional Information:

Job Posted:
April 10, 2026

Employment Type:
Fulltime
Work Type:
Remote work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Research Scientist / Engineer — Multimodal Agent

Research Scientist / Engineer — Foundation Model

This is a rare and foundational opportunity to define the future of multimodal A...
Location
Location
United States , Palo Alto
Salary
Salary:
Not provided
lumalabs.ai Logo
Luma AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong foundation in machine learning, foundation models and agentic systems
  • Deep understanding of agentic systems and approaches in LLM/VLM reasoning, coding models, LLM/VLM tool calling
  • Hands-on experience with PyTorch and large-scale training (distributed, mixed precision, large datasets)
  • Able to contribute continuous 6 months in the internship
Job Responsibility
Job Responsibility
  • Architect large-scale multimodal agentic models that use reasoning, planning, coding, and tool calling to achieve complex, multi-step multimodal work
  • Design, implement, and run robust data pipelines for constructing, enriching, and filtering agentic datasets
  • Train large-scale multimodal agents on massive datasets and GPU clusters
  • Define and build novel evaluation frameworks to measure multimodal agents
  • Fulltime
Read More
Arrow Right

Senior Research Engineer

As a Senior Research Engineer at Microsoft, you will advance Microsoft’s mission...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, Physics, or a related field and 4 or more years in applied ML or AI research and product engineering
  • OR Master’s degree and 3 or more years in applied ML or AI research and product engineering
  • OR PhD in a relevant field and 2 or more years with generative AI, LLMs, or related ML algorithms
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Bringing State-of-the-Art Research to Products
  • Design and implement AI systems using foundation models, prompt engineering, retrieval-augmented generation, multi-agent architectures, and classic ML
  • Fine-tune large language models on domain-specific data and evaluate via offline and online methods such as A/B testing, telemetry, and shadow deployments
  • Build and harden prototypes into production-ready services using robust software engineering and MLOps practices
  • Drive original research and thought leadership (whitepapers, internal notes, patents)
  • convert insights into shipped capabilities
  • Research Translation: Continuously review emerging work
  • identify high-potential methods and adapt them to Microsoft problem spaces
  • End-to-End System Development
  • ML Design & Architecture: Own end-to-end pipeline from data prep, training, evaluation, deployment, and feedback loops
  • Fulltime
Read More
Arrow Right

Deep Research Agent Tech Lead

Scale AI is seeking a highly technical and strategic Staff / Senior Staff Machin...
Location
Location
United States , San Francisco; New York
Salary
Salary:
252000.00 - 315000.00 USD / Year
scale.com Logo
Scale
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Electrical Engineering, a related field, or equivalent practical experience
  • 8+ years of experience in software development, with at least 6 years focused on Machine Learning, Deep Learning, or Applied Research in a production environment
  • 2+ years of experience in a formal or informal Technical Leadership role (Team Lead, Tech Lead) with a focus on setting technical direction for a domain
  • Deep expertise in Generative AI and Large Language Models (LLMs)
  • Demonstrated experience designing, building, and deploying AI Agents or complex Agentic systems in production at scale
  • Experience with large-scale distributed systems and real-time data processing
Job Responsibility
Job Responsibility
  • Set the Technical Roadmap: Define and own the technical strategy, architecture, and roadmap for Deep Research Agents for the Enterprise, ensuring alignment with Scale AI’s overall AI strategy and business goals
  • Drive Breakthrough Research to Production: Lead the end-to-end development, from initial research to production deployment, to landing on customer impact, with a focus on integrating diverse data modalities
  • Advanced Knowledge Retrieval: Architect and implement state-of-the-art retrieval systems to ensure the agents provide accurate and comprehensive answers from public and proprietary data sources from enterprises
  • Data analysis: Design and champion the development of data analysis agents that accurately translate complex natural language queries into executable SQL/code against diverse enterprise data schemas
  • Multimodal Intelligence: Lead the integration of Multimodal AI capabilities to process and extract structured information from visual documents, tables, and forms, enriching the agent's knowledge base
  • Architecture & Design: Design and champion highly scalable, reliable, and low-latency infrastructure and frameworks for building, orchestrating, and evaluating multi-agent systems at enterprise scale
  • Technical Excellence: Serve as the technical authority for the team, leading design reviews, defining ML engineering best practices, and ensuring code quality, security, and operational excellence for all agent systems
  • Lead and Mentor: Technically lead and mentor a team of Machine Learning Engineers and Research Scientists, fostering a culture of innovation, rigorous engineering, rapid iteration, and technical depth
  • Recruiting & Growth: Partner with management to hire, onboard, and grow top-tier talent, helping to shape the long-term structure and capabilities of the team
  • Cross-Functional Influence: Collaborate effectively with Product Managers, Data Scientists, and other engineering/science teams to translate ambiguous, high-level business problems into concrete, executable technical specifications and impactful agent solutions
What we offer
What we offer
  • Comprehensive health, dental and vision coverage
  • retirement benefits
  • a learning and development stipend
  • generous PTO
  • Fulltime
Read More
Arrow Right

Senior Applied Scientist

Microsoft Cloud Operations and Innovation (CO+I) underpins Microsoft’s global cl...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (e.g., statistics predictive analytics, research) OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience (e.g., statistics, predictive analytics, research) OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 1+ year(s) related experience (e.g., statistics, predictive analytics, research) OR equivalent experience.
  • Experience with Azure technology Stack
  • Experience with parameter‑efficient fine‑tuning techniques (e.g., LoRA) for LLMs or multimodal models.
  • Hands‑on experience building agentic or tool‑augmented LLM systems, including function calling, planners, and API/tool integration.
  • Experience with advanced RAG architectures, such as hybrid retrieval, reranking, grounding, and citation strategies.
  • Strong AIOps depth, including quality drift detection, alerting, rollback, and telemetry‑driven optimization.
  • Experience optimizing systems for latency, reliability, scalability, and cost efficiency at enterprise scale.
  • Prior experience working on mission‑critical or large‑scale enterprise AI systems.
  • Demonstrated mentorship or technical leadership within applied science or engineering teams.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
Job Responsibility
Job Responsibility
  • Advance Applied AI Research to Improve Quality and Reliability
  • Apply deep expertise in Generative AI, deep learning, NLP, and multimodal models to translate cutting-edge research into high-impact, production-ready AI solutions.
  • Design and execute experiments that measurably improve agent planning, memory, grounding, reasoning, and long‑horizon task completion.
  • Perform lightweight fine‑tuning (e.g., LoRA and related techniques) on multimodal and large multimodal language models—to improve entity recognition, reasoning accuracy, and response quality.
  • Implement state‑of‑the‑art approaches using foundation models, advanced prompt engineering, RAG, knowledge graphs, and multi‑agent architectures, complemented by classical ML techniques where appropriate.
  • Build and Ship Scalable Agentic AI Systems
  • Architect and implement end‑to‑end agent workflows that decompose user intent into executable plans, intelligently select and invoke tools, and recover gracefully from errors and edge cases.
  • Rapidly prototype solutions and partner with engineering teams to drive production deployment, including debugging live systems and building AIOps workflows.
  • Use data and telemetry to identify AI quality gaps, generate insights, and deliver proofs of concept that apply research innovations to real‑world operational challenges.
  • Design robust multi‑step reasoning and tool‑use strategies, including function calling, code execution, APIs, and secure connectors, with strong safety and reliability guardrails.
  • Fulltime
Read More
Arrow Right

Senior Research Engineer

As a Senior Research Engineer at Microsoft, you will help advance Microsoft’s mi...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience
  • Proficiency in Python and at least one deep learning framework such as PyTorch, JAX, or TensorFlow
  • Experience deploying Fine Tuned LLMs or multimodal models in live production environments
  • Experience shipping and maintaining production AI systems
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Build AI-First Contact Center Experiences
  • Bringing State-of-the-Art Research to Products
  • Design and implement AI systems using foundation models, prompt engineering, retrieval-augmented generation, multi-agent architectures, and classic ML
  • Fine-tune large language models on domain-specific data and evaluate via offline and online methods such as A/B testing, telemetry, and shadow deployments
  • Build and harden prototypes into production-ready services using robust software engineering and MLOps practices
  • Drive original research and thought leadership (whitepapers, internal notes, patents)
  • convert insights into shipped capabilities
  • Research Translation: Continuously review emerging work
  • identify high-potential methods and adapt them to Microsoft problem spaces
  • Partner with product teams to improve customer and agent outcomes
  • Fulltime
Read More
Arrow Right

AI Research Scientist, Multimodal Generation

Meta is seeking an AI Research Scientist to join our Multimodal Generation Resea...
Location
Location
Salary
Salary:
154000.00 - 217000.00 USD / Year
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • PhD in Computer Science, Machine Learning, or a relevant technical field
  • Practical experience with pre-training, mid-training or SFT data curation for large foundational models and experience working with organic, synthetic, agentic, or reasoning data for Multimodal LLMs
  • Direct experience in Generative AI and LLM research
  • Programming experience in Python and hands-on experience with frameworks such as PyTorch
Job Responsibility
Job Responsibility
  • Develop algorithms based on state-of-the-art machine learning and neural network methodologies
  • Advance our understanding of data research, such as how to overcome data walls and how best to create synthetic data
  • Post-train foundation models using techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), Direct Preference Optimization (DPO), and Low-Rank Adaptation (LoRA)
  • Work towards long-term research/development goals, while identifying intermediate milestones
  • Conduct research that enables learning the semantics of data across multiple modalities (audio, images, video, text, and other modalities)
  • Prioritize research that can be applied to Meta's product development
What we offer
What we offer
  • bonus
  • equity
  • benefits
Read More
Arrow Right

Member of Technical Staff, Integration/RL Team (Research Engineer)

The integration team is responsible for developing and scaling machine learning ...
Location
Location
Salary
Salary:
Not provided
cohere.com Logo
Cohere
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Extremely strong software engineering skills
  • Value test-driven development methods, clean code, and strive to reduce technical debts at all levels
  • Proficiency in Python and related ML frameworks such as JAX, Pytorch and/or XLA/MLIR
  • Experience using and debugging large-scale distributed training strategies (memory/speed profiling)
  • [Bonus] Experience with distributed training infrastructures (Kubernetes) and associated frameworks (Ray)
  • [Bonus] Hands-on experience with the post-training phase of model training, with a strong emphasis on scalability and performance
  • [Bonus] Experience in ML, LLM and RL academic research
Job Responsibility
Job Responsibility
  • Design and write high-performing and scalable software for training models
  • Develop new tools to support and accelerate research and LLM training
  • Coordinate with other engineering teams (Infrastructure, Efficiency, Serving) and the scientific teams (Agent, Multimodal, Multilingual, etc.) to create a strong and integrated post-training ecosystem
  • Craft and implement techniques to improve performance and speed up our training cycles, both on SFT, offline preference, and the RL regime
  • Research, implement, and experiment with ideas on our cluster and data infrastructure
  • Collaborate, Collaborate, and Collaborate with other scientists, engineers, and teams!
What we offer
What we offer
  • An open and inclusive culture and work environment
  • Work closely with a team on the cutting edge of AI research
  • Weekly lunch stipend, in-office lunches & snacks
  • Full health and dental benefits, including a separate budget to take care of your mental health
  • 100% Parental Leave top-up for up to 6 months
  • Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
  • Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
  • 6 weeks of vacation (30 working days!)
  • Fulltime
Read More
Arrow Right

Senior Research Engineer

As a Research Engineer at Microsoft, you will set the technical vision and lead ...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Proven track record leading large-scale AI systems and cross-org initiatives that shipped
  • Solid software engineering foundations and hands-on depth in Python plus deep-learning frameworks (PyTorch/ TensorFlow) and modern MLOps/tooling
  • Experience shipping and maintaining production AI systems
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Architect and deliver complex AI systems across model development, data, infra, evaluation, and deployment spanning multiple product lines
  • Set technical direction for large programs
  • drive alignment across Research, Engineering, and Product
  • Build and harden prototypes into production-ready services using robust software engineering and MLOps practices
  • Integrate LLMs, multimodal models, multi-agent architectures, and RAG into Microsoft’s ecosystem
  • Establish best practices for MLOps, governance, and Responsible AI, compliant with Microsoft principles and industry standards
  • Drive original research and thought leadership (whitepapers, internal notes, patents)
  • convert insights into shipped capabilities
  • Research Translation: Continuously review emerging work
  • identify high-potential methods and adapt them to Microsoft problem spaces
  • Fulltime
Read More
Arrow Right