CrawlJobs Logo

Research Scientist, Human AI Interaction

joinhandshake.com Logo

Handshake

Location Icon

Location:
United States , San Francisco, CA, New York, NY

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

350000.00 - 420000.00 USD / Year

Job Description:

As a Research Scientist, Human–AI Interaction, you will play a pivotal role in defining how AI systems support real human work by leading research at the intersection of Human–Computer Interaction (HCI), Large Language Models (LLMs), and task-level benchmarking. You will operate at the frontier of human-centered AI evaluation, with a focus on understanding what people actually do to accomplish meaningful work—and how AI systems change, accelerate, or reshape that activity. Your research will define jobs-to-be-done benchmarks, comparative evaluation frameworks, and empirical methods for measuring human effort, time, quality, and outcomes when working with AI copilots. Additionally, the Handshake AI platform is an interface used by thousands of the top subject matter experts in the world to evaluate AI systems, and offers numerous interesting HCI / HITL-AI research questions that will drive large business impact. You’ll set research direction, establish standards for measuring human activity in AI-mediated workflows, publish papers and open-source code, and lead the development of rigorous, scalable benchmarks that connect human work, AI assistance, and real economic value.

Job Responsibility:

  • Lead high-impact research on jobs-to-be-done benchmarks for AI systems, including: Defining task taxonomies grounded in real professional and economic activities
  • Identifying what constitutes meaningful task completion, quality, and success
  • Translating qualitative work understanding into measurable, repeatable benchmarks
  • Develop methods to measure human activity in AI-mediated workflows
  • Design benchmarks to assess AI-as-a-collaborator/copilot, rather than autonomous agents / basic Q&A
  • Design and run empirical studies of how people use AI to solve tasks, including: Controlled experiments and field studies measuring task performance
  • Instrumentation for capturing fine-grained interaction traces and outcomes
  • Drive strategy for professional-domain AI benchmarks, focusing on: Understanding domain-specific workflows (e.g., analysis, writing, planning, coordination)
  • Grounding benchmark design in how work is actually performed, not idealized tasks
  • Build and prototype AI systems and evaluation infrastructure to support research and Data production, including: LLM-powered copilots and experimental tools used for task-level measurement
  • Benchmark harnesses that evaluate both model behavior and human outcomes
  • Data pipelines for analyzing human–AI interaction at scale
  • The human-in-the-loop experience for Handshake fellows to produce effective evaluations and training data for frontier models, through structured UI/UX interactions with these models
  • Collaborate closely with User Experience Research (UXR) to: Leverage deep qualitative insights into real user behavior and workflows
  • Translate ethnographic and observational findings into formal research constructs
  • Publish and present research that advances the field of human-centered AI benchmarking, with an expectation of regular contributions to top-tier venues such as CHI (Conference on Human Factors in Computing Systems), and related HCI and AI conferences

Requirements:

  • PhD or equivalent experience in Human–Computer Interaction, Computer Science, Cognitive Science, or a related field, with a strong emphasis on empirical evaluation of interactive AI/LLM systems
  • 3+ years of academic or industry research experience post-PhD, including leadership on complex research initiatives and analyzing data from a real AI product
  • Strong publication record, with demonstrated impact in top-tier AI (NeurIPS, ICML, ICLR, ACL) and HCI (CHI) venues
  • Deep expertise in experimental design and measurement, particularly for: Task performance and human activity
  • Comparative evaluation frameworks
  • Mixed-methods research grounded in real-world behavior
  • Strong technical and coding skills, including: Python and data analysis / ML tooling
  • Experience building experimental systems and benchmark infrastructure
  • Familiarity working with LLM APIs, agent frameworks, or AI-assisted tooling
  • Proven ability to define and lead research agendas that connect human work, AI capability, and business or economic impact
  • Strong collaboration skills, especially working across research, engineering, product, and UXR teams

Nice to have:

  • Experience developing benchmarks or evaluation frameworks for human–AI systems or AI-assisted productivity tools
  • Prior work on copilot-style systems, agentic workflows, or automation of professional tasks
  • Familiarity with workplace studies, CSCW, or socio-technical systems research
  • Contributions to open-source tools, datasets, or benchmarks related to task-level evaluation
  • Interest in how AI reshapes labor, productivity, and the future of work
What we offer:
  • Equity in a fast-growing company
  • 401(k) match, competitive compensation, financial coaching
  • Paid parental leave, fertility benefits, parental coaching
  • Medical, dental, and vision, mental health support, $500 wellness stipend
  • $2,000 learning stipend, ongoing development
  • Internet, commuting, and free lunch/gym in our SF office
  • Flexible PTO, 15 holidays + 2 flex days, winter #ShakeBreak where our whole office closes for a week
  • Team outings & referral bonuses

Additional Information:

Job Posted:
February 20, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:
PREMIUM
More languages and countries
+ Unlock 31694 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Research Scientist, Human AI Interaction

Technology Consultant, UX

The Industry Solutions Delivery (ISD) Engineering & Architecture Group (EAG) is ...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 + years’ experience working as User Experience Designer either in Product teams or Consulting Roles with a strong track record
  • M.Des / B.Des / Graduate Diploma / Post Graduate Diploma in Industrial Design, Product Design, Visual Communication, Graphic Design, Interaction Design, User Experience Design, Information and Interface Design, New Media Design, Product and Interface Design, Human Factors, Human Computer Interaction, Cognitive psychology from reputed institutes like IDC IIT Bombay, NID, Srishti School of Art Design and Technology, other IIT’s OR considerable industry experience in lieu of the same
  • Candidate is required to furnish a portfolio of recent usability, design, or human factors related work
  • Strong background in user research, usability, or Human Factors
  • Expertise in user-centered design processes and research methodologies
  • Ability to create wireframes, user flows, interaction diagrams, and prototypes (low to high fidelity)
  • Ability to design intuitive, scalable user interfaces within technical and system constraints
  • Strong storytelling and presentation skills to communicate design solutions effectively
  • Experience in task analysis and user workflow design (current and future states)
  • Ability to synthesize research insights into actionable design strategies
Job Responsibility
Job Responsibility
  • Lead end-to-end UX activities including workshops, user research, personas, scenarios, information architecture, wireframes, navigation, and interactive design for complex applications and AI-driven experiences such as copilots and conversational interfaces
  • Plan and drive UX across the development lifecycle, ensuring delivery aligns with timelines, budget, usability, accessibility, and inclusive design best practices
  • Define UI specifications and secure stakeholder sign-off
  • Collaborate with cross-functional teams to translate AI capabilities (ML models, LLMs, automation) into intuitive experiences, designing prompt-based interactions, intelligent workflows, and AI-driven user journeys
  • Apply Human-AI interaction principles such as transparency, explainability, trust, and error handling, while accounting for edge cases, system limitations, and uncertainty
  • Ensure adherence to Responsible AI principles including fairness, privacy, and ethical design
  • Support pre-sales and scoping by evangelizing user-centered and AI-first design approaches, conducting research, and leveraging AI-powered design tools to accelerate outcomes
  • Fulltime
Read More
Arrow Right

AI Research Scientist, Robotics

The ideal Research Scientist candidate will use their skills in system design an...
Location
Location
United States , Redmond
Salary
Salary:
154000.00 - 217000.00 USD / Year
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • Currently has or is in the process of obtaining a PhD degree in the field of Artificial Intelligence, Robotics, Computer Vision, Machine Learning, Language, a related field, or equivalent practical experience
  • Experience with any of the following research areas: robotics, motion planning, embodied AI, human-robot interaction, sim-to-real transfer, learning from demonstration, reinforcement learning, dexterous manipulation, digital agents, vision language models, computer vision, egocentric perception, and/or LLMs
  • Experience in relevant robotics related research areas, such as: VLM, robot learning, reinforcement learning, imitation learning, action-conditioned world models, task and motion planning, sim-to-real transfer robotic control, manipulation, navigation, or generally embodied AI
Job Responsibility
Job Responsibility
  • Perform fundamental and applied research to push the scientific and technological frontiers of embodied artificial intelligence
  • Invent/improve novel data-driven paradigms for robotics, leveraging a variety of modalities (images, video, text, audio, tactile, etc)
  • Investigate paradigms that can deliver a spectrum of embodied behaviors - from simulated characters to real robots, and from short-horizon, low-level to long-horizon, high-level intelligence
  • Develop algorithms based on state-of-the-art machine learning and neural network methodologies
  • Define, build and benchmark new functionalities needed for the next generation of AI
  • Conduct research towards long-term product goals while identifying intermediate milestones
  • Plan and execute novel research based on long-term objectives of the organization
What we offer
What we offer
  • bonus
  • equity
  • benefits
Read More
Arrow Right

AI Research Scientist, Robotics

At Meta, we’re building the future of human connection and the technology that e...
Location
Location
United States , Redmond
Salary
Salary:
219000.00 - 301000.00 USD / Year
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • PhD degree in the field of Artificial Intelligence, Robotics, Computer Vision, Machine Learning, Language, a related field, or equivalent practical experience
  • Experience with any of the following research areas: robotics, motion planning, embodied AI, human-robot interaction, sim-to-real transfer, learning from demonstration, reinforcement learning, dexterous manipulation, digital agents, vision language models, computer vision, egocentric perception, and/or Large Language Models
  • 5+ years of industry experience in relevant robotics related research areas, such as: Vision Language Models robot learning, reinforcement learning, imitation learning, action-conditioned world models, task and motion planning, sim-to-real transfer robotic control, manipulation, navigation, or generally embodied AI
Job Responsibility
Job Responsibility
  • Perform fundamental and applied research to push the scientific and technological frontiers of embodied artificial intelligence
  • Invent/improve novel data-driven paradigms for robotics, leveraging a variety of modalities (images, video, text, audio, tactile, etc.)
  • Investigate paradigms that can deliver a spectrum of embodied behaviors - from simulated characters to real robots, and from short-horizon, low-level to long-horizon, high-level intelligence
  • Develop algorithms based on state-of-the-art machine learning and neural network methodologies
  • Define, build and benchmark new functionality needed for the next generation of AI
  • Conduct research towards long-term product goals while identifying intermediate milestones
  • Lead, plan, and execute novel research based on long-term objectives of the organization
What we offer
What we offer
  • bonus
  • equity
  • benefits
Read More
Arrow Right

AI Research Scientist, Robotics

Meta is seeking a Research Scientist to join Meta Superintelligence Labs. Indivi...
Location
Location
United States , Menlo Park
Salary
Salary:
184000.00 USD / Year
meta.com Logo
Meta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
  • PhD degree in the field of Artificial Intelligence, Robotics, Computer Vision, Machine Learning, Language, a related field, or equivalent practical experience
  • Experience with any of the following research areas: robotics, motion planning, embodied AI, human-robot interaction, sim-to-real transfer, learning from demonstration, reinforcement learning, dexterous manipulation, digital agents, vision language models, computer vision, egocentric perception, and/or LLMs
  • 2+ years of industry experience in relevant robotics related research areas, such as: robot learning, reinforcement learning, imitation learning, action-conditioned world models, task and motion planning, sim-to-real transfer robotic control, manipulation, navigation, or generally embodied AI
  • Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment
Job Responsibility
Job Responsibility
  • Perform fundamental and applied research to push the scientific and technological frontiers of embodied artificial intelligence
  • Invent/improve novel data-driven paradigms for robotics, leveraging a variety of modalities (images, video, text, audio, tactile, etc)
  • Investigate paradigms that can deliver a spectrum of embodied behaviors - from simulated characters to real robots, and from short-horizon, low-level to long-horizon, high-level intelligence
  • Develop algorithms based on state-of-the-art machine learning and neural network methodologies
  • Define, build and benchmark new capabilities needed for the next generation of AI
  • Conduct research towards long-term research goals while identifying intermediate milestones
  • Lead, plan, and execute novel research based on long-term objectives of the organization
What we offer
What we offer
  • bonus
  • equity
  • benefits
  • Fulltime
Read More
Arrow Right

Senior Applied Scientist and Principal Applied Scientist

Are you passionate about advancing Artificial Intelligence through innovative re...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (e.g., statistics predictive analytics, research)
  • OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience
  • OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 1+ year(s) related experience
  • OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Microsoft Cloud Background Check
  • Additional or Preferred Qualifications: Bachelor's Degree AND 6+ years related experience
  • OR Master's Degree AND 4+ years related experience
  • OR Doctorate AND 3+ years related experience
  • OR equivalent experience
Job Responsibility
Job Responsibility
  • Collaborate with AI researchers, engineers, and multidisciplinary teams within M365 Core to design, build, and deliver end-to-end machine learning solutions optimized for human-human and human-AI interactions across productivity and collaboration scenarios
  • Lead and contribute to research and development initiatives in language model training, evaluation, and quality assessment with a specific focus on advancing the foundational capabilities and enterprise solutions of M365 Core
  • Design, develop, and refine robust ML pipelines, including data cleaning, feature engineering, model training, evaluation, deployment, and optimization
  • Adapt, extend or enhance the capability of large language models (LLMs) using a combination of supervised finetuning, reinforcement learning, and apply advanced techniques such as chain of thought prompting, agent orchestration, and synthetic data generation to meet evolving business needs
  • Remain current with the latest advancements in ML, AI, digital signal processing, and responsible AI practices, contributing to intellectual property portfolios through publications, patents, and libraries
  • Develop and deploy live production systems as part of integrated product teams, supporting all phases of the product cycle from ideation to shipping, post-launch optimization, and ongoing improvements
  • Champion ethics, fairness, and privacy in AI system development, contributing to policy creation and ensuring responsible research and data collection practices
  • Document work in progress, experimentation results, and share findings to promote a culture of innovation and transparency
  • Mentor and develop early-in-career team members, establish best practices, and collaborate with academic communities to build a robust recruiting pipeline for research talent
  • Present research findings at conferences and industry events, shaping the dialogue around AI-powered solutions for enterprise and communication platforms
  • Fulltime
Read More
Arrow Right

Senior Applied Scientist

Microsoft is a company where innovators come to collaborate, envision what can b...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (e.g., statistics predictive analytics, research)
  • OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience (e.g., statistics, predictive analytics, research)
  • OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 1+ year(s) related experience (e.g., statistics, predictive analytics, research)
  • OR equivalent experience
  • 4+ years of experience in statistics, predictive analytics and research
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Collaborate with AI researchers and audio signal processing experts to design and build end-to-end ML systems, tuned for human-human and human-AI interactions
  • Design and develop ML pipelines involving data cleaning, feature engineering, model training, and evaluation
  • Work across the product lifecycle from prototyping to shipping production-grade code optimized for performance and memory and updating the deployed models based on A/B testing
  • Remain up to date with the latest advancements, trends and research and contribute towards our IP portfolio
  • Research and develop synthetic data generation strategies
  • Proactively follow state of the art research and share latest work, write papers, attend conferences and share knowledge in the wider team
  • Fulltime
Read More
Arrow Right

Senior Applied Scientist

We are developing the Intelligent Conversation and Communications Cloud (IC3) to...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (e.g., statistics predictive analytics, research)
  • OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience (e.g., statistics, predictive analytics, research)
  • OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 1+ year(s) related experience (e.g., statistics, predictive analytics, research)
  • OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
Job Responsibility
Job Responsibility
  • Collaborating with AI researchers and audio signal processing experts to design and build end-to-end ML systems, tuned for human-human and human-AI interactions
  • Design and develop ML pipelines involving data cleaning, feature engineering, model training, and evaluation
  • Work across the product lifecycle from prototyping to shipping production-grade code optimized for performance and memory and updating the deployed models based on A/B testing
  • Remain up to date with latest advancements, trends and research and contribute towards our IP portfolio
  • Fulltime
Read More
Arrow Right

Lead Product Designer – AI

The Lead Product Designer will focus on designing AI-enabled products, collabora...
Location
Location
United Kingdom , London
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years experience in product design
  • Recent experience working on AI enabled products or services
  • Strong experience working in agile, cross-functional squads
  • Demonstrated ability to design for machine learning, generative AI or intelligent automation
  • Experience designing conversational, adaptive or predictive interfaces
  • Strong UX and interaction design fundamentals
  • Ability to prototype complex interactions and flows using tools such as Figma
  • Experience collaborating closely with Engineers and Data Scientists
  • Solid understanding of AI capabilities and limitations, including model uncertainty and edge cases
  • Experience using AI tools within your own design workflow
Job Responsibility
Job Responsibility
  • Lead the design of AI enabled products and features from discovery to delivery
  • Shape user experiences that incorporate machine learning, generative AI or automation
  • Translate complex technical capabilities into simple, human centred interactions
  • Define the level of automation for a product or its functions to determine the most optimal level of human in the loop / over the loop
  • Use AI tools within your own workflow for research synthesis, ideation, prototyping and iteration
  • Collaborate closely with Data Science and Engineering to define feasible, responsible solutions
  • Prototype conversational, predictive and adaptive experiences
  • Define interaction patterns for explainability, feedback and human oversight
  • Facilitate co-creation workshops with clients to identify high value AI opportunities
  • Own the end-to-end user experience of AI driven features or products
What we offer
What we offer
  • Tailored benefits that support your physical, emotional, and financial wellbeing
  • Continuous growth and development opportunities
  • Flexible work options
Read More
Arrow Right