CrawlJobs Logo

AI Applied Scientist - PhD Intern, Evaluation Systems and Metrics

United States Employment contract 104000.00 - 166000.00 USD / Year · Job Posted May 26, 2026
Apply Position
Job Link Share

Job Description

We are seeking remote PhD interns for Summer 2026! As an intern, you will help develop cutting-edge evaluation methodologies for AI systems. Your research will focus on creating robust, scalable metrics and frameworks to assess the quality, consistency, and performance of generative models across multiple modalities. You may contribute in one or more of the following areas: Novel Evaluation Metrics: Develop innovative assessment methodologies for emerging AI capabilities, focusing on consistency and quality across complex multi-modal outputs; Self-Improving Assessment: Design evaluation systems that learn and adapt from feedback, automatically discovering new evaluation criteria and improving assessment quality over time; Privacy-Preserving Evaluation: Design frameworks that incorporate domain-specific implementations of differential privacy to protect sensitive user information while maintaining utility for model training and assessment; Ethical Fair Housing Evaluation: Develop scalable methodologies for assessing agentic systems, ensuring compliance with fair housing standards and promoting ethical, responsible AI deployment. This role has been categorized as a Remote position. 'Remote' employees do not have a permanent corporate office workplace and, instead, work from a physical location of their choice, which must be identified to the Company. U.S. employees may live in any of the 50 United States, with limited exceptions.

Job Responsibility

  • Help develop cutting-edge evaluation methodologies for AI systems
  • Research will focus on creating robust, scalable metrics and frameworks to assess the quality, consistency, and performance of generative models across multiple modalities
  • May contribute in one or more of the following areas: Novel Evaluation Metrics
  • Self-Improving Assessment
  • Privacy-Preserving Evaluation
  • Ethical Fair Housing Evaluation

Requirements

  • Currently enrolled as a PhD student in computer science, machine learning, computer vision, or a related field, with strong publication record
  • Background in one or more of the following areas: Evaluation methodologies for AI/ML systems
  • Computer vision metrics and 3D consistency assessment
  • Generative model evaluation (text, image, video, 3D)
  • Multi-modal assessment and automated feedback systems
  • Knowledge of data privacy methods (e.g., differential privacy, federated learning, secure ML) and their application
  • Single agent or multi-agent system evaluations
  • Familiarity with modern deep learning frameworks (e.g., PyTorch, Hugging Face Transformers)
  • Strong research mindset, with motivation to publish
  • Interest in applying AI to complex, multi-stakeholder domains
  • A record of publication in conferences, workshops, or journals is a plus

Nice to have

A record of publication in conferences, workshops, or journals is a plus

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

AI Applied Scientist - PhD Intern, Evaluation Systems and Metrics

8 matching positions

AI Applied Scientist - PhD Intern, Foundational IQ

As a PhD Research Intern on the Foundational IQ team, you will help train and ad...
Location
Location
United States
Salary
Salary:
104000.00 - 166000.00 USD / Year
Zillow
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Currently enrolled in a PhD program in Computer Science, Machine Learning, Artificial Intelligence or a related field with a strong research track record
  • Experience in one or more of the following: LLMs: instruction tuning/fine-tuning, prompting, and evaluation/measurement
  • Multimodal learning (image + text
  • familiarity with audio or geospatial a plus)
  • Representation learning with limited labels (self/semi/weakly-supervised)
  • User modeling for personalization systems
  • Reinforcement learning or sequential decision-making
  • Evaluating generative/agentic systems
  • privacy-aware and responsible AI practices (e.g., fair-housing considerations) are a plus
  • Proficiency in Python and modern ML frameworks (e.g., PyTorch, TensorFlow, Hugging Face)
Job Responsibility
Job Responsibility
  • Research and develop methods for adapting LLMs and foundation models with Zillow’s domain-specific data
  • Build and evaluate multimodal models that combine text, images, geospatial and tabular signals for home and user understanding
  • Explore reinforcement learning and sequential decision-making for long-horizon, user-centric outcomes
  • Prototype agentic workflows
  • define success metrics and run rigorous offline/online evaluations
  • Partner across science, engineering, product, and design
  • share results via docs, presentations, and publications
  • Fulltime
Read More
Arrow Right

AI Applied Scientist - PhD Intern, 3D Computer Vision

We are seeking remote PhD interns for Summer 2026! As a PhD Research Intern, you...
Location
Location
Germany , Munich
Salary
Salary:
Not provided
Zillow
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Currently enrolled in a PhD program in Computer Science, Machine Learning, Artificial Intelligence or a related field with a strong research track record
  • Experience in sparse-view reconstruction, camera localization and scene understanding
  • 3D Representations (Point Clouds, NeRFs, 3DGS)
  • Representation learning with limited labels (self/semi/weakly-supervised)
  • 2D, 3D or video Diffusion Models
  • Evaluating generative/agentic systems
  • Privacy‑aware and responsible AI practices (e.g., fair‑housing considerations) are a plus
  • Proficiency in Python and modern ML frameworks (e.g., PyTorch, TensorFlow, Hugging Face)
  • Clear communication and a collaborative mindset
  • Motivated to publish at top venues
Job Responsibility
Job Responsibility
  • Research and develop methods for adapting 2D and 3D foundation models with Zillow’s domain-specific data
  • Build and evaluate 3D computer vision and generative models
  • Prototype generative and ML workflows
  • Define metrics and run rigorous evaluations
  • Partner across science, engineering, product, and design
  • Share results via docs, presentations, and publications
What we offer
What we offer
  • 12-week paid internship
  • Flexible work arrangements (remote, hybrid, or in-person)
  • Direct supervision from experienced Applied Scientists
  • Access to Zillow's unique large-scale indoor datasets and computer resources
  • Opportunity to contribute to research projects that could lead to publications
  • Integration into Zillow's vibrant research community: Biweekly Applied Science Guild meetings, Weekly demo sessions, Research presentations and collaborations
  • Fulltime
Read More
Arrow Right

Applied Scientist II, GenAI Evaluation Media (GEM)

The North America Stores GenAI Evaluation Media (GEM) team is seeking an Applied...
Location
Location
United States , Sunnyvale; Seattle
Salary
Salary:
142800.00 - 222200.00 USD / Year
Amazon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
  • Experience in building models for business application
  • Experience programming in Java, C++, Python or related language
Job Responsibility
Job Responsibility
  • Develop core science primitives for vision and language understanding, visual content generation and editing, virtual try-on, and automated quality assurance via state-of-the-art computer vision, machine learning, and generative AI
  • Design and implement visual agentic systems, balancing visual quality, relevance, latency, and cost
  • Define metrics and success criteria for scientific initiatives, ensuring rigorous validation across customer touch points
  • Own end-to-end delivery of research initiatives from problem formulation through experimentation to production deployment
  • Stay current with latest advances in AI/ML and identify opportunities to apply them to problem space
  • Drive development and deployment of scalable agentic systems for visual content understanding and generation
  • Maintain high scientific and engineering standards in work
  • Tackle complex technical problems while maintaining practical focus on customer value
  • Contribute to team's culture of scientific excellence through presentations and publications at internal and external science forums
  • Partner with product and engineering teams to deliver customer-facing features
What we offer
What we offer
  • Health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
  • 401(k) matching
  • Paid time off
  • Parental leave
  • Sign-on payments
  • Restricted stock units (RSUs)
  • Fulltime
Read More
Arrow Right

Applied Scientist II, Translation Services

Have you ever wondered how Amazon launches and maintains a consistent customer e...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
Amazon
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of building models for business application experience
  • PhD, or Master's degree and 3+ years of CS, CE, ML or related field experience
  • Experience in patents or publications at top-tier peer-reviewed conferences or journals
  • Experience programming in Java, C++, Python or related language
  • Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing
Job Responsibility
Job Responsibility
  • Apply your expertise in LLM models to design, develop, and implement scalable machine learning solutions that address complex language translation-related challenges in the eCommerce space
  • Collaborate with cross-functional teams, including software engineers, data scientists, and product managers, to define project requirements, establish success metrics, and deliver high-quality solutions
  • Conduct thorough data analysis to gain insights, identify patterns, and drive actionable recommendations that enhance seller performance and customer experiences across various international marketplaces
  • Continuously explore and evaluate state-of-the-art modeling techniques and methodologies to improve the accuracy and efficiency of language translation-related systems
  • Communicate complex technical concepts effectively to both technical and non-technical stakeholders, providing clear explanations and guidance on proposed solutions and their potential impact
  • Fulltime
Read More
Arrow Right

Applied AI/ML Scientist

As an Applied AI Scientist in the FieldML team, you will be responsible for deve...
Location
Location
United Arab Emirates
Salary
Salary:
Not provided
cerebras.net Logo
Cerebras Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master’s or PhD in Computer Science, Machine Learning, or related fields
  • Expert-level understanding of modern model architectures, including dense transformers, MoEs, multimodal and sequence models, scaling laws and training dynamics
  • Proven track record of training and/or fine-tuning large models (1B+ parameters) and direct experience with the challenges of large-scale model training
  • Mastery of Python and PyTorch, experience with distributed training frameworks and large-scale distributed data processing pipelines and tools
  • Strong interpersonal and communication skills
  • Effective in collaborative and fast-paced team settings, able to work autonomously and within a team in a dynamic environment, managing multiple projects and pivoting as customer needs evolve
Job Responsibility
Job Responsibility
  • Customer Use Case Discovery & Project Scoping
  • Collaborate with customer stakeholders to identify the best approaches to their business problem with AI
  • Contribute to the technical scoping of engagements, including feasibility analysis, data quality/availability/readiness assessments, and the selection of optimal model architectures
  • Define project milestones, success metrics, and rigorous evaluation benchmarks
  • Custom SOTA Models and AI Systems Development
  • Architect and execute end-to-end training recipes for custom models, tailoring model architecture and training recipes to meet customer-specific performance and accuracy requirements
  • Design and implement sophisticated adaptation strategies, including continuous pre-training on private datasets, supervised fine-tuning (SFT), and post-training alignment via RLHF or DPO
  • Take full ownership of the training pipeline, from high-performance data preprocessing and tokenization to hyperparameter tuning and loss-curve analysis
  • Navigate the nuances of model convergence on specialized hardware
  • Scale training workloads across Cerebras clusters
What we offer
What we offer
  • Build a breakthrough AI platform beyond the constraints of the GPU
  • Publish and open source their cutting-edge AI research
  • Work on one of the fastest AI supercomputers in the world
  • Enjoy job stability with startup vitality
  • Our simple, non-corporate work culture that respects individual beliefs
Read More
Arrow Right

District Manager to Travel Management Consultant

This role requires 100% domestic travel year-round, with consultants typically f...
Location
Location
United States , Dallas
Salary
Salary:
Not provided
dbaresults.com Logo
DeWolff, Boberg & Associates
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Business, Management, Engineering, or a related field preferred
  • Minimum of 4+ years of direct leadership, supervision, or management experience, preferably within production or manufacturing environments
  • Proven ability to manage conflict, build consensus, and facilitate collaboration across cross-functional teams
  • Strong problem-solving skills with the ability to balance results delivery and client relationship management
  • Demonstrated ability to build credibility and influence at all levels of an organization, including executive leadership and external stakeholders
  • Strong analytical, observational, and numerical reasoning skills, paired with sound business judgment
  • Ability to thrive in fast-paced, high-pressure, and constantly evolving environments
  • Excellent communication skills, including verbal, written, and presentation capabilities
  • Team-oriented mindset with openness to providing and receiving constructive feedback daily
  • Advanced proficiency in Microsoft Office Suite
Job Responsibility
Job Responsibility
  • Executing client goals, objectives, and operational processes through frontline coaching and support
  • Working side-by-side with frontline teams to drive behavioral and management changes that improve performance
  • Evaluating operational processes and resource utilization to identify opportunities for efficiency and performance improvements
  • Building strong client relationships and maintaining effective communication across all levels of the organization
  • Addressing operational challenges directly, providing constructive feedback, and driving accountability
  • Coaching frontline supervisors to become more proactive, effective leaders
  • Increasing employee engagement and facilitating workshops that encourage collaboration and continuous improvement
  • Delivering meaningful metrics and performance data to frontline leadership, middle management, and executives
  • Identifying opportunities for process improvement and helping teams implement practical solutions
  • Implementing proven management systems that create long-term operational success
What we offer
What we offer
  • Medical, Dental, and Vision Insurance
  • Short-Term and Long-Term Disability Coverage
  • Flexible Spending Account (FSA)
  • 401(k) Retirement Plan
  • Two weeks of paid vacation
  • One week of paid PTO
  • Paid year-end holiday closure
  • Career Growth & Advancement
  • Three-tier Consultant Career Track and a Project Manager Career Track
  • Opportunity to advance from Senior Consultant to Project Manager
Read More
Arrow Right

Assistant Manager - London Flex Team

As a Flexi Assistant Manager you’ll be supporting your store team with driving s...
Location
Location
United Kingdom , London
Salary
Salary:
31768.31 GBP / Year
majestic.co.uk Logo
Majestic Wine Warehouses Ltd.
Expiration Date
June 12, 2026
Flip Icon
Requirements
Requirements
  • Excellent time-management, delegation and problem-solving skills
  • Be able to demonstrate your ability to deliver exceptional customer experience & service to every single customer
  • Self-motivated, able to thrive when working alone and as part of a team
  • A can-do attitude with a passion for seeing problems through to solutions
  • Adaptable and resilient to meet the ever-changing demands of our business
  • Excellent communication and time management skills
  • Wine knowledge is beneficial but passion to learn more is essential to pass level 2 WSET wine qualifications
  • Hold a full UK, manual transmission driving licence for at least 12 months with no more than 6 penalty points
Job Responsibility
Job Responsibility
  • Drive store performance by maximising sales opportunities
  • Support your team on meeting and exceeding targets through focusing on KPI delivery
  • Ability and willingness to cover and support different stores at short notice
  • Deliver exceptional market leading customer service to drive business growth through customer loyalty & repeat purchases
  • Offer customers a VIP concierge service, actively contacting them with updates on products and tastings
  • Sell the story not the Discount
  • Demonstrate and share your passion for product with customers through an in-depth knowledge of our range
  • Take ownership for your wine knowledge, constantly learning about our products to support your WSET qualification and confidence in selling
  • Take accountability and pride for the physical appearance and maintenance of your store both internally & externally
  • Involvement in all operational tasks required for the day-day operational running a Majestic Wine store - from delivering wine to our customers, merchandising stock deliveries and calling our valued customers to drive sales opportunities
What we offer
What we offer
  • Competitive Salary & Performance Bonus
  • Up to 20% staff discount
  • Career development opportunities
  • Full training provided for your first 3 months with us
  • Uniform provided
  • Fantastic incentives that take you around the world to explore our different vineyards
  • A contributory Company Pension Plan
  • Life Assurance (Worth 2 times your annual salary)
  • 29 days holiday, including public and bank holidays
  • Access to Retail Trust which includes: Retail Rewards including Instant savings with discounted e-vouchers, discounted reloadable shopping cards, gift vouchers and gift cards, Discounts of up to 30%, Access to free counselling and support phone line
  • Fulltime
!
Read More
Arrow Right

Pharmacist

Our pharmacists make a real difference in the communities we look after, deliver...
Location
Location
United Kingdom , Worcester Park
Salary
Salary:
Not provided
boots.com Logo
Boots
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Registered with the relevant pharmacy regulator (GPhC, PSNI, PSI)
  • Strong communication and relationship-building skills
  • Experience leading patient and customer care within a pharmacy setting
  • Passion for delivering essential, advanced, and private services
  • A collaborative, team-first mindset and an eagerness to coach and guide others
Job Responsibility
Job Responsibility
  • Delivering NHS, locally commissioned, and private services using both in-store and digital tools
  • Leading professional and legal standards for patient safety and pharmacy compliance
  • Monitoring, evaluating, and continually improving standards of care and safety
  • Working with the Store Manager to develop the capability of the wider healthcare team
  • Growing talent that reflects the communities we serve
  • coaching, mentoring and supporting your colleagues every step of the way
  • Representing Boots within the local community and with healthcare professionals
What we offer
What we offer
  • Boots Retirement Savings Plan
  • Generous employee discount across Boots and partner brands
  • Discretionary annual bonus
  • Enhanced maternity/paternity/adoption leave pay, and a gift card for those expecting or adopting
  • Flexible benefits scheme
  • holiday buying, gym discounts, life assurance and more
  • 24/7 counselling and wellbeing support through TELUS Health, our Employee Assistance Programme
  • CPD Days and protected learning time
  • GPhC/PSNI/PSI Fees reimbursed
  • Additional option to buy benefits, including the option to buy up to five extra holidays
  • Fulltime
Read More
Arrow Right