Research Scientist, AI Controls and Monitoring Job at Scale (San Francisco)

Senior Research Engineer

As a Senior Research Engineer at Microsoft, you will advance Microsoft’s mission...

Location

United States , Redmond

Salary:

119800.00 - 234700.00 USD / Year

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, Physics, or a related field and 4 or more years in applied ML or AI research and product engineering
OR Master’s degree and 3 or more years in applied ML or AI research and product engineering
OR PhD in a relevant field and 2 or more years with generative AI, LLMs, or related ML algorithms
Ability to meet Microsoft, customer and/or government security screening requirements
Microsoft Cloud Background Check upon hire/transfer and every two years thereafter

Job Responsibility

Bringing State-of-the-Art Research to Products
Design and implement AI systems using foundation models, prompt engineering, retrieval-augmented generation, multi-agent architectures, and classic ML
Fine-tune large language models on domain-specific data and evaluate via offline and online methods such as A/B testing, telemetry, and shadow deployments
Build and harden prototypes into production-ready services using robust software engineering and MLOps practices
Drive original research and thought leadership (whitepapers, internal notes, patents)
convert insights into shipped capabilities
Research Translation: Continuously review emerging work
identify high-potential methods and adapt them to Microsoft problem spaces
End-to-End System Development
ML Design & Architecture: Own end-to-end pipeline from data prep, training, evaluation, deployment, and feedback loops

Fulltime

Agentic AI Developer

The Applications Development Senior Programmer Analyst is an intermediate level ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8+ years of relevant experience
5+ years of professional experience in software development with a focus on AI, machine learning, or agent-based systems. Experience in finance industry a plus.
Strong proficiency in Python, SQL
Java is a plus.
Solid understanding of core AI concepts, including knowledge representation, automated planning, decision-making under uncertainty, and multi-agent systems.
Experience with machine learning frameworks (e.g., TensorFlow, PyTorch) and relevant libraries (e.g., scikit-learn, numpy, pandas).
Familiarity with large language models (LLMs) and their application in agentic systems.
Familiarity with specific agent frameworks (e.g., LangChain, AutoGen, CrewAI, RAG) or research in multi-agent reinforcement learning.
Experience in designing and implementing APIs for AI services.
Experience with software development best practices, including version control (Git), CI/CD pipelines, testing, and code reviews.

Job Responsibility

Agent Design and Development: Design and implement intelligent agents, including their perception, reasoning, planning, and action execution modules.
System Architecture: Develop scalable and robust architectures for agentic systems, ensuring high performance, reliability, and security.
Machine Learning Integration: Integrate various machine learning models (e.g., LLMs, reinforcement learning, predictive models) to enhance agent capabilities and decision-making.
Task Automation: Develop agents that can automate complex tasks, optimize workflows, and solve real-world problems across various domains.
Framework and Tooling: Utilize and contribute to agentic AI frameworks and development tools.
Evaluation and Optimization: Design and implement metrics and evaluation strategies for agent performance, continuously optimizing and improving agent behavior.
Research and Innovation: Stay abreast of the latest advancements in AI, particularly in agent-based systems, autonomous AI, and related fields, and propose innovative solutions.
Collaboration: Work closely with cross-functional teams including AI researchers, data scientists, product managers, and software engineers to integrate agentic solutions into broader products and services.
Documentation: Create comprehensive technical documentation for agent designs, implementations, and operational procedures.
Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users

Fulltime

Data Analyst / ML Engineer

ML Solutions team is seeking a highly skilled and experienced Data Analytics/Mac...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

5-8 years of relevant experience
Strong hands-on experience in Machine Learning, delivering complex solutions to production
Experience with Generative AI technologies essential
Knowledge of NLP, Name Entity Recognition
In-depth knowledge of deep learning and Generative AI frameworks such as, Langchain, Lang Graph, Crew AI or similar
Experience with and other open-source frameworks/ libraries/ APIs like Hugging Face Transformers, Spacy, Pandas, scikit-learn, NumPy
Experience in using Machine Learning/Deep Learning: XGBoost, LightGBM, TensorFlow, PyTorch
Proficiency in Python Software Development, following Object-Oriented design patterns and best practices
Strong background in mathematics: linear algebra, probability, statistics, and optimization
Experience with evaluation, scoring with framework like ML Flow, Dagster, etc.

Job Responsibility

Hands-On Execution and Delivery: Actively contribute to the development and delivery of AI solutions, driving innovation and excellence within the team. Take a hands-on approach to ensure AI models are successfully deployed into production environments, meeting high-quality standards and performance benchmarks
Quality Control: Ensure the quality and performance of generative AI models, conducting rigorous testing and evaluation
Research and Development: Participate in research activities to explore and advance state-of-the-art generative AI techniques. Stay actively engaged in monitoring ongoing research efforts, keeping abreast of emerging trends, and ensuring that the Generative AI team remains at the forefront of the field
Cross-Functional Collaboration: Collaborate effectively with various teams, including product managers, engineers, and data scientists, to integrate AI technologies into products and services

Fulltime

Data Scientist/Machine Learning Engineer

The ML Solutions team is seeking a Data Scientist/Machine Learning Engineer to d...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

8 to 13 years of Strong hands-on experience in Machine Learning, delivering complex solutions to production
Experience with Generative AI technologies essential
Understanding of concepts like supervised, unsupervised, clustering, embedding
Knowledge of NLP, Name Entity Recognition, Computer Vision, Transformers, Large Language Models
In-depth knowledge of deep learning and Generative AI frameworks such as, Langchain, Lang Graph, Crew AI or similar
Experience with and other open-source frameworks/ libraries/ APIs like Hugging Face Transformers, Spacy, Pandas, scikit-learn, NumPy, OpenCV
Experience in using Machine Learning/Deep Learning: XGBoost, LightGBM, TensorFlow, PyTorch, Keras
Proficiency in Python Software Development, following Object-Oriented design patterns and best practices
Strong background in mathematics: linear algebra, probability, statistics, and optimization
Experience with evaluation, scoring with framework like ML Flow

Job Responsibility

Hands-On Execution and Delivery: Actively contribute to the development and delivery of AI solutions, driving innovation and excellence within the team. Take a hands-on approach to ensure AI models are successfully deployed into production environments, meeting high-quality standards and performance benchmarks
Mentoring Young Talents: Mentoring team, guiding data analysts/ML engineers from concept to production. This involves fostering technical growth, providing project oversight, and ensuring adherence to best practices, ultimately building a high-performing and innovative team
Quality Control: Ensure the quality and performance of generative AI models, conducting rigorous testing and evaluation
Research and Development: Participate in research activities to explore and advance state-of-the-art generative AI techniques. Stay actively engaged in monitoring ongoing research efforts, keeping abreast of emerging trends, and ensuring that the Generative AI team remains at the forefront of the field
Cross-Functional Collaboration: Collaborate effectively with various teams, including product managers, engineers, and data scientists, to integrate AI technologies into products and services

Fulltime

ML Ops Engineer

The MLOps Engineer will work closely with the Data Science, Analytics, and Data ...

Location

United States

Salary:

127000.00 - 160550.00 USD / Year

Zelis

Expiration Date

Until further notice

Requirements

2–5 years of experience in ML Ops, ML Engineering, or a related role with a focus on production-level model monitoring, automation, and deployment
Strong experience with ML observability tools or custom-built monitoring systems
Experience with monitoring LLMs and Generative AI models, including prompt evaluation, hallucination tracking, and agent behavior auditing
Experience in deploying and managing ML workloads using containerization and orchestration platforms such as Docker, Kubernetes, Kubeflow, or TensorFlow Extended
Familiarity with AutoML pipelines and workflow management tools (e.g., MLflow, SageMaker Autopilot)
Experience working in cloud environments, preferably AWS (e.g., SageMaker, S3, Lambda, ECS/EKS)
Understanding of ML lifecycle tools (e.g., MLflow, SageMaker Pipelines) and CI/CD practices
Strong security and compliance awareness, particularly related to model/data governance (e.g., HIPAA, GDPR)
Proficiency in Python and key data libraries (Pandas, Numpy, Matplotlib, etc.)
Advanced SQL skills and experience with Snowflake or similar data warehousing platforms

Job Responsibility

Build and maintain monitoring infrastructure for conventional machine learning models, with capabilities for performance tracking, drift detection, and alerting
Research, evaluate, and implement monitoring strategies and tools for Generative AI systems, including LLMs and Agentic AI architectures
Collaborate with ML Engineers, Data Scientists, and DevOps teams to deploy, manage, and monitor models in production
Develop and support scalable, secure, and automated data pipelines using Snowflake, SQL, and Python for training, serving, and monitoring ML and GenAI models
Leverage AutoML tools and frameworks (e.g., MLflow, Kubeflow, SageMaker Autopilot) to streamline experimentation and deployment
Design dashboards and reporting systems to visualize model health metrics and surface key operational insights
Ensure auditability, reproducibility, and compliance for model performance and data flow in production environments, with consideration for regulatory standards like GDPR and HIPAA
Maintain CI/CD workflows and version-controlled codebases (e.g., Git) for ML infrastructure and pipelines
Utilize containerization and orchestration technologies (e.g., Docker) to manage scalable ML infrastructure
Leverage tools such as Streamlit and Python visualization libraries to present insights from model and data monitoring

What we offer

401k plan with employer match
flexible paid time off
holidays
parental leaves
life and disability insurance
health benefits including medical, dental, vision, and prescription drug coverage

Fulltime

Senior Software Engineer

As a Senior Research Engineer at Microsoft, you will advance Microsoft’s mission...

Location

India , Hyderabad

Salary:

Not provided

Microsoft Corporation

Expiration Date

Until further notice

Requirements

Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, Physics, or a related field and 4 or more years in applied ML or AI research and product engineering
Master’s degree and 3 or more years in applied ML or AI research and product engineering
PhD in a relevant field and 2 or more years with generative AI, LLMs, or related ML algorithms
Proficiency in Python and at least one deep learning framework such as PyTorch, JAX, or TensorFlow
Experience deploying Fine Tuned LLMs or multimodal models in live production environments
Experience shipping and maintaining production AI systems
Ability to meet Microsoft, customer, and government security screening requirements
Microsoft Cloud Background Check upon hire or transfer and every two years thereafter

Job Responsibility

Bringing State-of-the-Art Research to Products
Design and implement AI systems using foundation models, prompt engineering, retrieval-augmented generation, multi-agent architectures, and classic ML
Fine-tune large language models on domain-specific data and evaluate via offline and online methods such as A/B testing, telemetry, and shadow deployments
Build and harden prototypes into production-ready services using robust software engineering and MLOps practices
Drive original research and thought leadership (whitepapers, internal notes, patents)
convert insights into shipped capabilities
Research Translation: Continuously review emerging work
identify high-potential methods and adapt them to Microsoft problem spaces
End-to-End System Development
ML Design & Architecture: Own end-to-end pipeline from data prep, training, evaluation, deployment, and feedback loops

Fulltime

Vp/lead Vp, Mrm Ai Coe

Model Risk Management (MRM) at HSBC is structured as a global function, headed u...

Location

Poland , Krakow

Salary:

23300.00 - 30900.00 PLN / Month

HSBC

Expiration Date

June 08, 2026

Requirements

Master’s or PhD degree in a quantitative discipline like Science, Engineering, Mathematics, Statistics, Quantitative Finance, or Engineering
Data Scientist with relevant experience leading the building or validating AI products
Deep understanding of AI models, algorithms and the associated mathematics
Experience with Python, including the main libraries used for data science and AI (e.g. PyTorch, TensorFlow, scikit-learn)
Knowledge of the risks associated with developing, deploying and using AI in large commercial organizations
Knowledge of the regulatory landscape for AI and ability to access the impact of proposed changes in these regulatory rules to the bank
Knowledge of AI research, methodologies and techniques, particularly General Purpose AI, Deep Neural Networks, Agentic AI frameworks and statistical analysis (Variable Reduction, Feature engineering) with Supervised and Unsupervised machine learning algorithms
Expertise with data cleaning, feature engineering, and data normalization techniques
Active interest in the latest tools and ways of working linked to AI, with the ability to apply this knowledge in a flexible way adapting to varied circumstances across HSBC
Ability to promote a strong risk control culture and continually improve risk awareness

Job Responsibility

Lead the local MRM AI CoE team in Krakow based on direction from the Global Head, MRM AI CoE, coordinate with the Head, MRM AI CoE in India
Monitor academic, industry and internal HSBC developments in AI tools, techniques and ways of working to support a forward-looking agenda for emerging risks and challenges that ERM and the Bank may face, and ensure MRM’s approach to AI remains effective, efficient and up to date
Undertake AI related model validation activities as dictated by the Global Model Risk Policy including the assessment of
model inputs, calculations, reporting outputs, conceptual soundness of the underlying theory and the suitability of the use for its intended purpose, relevance and completeness of data, qualitative information and judgements, documentation, and implementation of the model
Identify opportunities to use AI tools and techniques to improve internal MRM processes
Adapt or refine existing AI tools to enable their use for internal MRM processes
Provide written reports detailing the results of validations highlighting issues identified during the validation
Validate remediation activities completed by the ILOD to ensure appropriate resolution of identified issues
Work with relevant stakeholders to support the embedding of new Global Model Risk Policies and Procedures
Provide model users, model owners, senior management, audit, and regulators (across 1LOD, 2LOD, 3LOD) with confidence that the models and tools developed, maintained, and used within the Group are compliant with internal and regulatory expectations and fit for the intended purpose

What we offer

Additional car allowance in the amount of 4,786 PLN (monthly, gross)
Additional bonuses for recognition awards
Multisport card
Private medical care
Life insurance
One-time reimbursement of home office set-up (up to 800 PLN)
Cafeteria platform
Employee assistance program
Additional contributions to PPK scheme
Corporate parties & events

Fulltime

Senior Associate, Data Scientist - US Card (Applied GenAI)

Senior Associate, Data Scientist - US Card (Applied GenAI). Data is at the cente...

Location

United States , McLean, Virginia; New York, New York

Salary:

135600.00 - 168900.00 USD / Year

Capital One

Expiration Date

Until further notice

Requirements

Currently has, or is in the process of obtaining one of the following with an expectation that the required degree will be obtained on or before the scheduled start date: A Bachelor's Degree in a quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Computer Science, or a related quantitative field) plus 2 years of experience performing data analytics
A Master's Degree in a quantitative field (Statistics, Economics, Operations Research, Analytics, Mathematics, Computer Science, or a related quantitative field) or an MBA with a quantitative concentration

Job Responsibility

Apply expertise in unstructured data (text, image) to harness the power of open source large language models (LLMs) and visual language models (VLMs)
Leverage a broad stack of technologies — LangGraph, LlamaIndex, Weights and Biases Weave, Hugging Face, PyTorch, AWS, and more — to automate workflows using huge volumes of text and vision data
Build machine learning and NLP models through all phases of development, from design through training, evaluation, and validation
partnering with engineering teams to operationalize them in scalable and resilient production systems that serve 80+ million customers
Assessing GenAI or LLM-Powered application architectures in production, including best practices for Generative AI development and deployments
Define requirements for AI observability, focusing on the traceability of autonomous decisions and comprehensive system audit trails
Evaluate the dynamic behavior of AI systems and oversee the development of key continuous monitoring controls and testing, ensuring that non-deterministic outputs and autonomous actions remain within risk appetite
Get into the weeds of internal business processes and data operations by guiding annotators to curate high quality, consistent datasets for model training, evaluation, and ongoing AI monitoring
Collaborate on a team of data scientists through all phases of project development, from design through training, evaluation, validation, implementation, and maintenance
Interact with a variety of internal stakeholders to ensure the alignment of data science solutions with business outcomes

What we offer

Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
comprehensive, competitive, and inclusive set of health, financial and other benefits

Fulltime

Select Country

Research Scientist, AI Controls and Monitoring

Job Description

Job Responsibility

Requirements

Nice to have

What we offer

Looking for more opportunities?