CrawlJobs Logo
Stanford Health Care Logo Stanford Health Care · IT - Software Development

Lead Data Engineer - AI/ML

United States of America, Palo Alto 94.35 - 125.03 USD / Hour · Job Posted March 04, 2026
Apply Position
Job Link Share

Job Description

The Lead Data Engineer will be part of a team building Stanford Health Care's (SHC) solutions incorporating Artificial Intelligence including providing health care solutions in the areas of patient care, medical research and administrative services. This group is designed to bring Artificial Intelligence (AI) and other emerging machine learning (ML) based innovations in data science into healthcare and will partner closely with individuals across clinical specialties and operations areas to deploy algorithms that can lead to better patient outcomes. Reporting to the Data Science Director and working closely with Stanford Medicine's inaugural Chief Data Scientist, this role will be responsible for building, scaling and maintaining the compute frameworks, analysis tooling, model implementations and agentic solutions that form our core AI platform.

Job Responsibility

  • Build end-to-end data pipelines and infrastructure for ML models used by the Data Science team and others at SHC
  • Understand the requirements of data processing and analysis pipelines and make appropriate technical design and interface decisions
  • Understand data flows among the SHC applications and use this knowledge to make recommendations and design decisions for languages, tools, and platforms used in software and data projects
  • Troubleshoot and debug environment and infrastructure problems found in production and non-production environments for projects by the Data Science Team
  • Work with other groups at SHC and the Technology and Digital Solutions (TDS) group to ensure servers and system maintenance based on updates, system requirements, data usage, and security requirements.

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related, or equivalent working experience
  • 5+ years experience in building data infrastructure for analytics teams, including ability to write code in SQL, R, or Python for processing large datasets in distributed cloud environments
  • Experience with cloud deployment strategies and CI/CD
  • Experience building and working with data infrastructure in a SaaS environment
  • Experience overseeing, developing or implementing machine learning operations (MLOps) processes
  • Experience mentoring junior engineers and enforcing best practices around code quality
  • Knowledge of multiple programming languages, commitment to choosing languages based on project-specific requirements, and willingness to learn new programming languages as necessary
  • Knowledge of resource management and automation approaches such as workflow runners
  • Collaborative mentality and excitement for iterative design working closely with the Data Science team.
Stanford Health Care - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Lead Data Engineer - AI/ML

8 matching positions

Senior Data & AI/ML Engineer - GCP Specialization Lead

We are on a bold mission to create the best software services offering in the wo...
Location
Location
United States , Menlo Park
Salary
Salary:
Not provided
techjays.com Logo
techjays
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • GCP Services: BigQuery, Dataflow, Pub/Sub, Vertex AI
  • ML Engineering: End-to-end ML pipelines using Vertex AI / Kubeflow
  • Programming: Python & SQL
  • MLOps: CI/CD for ML, Model deployment & monitoring
  • Infrastructure-as-Code: Terraform
  • Data Engineering: ETL/ELT, real-time & batch pipelines
  • AI/ML Tools: TensorFlow, scikit-learn, XGBoost
  • Min Experience: 10+ Years
Job Responsibility
Job Responsibility
  • Design and implement data architectures for real-time and batch pipelines, leveraging GCP services such as BigQuery, Dataflow, Dataproc, Pub/Sub, Vertex AI, and Cloud Storage
  • Lead the development of ML pipelines, from feature engineering to model training and deployment using Vertex AI, AI Platform, and Kubeflow Pipelines
  • Collaborate with data scientists to operationalize ML models and support MLOps practices using Cloud Functions, CI/CD, and Model Registry
  • Define and implement data governance, lineage, monitoring, and quality frameworks
  • Build and document GCP-native solutions and architectures that can be used for case studies and specialization submissions
  • Lead client-facing PoCs or MVPs to showcase AI/ML capabilities using GCP
  • Contribute to building repeatable solution accelerators in Data & AI/ML
  • Work with the leadership team to align with Google Cloud Partner Program metrics
  • Mentor engineers and data scientists toward achieving GCP certifications, especially in Data Engineering and Machine Learning
  • Organize and lead internal GCP AI/ML enablement sessions
What we offer
What we offer
  • Best in class packages
  • Paid holidays and flexible paid time away
  • Casual dress code & flexible working environment
  • Medical Insurance covering self & family up to 4 lakhs per person
Read More
Arrow Right

Data Engineer Lead (OT Data)

Data Engineer (OT Data) (Category - Engineer) Sector: Oil and Gas Location: Doha...
Location
Location
Qatar , Doha
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's in engineering, Information Systems, or a related quantitative field
  • 5+ years of proven experience in a data engineering role
  • Experience within oil and gas industry is highly preferred
  • Demonstrable experience building and operationalizing large-scale data pipelines and applications
Job Responsibility
Job Responsibility
  • Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines
  • Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians (like OSIsoft PI, Aspen InfoPlus.21), DCS, PLC, and IoT sensors
  • Cloud Data Platform Development: Implement and manage data solutions on the Microsoft Azure cloud platform, Leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics for real-time ingestion and processing of operational technology (OT) data
  • Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics
  • Enable Advanced AI: Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data
  • Champion Master Data Management (MDM): Design and implement MDM strategies and solutions to create a single, authoritative source of truth for critical data domains such as wells, equipment, and assets, ensuring data consistency across the enterprise
  • Ensure Data Quality & Governance: Implement robust data quality checks, validation rules, and monitoring to ensure the accuracy, consistency, and reliability of our data. Adhere to and help shape our data governance policies
  • Embrace Industry Standards: Champion and implement industry-specific data standards and models, such as the OSDU™ Data Platform, to ensure interoperability and a unified data view across the upstream lifecycle
  • Collaborate & Innovate: Work closely with a cross-functional team of geoscientists, drilling engineers, data scientists, and business analysts to understand their data needs and deliver effective solutions
  • Automate & Optimize: Identify opportunities for process automation and infrastructure optimization to improve data delivery, scalability, and cost-effectiveness
  • Fulltime
Read More
Arrow Right

Lead, Data Engineer

The Lead Data Engineer provides technical leadership for the design, development...
Location
Location
Canada , Vancouver
Salary
Salary:
120000.00 - 140000.00 USD / Year
canfor.com Logo
Canfor
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in Computer Science, Information Systems, Mathematics, or Statistics
  • 7–10+ years of experience in enterprise data warehousing, analytics, and data platform engineering
  • 7+ years of demonstrated experience designing and delivering data engineering, ELT, and streaming solutions, including technical leadership across complex initiatives
  • Proven track record designing and evolving scalable, secure, and cost-effective cloud data architectures in an enterprise environment
  • Proven experience in database design, data schema design and data modeling
  • Experience defining and governing enterprise data architecture standards, engineering frameworks, and reference patterns
  • Experience delivering ETL/ELT solutions for importing data from a wide variety of sources
  • Experience working with Microsoft Fabric or Azure Products: Microsoft Fabric, Azure Synapse, Azure Data Factory, Azure Purview, Azure DevOps, etc.
  • Experience designing and building cloud data architecture to facilitate data pipelines and analysis workflows within a cloud environment (preferably Microsoft Fabric)
  • Experience with Enterprise Resource Planning systems (preferably Oracle) is an asset
Job Responsibility
Job Responsibility
  • Lead the design and implementation of batch, real-time, and streaming data pipelines in Microsoft Fabric, while establishing engineering standards, reusable patterns, and delivery best practices
  • Design and build event-driven and near real-time data processing solutions that enable reliable, scalable data movement
  • Own the evolution of the data platform, including scalability, reliability, observability, resiliency, and cost optimization
  • Enable data platforms that support AI/ML and Generative AI use cases
  • Establish data observability, monitoring, and data quality SLAs to ensure trusted, resilient data solutions across the analytics platform
  • Establish and enforce processes and controls that maintain the security, quality, accountability, availability, and compliance of enterprise data assets
  • Partner with business and IT leaders to shape roadmaps, lead delivery planning, manage technical tradeoffs, and execute projects within time and budget constraints
  • Establish and maintain architecture, engineering, operational, and support documentation standards for the data platform
  • Translate business requirements into scalable data platform designs and guide solution implementation to support analytics and reporting delivery
  • Partner with analytics and data science teams to operationalize advanced analytics solutions
What we offer
What we offer
  • performance-based incentive plans
  • recognition programs
  • benefits
  • paid leaves
  • pension plans with base and matching contributions
  • savings options
  • robust health & well-being initiatives
  • Fulltime
Read More
Arrow Right

Data Engineer – Lead

Data Engineer – Lead
Location
Location
India , Bengaluru Urban
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience with Microsoft Fabric, including Lakehouse, Warehouse, OneLake, Pipelines, Dataflows Gen2, Notebooks, and Power BI integration
  • Expertise in ETL/ELT, data pipelines, distributed data processing, and cloud-scale data engineering
  • Strong SQL, Python, PySpark, and data modeling skills
  • Experience with Lakehouse, Warehouse, and Medallion Architecture
  • Understanding of Delta tables, dimensional modeling, star schema, facts, dimensions, and curated analytical datasets
  • Experience integrating structured, semi-structured, file-based, API-based, enterprise application, and cloud data sources
  • Experience with data quality, reconciliation, logging, monitoring, and error-handling frameworks
  • Experience leading technical teams and coordinating onshore/offshore delivery
  • Experience with Git, CI/CD, Azure DevOps, branching, code reviews, and release management
  • Good to Have: Experience with Azure Data Factory, Synapse, Databricks, ADLS Gen2, Azure SQL, Microsoft Purview, or related Azure services
Job Responsibility
Job Responsibility
  • Lead the design and implementation of scalable data pipelines and data processing frameworks in Microsoft Fabric
  • Define data engineering standards, development practices, naming conventions, coding guidelines, and reusable technical patterns
  • Lead implementation of Bronze, Silver, and Gold layers in the Medallion Architecture
  • Oversee ingestion, transformation, orchestration, validation, and publication of data from multiple enterprise, clinical, operational, and cloud-based sources
  • Guide development of Fabric Pipelines, Dataflows Gen2, Notebooks, Lakehouse tables, Warehouse objects, and curated datasets
  • Ensure scalability, performance, reliability, maintainability, security, monitoring, and optimization of data solutions
  • Define standards for data quality, reconciliation, logging, error handling, auditability, and lineage
  • Conduct technical design reviews, code reviews, performance reviews, and deployment readiness reviews
  • Mentor and guide data engineering teams across onshore/offshore locations
  • Collaborate with architects, platform engineers, BI teams, QA teams, AI/ML teams, functional consultants, and stakeholders
Read More
Arrow Right

Lead Data Engineer

We’re looking for a Senior/Lead Data Engineer to join our team. We are seeking a...
Location
Location
India , Noida
Salary
Salary:
Not provided
taazaa.com Logo
Taazaa Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field
  • 7–9 years of experience as a Data Engineer or in a similar role
  • Strong experience in building and maintaining ETL/ELT pipelines
  • Experience supporting analytics and AI/ML data workflows
  • Hands-on experience with ETL tools such as Apache Airflow, Airbyte, and dbt
  • Strong expertise in SQL and NoSQL databases (MySQL, PostgreSQL, MongoDB, Redis)
  • Proficiency in Python and Shell scripting for data processing and automation
  • Experience with cloud platforms (AWS, Azure, GCP) and their data services
  • Familiarity with data warehousing solutions such as Amazon Redshift or Snowflake
  • Experience with containerization tools like Docker and orchestration using Kubernetes
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable ETL/ELT pipelines for processing large datasets
  • Build data pipelines to ingest, transform, and load data into data lakes, warehouses, and feature stores
  • Optimise data workflows for performance, reliability, and scalability
  • Integrate data from multiple sources, including APIs, databases, and file systems (JSON, CSV, Parquet)
  • Manage relational and non-relational databases, including columnar databases like ClickHouse
  • Ensure efficient data storage and retrieval for both analytics and ML workloads
  • Implement data quality checks, validation frameworks, and monitoring systems
  • Maintain data lineage, metadata management, and governance standards
  • Ensure data accuracy, consistency, and compliance for analytics and AI/ML use cases
  • Deploy and manage data solutions on cloud platforms (AWS, Azure, GCP) or on-prem environments
What we offer
What we offer
  • Competitive salaries
  • Health benefits
  • Various perks
  • Competitive compensation and performance-based incentives
  • Opportunities for professional growth through workshops and certifications
  • Flexible work-life balance with remote options
  • Collaborative culture
  • Exposure to diverse projects across various industries
  • Clear career advancement pathways
  • Comprehensive health benefits
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

This role leads enterprise data architecture, focusing on scalable solutions, go...
Location
Location
United States , Fort Washington
Salary
Salary:
145000.00 - 150000.00 USD / Year
pipercompanies.com Logo
Piper Companies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s in Business, CS, Information Systems, or related
  • 7+ years in data engineering & infrastructure
  • Proficient in Python, SQL, ETL, and data integration
  • Experience with BI tools, data modeling, and cloud platforms (Azure)
  • Knowledge of data governance, security, and compliance frameworks
Job Responsibility
Job Responsibility
  • Define enterprise data architecture & infrastructure strategy
  • Design & manage data warehouses, lakes, hybrid environments
  • Implement automation, DevOps, and CI/CD best practices
  • Evaluate & adopt new technologies (AI/ML, advanced analytics)
  • Collaborate with leadership & cross-functional teams
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • 401K
  • PTO
  • Fulltime
Read More
Arrow Right

Lead AI/ML Engineer

Location
Location
India , Hyderabad & Pune
Salary
Salary:
Not provided
fissionlabs.com Logo
Fission Labs
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience with cloud-based platforms (AWS, Azure), API integrations, and data models
  • Exposure to AI/ML-enabled platforms or decision-intelligence systems
  • Certifications: CBAP / PMI-PBA / Agile BA / SAFe Product Owner / Scrum Master
  • Experience in stakeholder training, change management, or workshop facilitation
  • At least 4-5 years in AI/ML infrastructure and large-scale training environments
  • Expert in AWS cloud services (EC2, S3, EKS, SageMaker, Batch, FSx, etc.) and familiar with Azure, GCP, and hybrid/multi-cloud setups
  • Strong knowledge of AI/ML training frameworks (PyTorch, TensorFlow, Hugging Face, DeepSpeed, Megatron, Ray, etc.)
  • Proven experience with cluster orchestration tools (Kubernetes, Slurm, Ray, SageMaker, Kubeflow)
  • Deep understanding of hardware architectures for AI workloads (NVIDIA, AMD, Intel Habana, TPU)
  • Expert knowledge of inference optimization techniques including speculative decoding, KV cache optimization (MQA/GQA/PagedAttention), and dynamic batching
Job Responsibility
Job Responsibility
  • Design, implement, and optimize end-to-end ML training workflows including infrastructure setup, orchestration, fine-tuning, deployment, and monitoring
  • Evaluate and integrate multi-cloud and single-cloud training options across AWS and other major platforms
  • Lead cluster configuration, orchestration design, environment customization, and scaling strategies
  • Compare and recommend hardware options (GPUs, TPUs, accelerators) based on performance, cost, and availability
What we offer
What we offer
  • Opportunities for continuous learning and certification support
  • Collaborative and growth-oriented work culture
  • Competitive compensation and comprehensive benefits
  • Exposure to modern cloud and integration technologies
  • Fulltime
Read More
Arrow Right

Senior AI/ML Lead Engineer

As the Senior AI/ML Lead Engineer, you will spearhead the development and deploy...
Location
Location
United States , Dayton
Salary
Salary:
Not provided
altamiracorp.com Logo
Altamira Technologies
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or above in Computer Science, AI, or a related quantitative field
  • 7+ years of professional experience in machine learning
  • at least 3 years in a leadership or "tech lead" capacity
  • Proven track record in developing computer vision models for object tracking (e.g., CNNs, Transformers) and real-time video analytics
  • Expert-level proficiency in Python
  • familiarity with backend deployment tools (Docker, Kubernetes)
  • Experience overseeing the full data lifecycle, from acquisition and cleaning to high-fidelity labeling for specialized collections
  • Ability to articulate complex AI concepts to non-technical stakeholders and executive leadership
  • Must be a US citizen and hold a current Secret clearance or higher
Job Responsibility
Job Responsibility
  • Design and implement scalable, high-performance infrastructures for real-time object detection and tracking using frameworks like PyTorch or TensorFlow
  • Develop automated triage systems that prioritize and categorize incoming sensor or project data, ensuring critical events are escalated instantly
  • Architect multi-agent or agentic workflows to synchronize data collection efforts across various projects, optimizing resource allocation
  • Lead a high-performing team of engineers, conducting code reviews and setting engineering standards for MLOps and production pipelines
  • Collaborate with project managers to translate complex client requirements into actionable AI/ML roadmaps
  • Fulltime
Read More
Arrow Right