CrawlJobs Logo

Senior Data Pipeline & AI Engineer

United States, Atlanta Employment contract 160000.00 - 170000.00 USD / Year · Job Posted June 10, 2026
Apply Position
Job Link Share

Job Responsibility

  • Maintain, extend, and scale Go microservices that transform and deliver Fullstory session data into customer warehouses and power the team's MCP server that enables AI agent integrations
  • Develop and maintain dbt models and pipeline orchestration to ensure timely, fault-tolerant data migrations across hundreds of customer destinations
  • Define evaluation frameworks for LLM outputs using tools like Langsmith and Vertex AI, ensuring AI-powered customer agents produce accurate, useful results
  • Investigate and resolve production incidents across the data pipeline, implementing systemic fixes that prevent entire classes of failure from recurring
  • Write technical design documents that drive consensus on architectural changes, proactively surfacing scaling bottlenecks, edge cases, and cross-team dependencies
  • Demonstrate sound technical judgment by de-risking work through spikes, taking on tech debt deliberately, and knowing when to escalate versus dig in

Requirements

  • Significant experience building and operating high-throughput data pipelines (batch and/or streaming) in a major cloud platform, including work with cloud data warehouses like BigQuery, Snowflake, or Databricks
  • Proficiency in Go, Python, Java or a similar language
  • Hands-on experience with data transformation tooling such as dbt, with a strong understanding of data modeling and pipeline observability
  • Familiarity with LLM integration patterns and evaluation approaches (e.g., LangSmith, Vertex AI, or comparable frameworks), or demonstrated ability to ramp quickly in applied AI
  • A track record of owning major system areas end-to-end: driving architectural decisions, maintaining production health, and improving reliability over time

What we offer

  • Flexibility and Connection: vibrant HQ in Atlanta and a tight-knit group in London, come to the office at least one day a week, flexible PTO policy, annual company-wide closure, federal holidays
  • Benefits: sponsored benefit packages for US-based Fullstorians, supplemental coverage options for international Fullstorians
  • Learning opportunities: professional development opportunities through training programs and an annual learning subsidy for US and EMEA-based employees
  • Productivity support: monthly productivity stipend for US and EMEA-based Fullstorians
  • Team Collaboration: team off-sites and an annual full-company meet-up
  • Paid parental leave
  • Bereavement leave, including miscarriage/pregnancy loss

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Senior Data Pipeline & AI Engineer

8 matching positions

Senior AI Data Pipeline Engineer

Shape the Future of Intelligence as our next Senior AI Data Pipeline Engineer! A...
Location
Location
United States , Lake Oswego
Salary
Salary:
105600.00 - 145200.00 USD / Year
trimble.com Logo
Trimble Inc.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of experience in data engineering or a related field
  • Strong hands-on experience managing and optimizing Databricks
  • Experience building and maintaining streaming pipelines with Kafka
  • Experience implementing Change Data Capture (CDC) using Debezium connectors
  • Practical experience deploying and operating services in Kubernetes
  • Strong proficiency in Python and/or Scala
  • Experience with SQL and distributed data processing frameworks (e.g., Spark)
  • Familiarity with cloud platforms (AWS, Azure, or GCP)
  • Experience with infrastructure-as-code tools (Terraform, etc.)
  • Strong understanding of distributed systems concepts
Job Responsibility
Job Responsibility
  • Design, build, and optimize scalable batch and real-time data pipelines
  • Manage and administer Databricks workspaces, clusters, jobs, and performance tuning
  • Develop and maintain streaming architectures using Kafka
  • Implement and manage Change Data Capture (CDC) pipelines using Debezium connectors
  • Deploy, monitor, and manage containerized workloads using Kubernetes
  • Implement CI/CD practices for data engineering workflows
  • Ensure data quality, observability, governance, and security best practices
  • Collaborate with data scientists, ML engineers, and software engineers to deliver production-grade data solutions
  • Support and optimize AI/ML data pipelines and model deployment workflows
  • Troubleshoot production issues and implement performance improvements
What we offer
What we offer
  • Medical
  • Dental
  • Vision
  • Life
  • Disability
  • Time off plans
  • retirement plans
  • tax savings plans for health, dependent care and commuter expenses
  • Paid Parental Leave
  • Employee Stock Purchase Plan
  • Fulltime
Read More
Arrow Right

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...
Location
Location
India , Chennai, Madurai, Coimbatore
Salary
Salary:
Not provided
optisolbusiness.com Logo
OptiSol Business Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
  • Excellent Python programming skills with data engineering libraries and frameworks
  • Advanced SQL capabilities including window functions, CTEs, and performance tuning
  • Solid understanding of distributed processing using Spark/PySpark
  • Experience designing and implementing scalable ETL/ELT workflows
  • Good understanding of data modeling concepts (dimensional, star, snowflake)
  • Familiarity with GenAI/LLM-based integration for data workflows
  • Experience working with Git, CI/CD, and Agile delivery frameworks
  • Strong communication skills for interacting with clients, stakeholders, and internal teams
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
  • Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
  • Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
  • Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
  • Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
  • Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
  • Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
  • Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations
What we offer
What we offer
  • Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
  • Hands-on exposure to modern data stacks and emerging AI technologies
  • Collaboration with experts across Data, AI/ML, and cloud practices
  • Access to structured learning, certifications, and leadership mentoring
  • Competitive compensation with fast-track career growth and visibility
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - AI and Analytics

We're building a world of health around every individual — shaping a more connec...
Location
Location
United States , Buffalo Grove
Salary
Salary:
101970.00 - 203940.00 USD / Year
https://www.cvshealth.com/ Logo
CVS Health
Expiration Date
June 24, 2026
Flip Icon
Requirements
Requirements
  • 3-5+ years of experience with SQL, NoSQL
  • 3-5+ years of experience with Python
  • 3+ years of experience with Data warehouses (such as data modeling and technical architectures) and infrastructure components
  • 3+ years of experience with ETL/ELT, and building high-volume data pipelines
  • 3+ years of experience with reporting/analytic tools
  • 3+ years of experience with Query optimization, data structures, transformation, metadata, dependency, and workload management
  • 3+ years of experience with Big data and cloud architecture
  • 3+ years of hands-on experience building modern data pipelines within a major cloud platform (preferably GCP, open to AWS or Azure)
  • 3+ years of experience with deployment/scaling of apps on containerized environment (i.e. Kubernetes, AKS)
  • 3+ years of experience with real-time and streaming technology (i.e. Azure Event Hubs, Azure Functions, Kafka, Spark Streaming)
Job Responsibility
Job Responsibility
  • Design, develop, and maintain optimal data pipelines to assemble large and intricate datasets
  • Cater to the business requirements of various CVS lines of business
  • Collaborate closely with teams to craft tools to provide actionable insights and integrate them with consumer touchpoints
  • Solve problems associated with large scale complex, structured and unstructured data
What we offer
What we offer
  • Medical, dental, and vision coverage
  • paid time off
  • retirement savings options
  • wellness programs
  • bonus, commission or short-term incentive program
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - AI Infrastructure

We are building a large-scale data platform that transforms raw system logs into...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role.
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Design and implement large-scale data pipelines using PySpark and distributed processing frameworks
  • Build and maintain data models that accurately represent underlying system behavior and business logic
  • Ensure high standards of data correctness, completeness, and consistency across datasets
  • Develop validation, monitoring, and alerting mechanisms to detect data quality issues
  • Partner with data scientists to support experimentation and analytics use cases
  • Collaborate with platform engineers to ensure efficient data ingestion, processing, and storage
  • Optimize pipelines for performance, scalability, and cost efficiency
  • Define and enforce best practices for schema design, data transformations, and pipeline reliability
  • Fulltime
Read More
Arrow Right

Senior Data Engineer - AI Focused

At Doctolib, we're on a mission to transform healthcare through the power of AI....
Location
Location
France , Paris
Salary
Salary:
Not provided
doctolib.fr Logo
Doctolib
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master’s or Ph.D. degree in Computer Science, Data Engineering, or a related field
  • 5+ years of experience in Data Engineering, ideally supporting AI or ML workloads
  • Strong experience with the GCP data ecosystem
  • Proficiency in Python and SQL, with experience in data pipeline orchestration (e.g., Airflow, Dagster, Cloud Composer)
  • Deep understanding of NoSQL systems (e.g., MongoDB) and vector databases (e.g., FAISS, Vector Search)
  • Experience designing data architectures for RAG, embeddings, or model training pipelines
  • Knowledge of data governance, security, and compliance for sensitive or regulated data
  • Familiarity with W&B / MLflow / Braintrust / DVC for experiment tracking and dataset versioning (extract snapshots, change tracking, reproducibility)
  • Familiarity with containerized environments (Docker, Kubernetes) and CI/CD for data workflows
  • A collaborative mindset and passion for building the data foundations of next-generation AI systems
Job Responsibility
Job Responsibility
  • Ensure high standards of data quality for AI model inputs
  • Design, build, and maintain scalable data pipelines on Google Cloud Platform (GCP) for AI and machine learning use cases
  • Implement data ingestion and transformation frameworks that power Retrieval systems and training datasets for LLMs and multimodal models
  • Architect and manage NoSQL and Vector Databases to store and retrieve embeddings, documents, and model inputs efficiently
  • Collaborate with ML and platform teams to define data schemas, partitioning strategies, and governance rules that ensure privacy, scalability, and reliability
  • Integrate unstructured and structured data sources (text, speech, image, documents, metadata) into unified data models ready for AI consumption
  • Optimize performance and cost of data pipelines using GCP native services (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Vertex AI)
  • Contribute to data quality and lineage frameworks, ensuring AI models are trained on validated, auditable, and compliant datasets
  • Continuously evaluate and improve our data stack to accelerate AI experimentation and deployment
What we offer
What we offer
  • Free comprehensive health insurance for you and your children
  • Parent Care Program: additional leave on top of the legal parental leave
  • Free mental health and coaching services through our partner Moka.care
  • For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support
  • Work from EU countries and the UK for up to 10 days per year, thanks to our flexibility days policy
  • Work Council subsidy to refund part of a sport club membership or a creative class
  • Up to 14 days of RTT
  • Lunch voucher with Swile card
  • Fulltime
Read More
Arrow Right

Senior AI & Data Engineer

The Senior AI & Data Engineer is an individual contributor role that acts as the...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.hpe.com/ Logo
Hewlett Packard Enterprise
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Data Science, AI/ML, Engineering, Mathematics, or a related technical discipline
  • PhD is a plus
  • 7 – 10 years of hands-on experience in AI/ML engineering, applied data science, or LLM engineering roles
  • Proven track record of delivering production AI systems
  • Deep expertise with at least two major LLM platforms (Claude, GPT, Gemini, or equivalent)
  • Significant experience with Collibra or an equivalent enterprise data governance platform
  • Demonstrated experience leading cross-functional AI initiatives and mentoring junior engineers
  • Strong ML fundamentals alongside modern generative AI skills
  • Experience with responsible AI practices, including fairness auditing, explainability, and content safety, is strongly preferred
Job Responsibility
Job Responsibility
  • Serve as the dual AI & data SME for the team and organization
  • Define and uphold engineering standards, design patterns, and best practices across both AI and data engineering disciplines
  • Lead technical discovery for new AI and data use cases
  • Participate in and lead cross-functional initiatives where AI and data strategy intersect
  • Mentor and upskill the Applied AI Engineer and AI Data Engineer
  • Architect and deliver complex agentic AI systems
  • Design and implement advanced RAG architectures
  • Lead LLM evaluation frameworks
  • Assess and implement LLM fine-tuning and alignment strategies
  • Own LLM integration architecture
What we offer
What we offer
  • Health & Wellbeing
  • Personal & Professional Development
  • Unconditional Inclusion
Read More
Arrow Right

Senior AI Data Engineer

We are looking for a Senior AI Data Engineer to join a high-impact AI product in...
Location
Location
United States
Salary
Salary:
Not provided
velvetech.com Logo
Velvetech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in Data Engineering / ML Engineering / AI Engineering
  • Strong programming skills in Python
  • Hands-on experience with PyTorch (training and deploying deep learning models)
  • Experience working with Vertex AI or similar ML platforms (GCP preferred)
  • Proven experience with vector databases (Milvus, Pinecone, or similar)
  • Strong knowledge of: Feature engineering techniques, Model evaluation and validation frameworks, Predictive inference systems
  • Experience with multiple database paradigms: Relational (PostgreSQL), Time-series (InfluxDB), Graph (Neo4j)
  • Solid understanding of embeddings and semantic/vector search systems
  • Experience implementing model lifecycle management, including: Drift detection, Monitoring, Governance
  • Strong understanding of scalable system design and performance optimization
Job Responsibility
Job Responsibility
  • Own and develop the biometric extraction model lifecycle (training, validation, deployment)
  • Design and maintain a vector memory layer using tools such as Milvus or Pinecone
  • Build and optimize predictive inference services for real-time and batch use cases
  • Develop and maintain data pipelines for PFM (Personal Financial Management) data preparation
  • Implement advanced feature engineering frameworks and model evaluation pipelines
  • Work with Vertex AI for model training, deployment, and orchestration
  • Manage and integrate heterogeneous data storage systems: InfluxDB (time-series data), PostgreSQL (relational data), Neo4j (graph data)
  • Develop vector embeddings pipelines and similarity search logic
  • Implement model governance processes: Drift detection and monitoring, Shadow-mode validation, Performance tracking and reporting
  • Design and apply optimization policies for inference latency, cost, and accuracy
What we offer
What we offer
  • FLEXIBLE working conditions
  • COOPERATIVE environment
  • Competitive salary
  • Many CHALLENGING and exciting projects with new opportunities and learning
  • GROWTH opportunities, skills and competencies improvement, and professional certification
  • In-company TRAINING (English, Software / DevOps / Project management / Design / Business)
  • Fulltime
Read More
Arrow Right

Senior AI Data Engineer

VideoAmp is seeking a Senior AI Data Engineer to join our Linear Data Processing...
Location
Location
United States
Salary
Salary:
150000.00 - 170000.00 USD / Year
videoamp.com Logo
VideoAmp
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in AI/ML engineering, applied ML systems, or data engineering
  • Demonstrated experience shipping LLM-powered systems into production
  • Strong understanding of: Prompt engineering and evaluation frameworks
  • Embeddings and similarity search on structured and semi-structured data
  • Hybrid AI systems that combine LLMs with deterministic logic
  • Proficiency in Python and SQL
  • strong software engineering fundamentals
  • Experience working with large-scale data platforms and distributed systems
  • Ability to design AI systems that meet measurement-grade reliability, explainability, and auditability requirements
Job Responsibility
Job Responsibility
  • Design and operate AI- and LLM-enabled data pipelines processing billions of linear and digital records
  • Build and productionize LLM-driven systems for concrete LDP problems, including: Network & Program Schedules
  • Hourly Viewership Logs
  • Commercial & Ad Spot Metadata
  • Combine LLMs with rules, heuristics, and ML models to ensure deterministic, auditable outcomes
  • Own the end-to-end AI lifecycle, including: Prompt design and evaluation
  • Feature engineering and training pipelines
  • Deployment, monitoring, drift detection, and retraining
  • Integrate LLMs responsibly into batch and event-driven LDP workflows, balancing accuracy, latency, and cost
  • Build human-in-the-loop workflows for high-impact or low-confidence cases
What we offer
What we offer
  • Equity
  • Discretionary and flexible paid time off
  • In addition to standard US holidays off, VideoAmp employees also partake in Spring, Summer and Winter breaks
  • Comprehensive medical, dental, and vision benefits for you and your dependents—including multiple options fully covered by VideoAmp
  • Unlimited financial wellness sessions with Origin financial advisors
  • 401k Plan with matching
  • HSA & FSA
  • Commuter Benefits
  • Cell Phone Reimbursement
  • Paid Maternity and Parental Leave for All Family Additions
  • Fulltime
Read More
Arrow Right