CrawlJobs Logo
Robert Half Logo Robert Half · IT - Software Development

Senior Data Engineer – Ingest

United States, Glendale · Job Posted March 05, 2026
Apply Position
Job Link Share

Job Description

We are seeking a Senior Data Engineer – Ingest to help transform data into meaningful insights and power innovation across the organization. In this role, you will work with a collaborative team of technologists to build scalable data solutions, integrate diverse data sources, and strengthen the core data platform. Your engineering expertise will directly support analytics, data science, operations, and key business stakeholders. If you’re passionate about building high‑quality data systems that make a measurable impact, this role offers the opportunity to shape the future of a large, data‑driven organization.

Job Responsibility

  • Maintain, update, and expand configuration‑driven data pipelines within the core data platform
  • Build tools and services supporting data discovery, lineage, governance, and privacy
  • Partner with software engineers, data engineers, architects, and product managers to deliver reliable and scalable data solutions
  • Help define and document data standards, naming conventions, pipeline best practices, and system guidelines
  • Ensure the reliability, accuracy, and operational efficiency of datasets to meet SLAs
  • Participate in Agile/Scrum ceremonies and contribute to ongoing process improvements
  • Collaborate closely with users and stakeholders to understand needs and prioritize enhancements
  • Maintain detailed technical documentation to support data quality, governance, and compliance requirements

Requirements

  • 5+ years of Data Engineering experience
  • Strong SQL expertise
  • Hands-on experience with Databricks
  • Proficiency with Python and Snowflake
  • Experience working in Agile environments
  • Data analysis capabilities
  • Bachelor’s degree in Computer Science, Information Systems, or equivalent experience

Nice to have

  • 7+ years developing large-scale data pipelines
  • Experience with at least one major programming language (Python, Java, or Scala)
  • Experience with pipeline orchestration tools (Airflow preferred)
  • Cloud experience (AWS preferred), Kubernetes a plus
  • Experience with ingest tools (Fivetran, Airbyte, DataFlow, Matillion, etc.)
  • Strong understanding of cloud infrastructure and IaC
  • Familiarity with data modeling and data warehousing methodologies
  • Strong algorithmic problem-solving skills
  • Excellent written and verbal communication skills
  • Experience with distributed data processing, data services, or data modeling
  • Familiarity with Scrum/Agile methodologies
  • Ability to learn quickly and work independently with high attention to detail

What we offer

  • medical
  • vision
  • dental
  • life and disability insurance
  • 401(k) plan
Robert Half - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Data Engineer – Ingest

8 matching positions

Senior AWS Data Engineer / Data Platform Engineer

We are seeking a highly experienced Senior AWS Data Engineer to design, build, a...
Location
Location
United Arab Emirates , Dubai
Salary
Salary:
Not provided
northbaysolutions.com Logo
NorthBay
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of experience in data engineering and data platform development
  • Strong hands-on experience with: AWS Glue
  • Amazon EMR (Spark)
  • AWS Lambda
  • Apache Airflow (MWAA)
  • Amazon EC2
  • Amazon CloudWatch
  • Amazon Redshift
  • Amazon DynamoDB
  • AWS DataZone
Job Responsibility
Job Responsibility
  • Design, develop, and optimize scalable data pipelines using AWS native services
  • Lead the implementation of batch and near-real-time data processing solutions
  • Architect and manage data ingestion, transformation, and storage layers
  • Build and maintain ETL/ELT workflows using AWS Glue and Apache Spark on EMR
  • Orchestrate complex data workflows using Apache Airflow (MWAA)
  • Develop and manage serverless data processing using AWS Lambda
  • Design and optimize data warehouses using Amazon Redshift
  • Implement and manage NoSQL data models using Amazon DynamoDB
  • Utilize AWS DataZone for data governance, cataloging, and access management
  • Monitor, log, and troubleshoot data pipelines using Amazon CloudWatch
  • Fulltime
Read More
Arrow Right

Senior Data Engineer – Data Engineering & AI Platforms

We are looking for a highly skilled Senior Data Engineer (L2) who can design, bu...
Location
Location
India , Chennai, Madurai, Coimbatore
Salary
Salary:
Not provided
optisolbusiness.com Logo
OptiSol Business Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on expertise in cloud ecosystems (Azure / AWS / GCP)
  • Excellent Python programming skills with data engineering libraries and frameworks
  • Advanced SQL capabilities including window functions, CTEs, and performance tuning
  • Solid understanding of distributed processing using Spark/PySpark
  • Experience designing and implementing scalable ETL/ELT workflows
  • Good understanding of data modeling concepts (dimensional, star, snowflake)
  • Familiarity with GenAI/LLM-based integration for data workflows
  • Experience working with Git, CI/CD, and Agile delivery frameworks
  • Strong communication skills for interacting with clients, stakeholders, and internal teams
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ETL/ELT pipelines across cloud and big data platforms
  • Contribute to architectural discussions by translating business needs into data solutions spanning ingestion, transformation, and consumption layers
  • Work closely with solutioning and pre-sales teams for technical evaluations and client-facing discussions
  • Lead squads of L0/L1 engineers—ensuring delivery quality, mentoring, and guiding career growth
  • Develop cloud-native data engineering solutions using Python, SQL, PySpark, and modern data frameworks
  • Ensure data reliability, performance, and maintainability across the pipeline lifecycle—from development to deployment
  • Support long-term ODC/T&M projects by demonstrating expertise during technical discussions and interviews
  • Integrate emerging GenAI tools where applicable to enhance data enrichment, automation, and transformations
What we offer
What we offer
  • Opportunity to work at the intersection of Data Engineering, Cloud, and Generative AI
  • Hands-on exposure to modern data stacks and emerging AI technologies
  • Collaboration with experts across Data, AI/ML, and cloud practices
  • Access to structured learning, certifications, and leadership mentoring
  • Competitive compensation with fast-track career growth and visibility
  • Fulltime
Read More
Arrow Right

Senior Data Engineer / Data Analyst

N-iX is a global software development service company. Our customer is the Europ...
Location
Location
Ukraine
Salary
Salary:
Not provided
n-ix.com Logo
N-iX
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4+ years of experience in analytics, sales operations, revenue insights, or data-driven consulting
  • Strong SQL skills (joins, window functions, performance tuning)
  • Production experience with Apache Airflow
  • Python skills for pipeline development
  • Ability to build and maintain database views (PostgreSQL, Snowflake, Redshift)
  • Solid ETL/ELT understanding
  • Clear communication with non-technical stakeholders
  • GitHub experience (branches, PRs, reviews)
  • English level - at least Upper-Intermediate, both spoken and written
Job Responsibility
Job Responsibility
  • Work with Salesforce objects and relationships to ensure correct ingestion, transformation, and integration into our reporting environment
  • Design, build, and maintain data pipelines using Apache Airflow (DAG creation, scheduling, monitoring, troubleshooting)
  • Write, optimize, and maintain SQL queries, stored procedures, and functions for data transformation and extraction
  • Create and manage database views to support analytics, reporting, and downstream applications
  • Ensure data quality, consistency, and reliability across pipelines and views (validation checks, monitoring)
  • Support QuickSuite dataset adjustments (new fields, logic changes, view extensions)
  • Document data flows, data models, and pipeline logic for long-term maintainability and handover
What we offer
What we offer
  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits
Read More
Arrow Right

Senior Data Engineer Lead / Architect - Senior Vice President

At Citi Services - Global Trade Technology Organization, we are on a mission to ...
Location
Location
India , Pune, Maharashtra, India, Chennai, Tamil Nadu, India
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of professional experience in data engineering, with a proven track record of designing and building large-scale data systems
  • 3+ years in a technical leadership or architect role, with experience mentoring junior and senior engineers
  • Expert-level proficiency in at least one programming language (Python or Scala preferred) and exceptional SQL skills
  • Proven hands-on experience with Python or Scala for data manipulation, scripting, machine learning, and backend development
  • Deep, hands-on experience with a major cloud platform (AWS, GCP, or Azure) and its data ecosystem (e.g., S3/GCS, Redshift/BigQuery, EMR/Dataproc, Kinesis/Dataflow)
  • Extensive hands-on experience with modern big data technologies and Data streaming (like Hadoop, Hive, Impala, Apache Spark, Kafka, or Flink)
  • Proficiency with workflow orchestration tools such as Airflow, Dagster, or Prefect
  • Proficiency in designing and implementing microservices architectures, RESTful APIs, and event-driven systems with 'Data as a Product' Principle
  • Solid understanding of data modeling concepts and database design for both analytical (OLAP) and transactional (OLTP) workloads
  • Deep understanding and hands-on experience with relational databases (e.g., PostgreSQL, Oracle), NoSQL databases (e.g., MongoDB, Cassandra), data warehousing, and big data technologies (e.g., Spark, Kafka)
Job Responsibility
Job Responsibility
  • Architect & Design: Design, architect, and oversee the development of robust, scalable, and reliable data infrastructure, including data lakes, data warehouses, and real-time streaming platforms on the cloud
  • Build & Code: Act as a senior individual contributor and hands-on technical leader. Write clean, maintainable, and high-performance code for data ingestion, transformation, and serving layers (e.g., using Python, Scala, SQL, and Spark)
  • Lead & Mentor: Lead a team of data engineers, providing technical guidance, mentorship, and career development support. Foster a collaborative and inclusive team environment
  • Champion Culture: Define, document, and champion data engineering best practices across the organization, including CI/CD, data quality, testing frameworks, observability, and code review standards
  • Drive Strategy: Partner with leadership, product managers, data scientists, and analysts to understand data needs and develop a long-term data strategy and roadmap
  • Innovate & Evaluate: Stay at the forefront of data engineering technologies. Evaluate, prototype, and recommend new tools and frameworks to continuously improve our data platform
  • Ensure Governance: Implement and enforce robust data governance, security, and privacy policies in partnership with our security and compliance teams
  • Fulltime
Read More
Arrow Right

Senior Staff Data Engineer- Data Platform

At Marktplaats, data is at the heart of everything we do, but Intelligence is wh...
Location
Location
Netherlands , Amsterdam
Salary
Salary:
Not provided
adevinta.com Logo
Adevinta
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of hands-on experience in Software Development or Data Engineering
  • at least 5+ years specifically focused on building Data Platforms
  • deep understanding of how Platform infra supports Analytics workloads
  • proven experience evolving complex platforms from legacy patterns to modern, cloud-native solutions
  • deep knowledge of Spark internals, JVM tuning, and performance optimization for high-scale batch and streaming datasets
  • deep expertise in Unity Catalog, Delta Lake internals, and optimizing high-volume workloads
  • strict software engineering discipline (CI/CD, Testing, OOP) applied to data pipelines
  • understanding of microservices architecture
  • understanding the needs of Analytics/DWH teams (Data Modeling, dbt)
  • strong background in building automated pipelines using Terraform/Terragrunt and ensuring system observability
Job Responsibility
Job Responsibility
  • Lead the evolution of our Data Platform and architect the "Data Exchange" strategy
  • define robust patterns for API-based ingestion, Event-Driven Architectures (Kafka), and Reverse ETL
  • ensure architectures are optimized for cost and performance on AWS
  • act as a catalyst for technical evolution
  • constantly scan the horizon for next-generation technologies
  • lead the implementation of new paradigms
  • design the strategy for Unity Catalog implementation and Data Contracts
  • champion FinOps, automating cost controls for our highest-volume workloads
  • build the underlying infrastructure that allows Analytics/DWH teams to run efficient transformations
  • elevate the technical bar of the team, mentoring Staff and Senior engineers
What we offer
What we offer
  • An attractive Base Salary
  • Participation in our Short Term Incentive plan (annual bonus)
  • Work From Anywhere: Enjoy up to 20 days a year of working from anywhere
  • A 24/7 Employee Assistance Program for you and your family
  • a collaborative environment with an opportunity to explore your potential and grow
  • a range of locally relevant benefits
  • Fulltime
Read More
Arrow Right

Senior Solutions Engineer – Big Data & Data Infrastructure

This is a great opportunity to be part of one of the fastest-growing infrastruct...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
vastdata.com Logo
VAST Data
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2–4 years in software / solution or infrastructure engineering
  • 2–4 years focused on building / maintaining large-scale data pipelines / storage & database solutions
  • Proficiency in Trino, Spark (Structured Streaming & batch) and solid working knowledge of Apache Kafka
  • Coding background in Python (must-have)
  • Deep understanding of data storage architectures including SQL, NoSQL, and HDFS
  • Solid grasp of DevOps practices, including containerization (Docker), orchestration (Kubernetes), and infrastructure provisioning (Terraform)
  • Experience with distributed systems, stream processing, and event-driven architecture
  • Hands-on familiarity with benchmarking and performance profiling for storage systems, databases, and analytics engines
  • Excellent communication skills
Job Responsibility
Job Responsibility
  • Build distributed data pipelines using technologies like Kafka, Spark (batch & streaming), Python, Trino, Airflow, and S3-compatible data lakes
  • Design, deploy, and troubleshoot hybrid cloud/on-prem environments using Terraform, Docker, Kubernetes, and CI/CD automation tools
  • Implement event-driven and serverless workflows
  • Create technical guides, architecture docs, and demo pipelines
  • Integrate data validation, observability tools, and governance directly into the pipeline lifecycle
  • Own end-to-end platform lifecycle: ingestion → transformation → storage (Parquet/ORC on S3) → compute layer (Trino/Spark)
  • Benchmark and tune storage backends (S3/NFS/SMB) and compute layers for throughput, latency, and scalability using production datasets
  • Work cross-functionally with R&D to push performance limits across interactive, streaming, and ML-ready analytics workloads
  • Operate and debug object store–backed data lake infrastructure
Read More
Arrow Right
New

Senior Data Engineer

Location
Location
India , Bangalore
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5–8 Years Experience
  • Apache Iceberg
  • Python
  • SQL
  • Data Modeling
  • ETL/ELT
  • AWS Data Services
Job Responsibility
Job Responsibility
  • Build governed data products using Apache Iceberg
  • Develop scalable ingestion pipelines
  • Implement data validation, reconciliation, and lineage
  • Maintain schema evolution and auditing capabilities
  • Integrate structured and unstructured datasets
  • Enable analytics and AI consumption
  • Optimize storage and query performance
  • Support data governance initiatives
Read More
Arrow Right
New

Senior Data Engineer

Radancy’s Data Engineering team is seeking a Senior Data Engineer to join our Ba...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
radancy.com Logo
Radancy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 4-6 years of experience in data engineering, including strong hands-on experience with SQL, Python, and data modeling
  • Experience building and optimizing data pipelines using Apache Airflow or similar orchestration tools
  • Strong experience with cloud data platforms such as Google BigQuery, Amazon Redshift, Databricks, Snowflake, or similar technologies
  • Experience working with large-scale structured and semi-structured datasets
  • Familiarity with distributed data processing technologies such as Spark, Kafka, or similar frameworks
  • Experience with data quality, data validation, pipeline monitoring, and automated testing practices
  • Exposure to Generative AI tools such as OpenAI, Gemini, GitHub Copilot, Cursor, or similar tools is strongly preferred
  • Ability to use AI-assisted or agentic development tools to improve productivity, troubleshoot issues, generate tests, and accelerate delivery
  • Experience with Machine Learning, MLOps, predictive analytics, or AI-powered analytics is a plus
  • Experience building datasets or platforms that support analytics products, conversational insights, data agents, or business intelligence solutions is a plus
Job Responsibility
Job Responsibility
  • Design, develop, and optimize data models, schemas, and datasets for cloud-based data platforms
  • Build, optimize, and maintain reliable data pipelines using Python, SQL, Airflow, and cloud data technologies
  • Ingest, transform, aggregate, and curate data from diverse internal and external sources to create high-quality datasets for analytics, reporting, AI, and agentic solutions
  • Work with domain experts, product teams, analytics teams, and engineering stakeholders to understand business needs and translate them into scalable data solutions
  • Support the development of Insights Agents and Data Agents that help users ask questions, generate insights, summarize trends, and make data-driven decisions
  • Develop AI-assisted and agentic solutions that generate actionable insights, business narratives, and compelling data stories from structured and semi-structured data
  • Use agentic software development tools and Generative AI tools to improve engineering productivity, including code generation, test creation, documentation, data validation, troubleshooting, and workflow automation
  • Continuously improve pipeline performance, reliability, observability, and cost efficiency through optimization, automation, and proactive monitoring
  • Build and maintain data quality frameworks, automated tests, validation checks, and monitoring processes to ensure trustworthy data
  • Collaborate with global engineering teams to define standards for data collection, modeling, governance, and AI-readiness
  • Fulltime
Read More
Arrow Right