CrawlJobs Logo
Robert Half Logo Robert Half · IT - Software Development

Sr. AWS Data Engineer

United States, Atlanta · Job Posted April 10, 2026
Apply Position
Job Link Share

Job Description

We are looking for a Data Operations Engineer to support and oversee the automated data‑pipeline environment built on AWS. This position bridges data engineering and customer operations, ensuring that incoming datasets are processed accurately, consistently, and securely within established ingestion and transformation frameworks. Key responsibilities include monitoring automated workflows, troubleshooting processing failures, validating data quality, and helping onboard new customers by aligning their data formats to a standardized internal model.

Job Responsibility

  • Monitor automated batch and streaming data pipelines in AWS
  • Identify, troubleshoot, and resolve data processing failures
  • Investigate file‑level errors, schema mismatches, and transformation issues
  • Perform root‑cause analysis and document resolutions
  • Ensure data integrity, completeness, and timeliness across environments
  • Escalate architectural or systemic issues to the Data Engineering team
  • Collaborate directly with customers to understand their file formats and data structures
  • Create and maintain mapping templates to align customer data to a normalized data model
  • Validate sample files and run tests on ingestion workflows
  • Configure ingestion parameters within predefined frameworks
  • Support customer go‑live processes and initial data processing cycles
  • Write SQL queries to validate data accuracy and research anomalies
  • Develop lightweight Python scripts for validation, transformation checks, or automation tasks
  • Improve monitoring processes, internal documentation, and operational playbooks
  • Work with engineering teams to strengthen platform reliability and observability
  • Communicate clearly with customers regarding file issues or data discrepancies
  • Partner with internal teams including Data Engineering, Product, and Support
  • Provide feedback to enhance scalability, resilience, and overall platform performance

Requirements

  • 6–10 years of experience in data operations, data engineering, or data support roles
  • Experience with ETL/ELT orchestration tools
  • Hands‑on experience with AWS services such as S3, Glue, Lambda, CloudWatch, Redshift, or similar
  • Strong SQL skills (joins, aggregations, troubleshooting data discrepancies)
  • Working knowledge of Python for scripting and data validation
  • Experience troubleshooting automated data pipelines
  • Familiarity with structured and semi‑structured data formats (CSV, JSON, Parquet)
  • Strong analytical and problem‑solving skills
  • Comfortable interacting with customers in technical discussions
  • Experience working in a healthcare or pharmaceutical environment

Nice to have

  • Experience working within a data lake or data‑warehouse architecture
  • Familiarity with data catalogs or governance frameworks
  • Understanding of data normalization and schema design
  • Experience working in a SaaS environment

What we offer

  • medical
  • vision
  • dental
  • life and disability insurance
  • company 401(k) plan
Robert Half - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Sr. AWS Data Engineer

8 matching positions

Sr. Data Engineer - AWS

Location
Location
Salary
Salary:
Not provided
3pillarglobal.com Logo
3Pillar Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • How many years of experience do you have as AWS Data Engineer?
  • How many years of experience do you have working with Core Python and its libraries?
  • Please list the AWS services you have used (Skills level should be intermediate /expertise)
  • Please share your current CTC and expected CTC.
  • Please share your official notice period
  • Fulltime
Read More
Arrow Right

Sr. Data Engineer - AWS

Embark on an exciting journey into the realm of software development with 3Pilla...
Location
Location
India , Noida
Salary
Salary:
Not provided
3pillarglobal.com Logo
3Pillar Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 5 years of experience in data engineering or a related field
  • Strong proficiency in Python programming language
  • Deep understanding of AWS services, including Kinesis, S3, Athena, Redshift, DynamoDB, and Lambda
  • Experience with data ingestion pipelines, ETL processes, and data warehousing concepts
  • Proficiency in SQL and NoSQL databases
  • Experience with data modeling and schema design
  • Strong problem-solving and analytical skills
  • Excellent communication and collaboration skills
  • Experience with data security and privacy best practices
Job Responsibility
Job Responsibility
  • Develop, Maintain and Enhance ETL Pipelines
  • Craft client code that is not just efficient, but also performant, testable, scalable, secure, and of the highest quality
  • Provide accurate status tracking, reporting, and estimation using project methods and tools
  • Gather requirements, validate their understanding amongst the team, create and maintain relevant documentation
  • Execute activities within current methodology and upholding the highest quality standards
  • Foster collaboration with fellow engineers, designers, and managers to comprehend user pain points and iterate on solutions that drive client success
  • Take ownership of projects, from technical design to a successful launch
What we offer
What we offer
  • Imagine a flexible work environment – whether it's the office, your home, or a blend of both
  • From interviews to onboarding, we embody a remote-first approach
  • You will be part of a global team, learning from top talent around the world and across cultures, speaking English everyday
  • Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams
  • We’re big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing
  • Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best
  • Our professional services model enables us to accelerate career growth and development opportunities - across projects, offerings, and industries
  • We are an equal opportunity employer
  • It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally
  • Fulltime
Read More
Arrow Right

Data Engineer Sr (AWS)

Data Engineers are responsible for designing, building, and maintaining the syst...
Location
Location
Mexico , GDL
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6 - 8 years of experience supporting Software Engineering, Data Engineering, or Data Analytics projects
  • 2+ years of experience leading a team supporting data related projects to develop end-to-end technical solutions
  • Strong proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server)
  • Experience with data pipeline tools (e.g., Apache Airflow, Luigi, Prefect)
  • Proficiency with at least one programming language (Python, Java, or Scala)
  • Experience with cloud platforms (AWS) and their data services (e.g., Redshift, BigQuery, Snowflake, Databricks)
  • Familiarity with data modeling, warehousing, and schema design
  • Understanding of data governance, privacy, and security best practices
Job Responsibility
Job Responsibility
  • Design and implement tailored data solutions to meet customer needs and use cases
  • Provide thought leadership by recommending the most appropriate technologies and solutions
  • Demonstrate proficiency in coding skills to efficiently move solutions into production while prioritizing performance, security, scalability, and robust data integrations
  • Collaborate seamlessly across diverse technical stacks
  • Develop and deliver detailed presentations to effectively communicate complex technical concepts
  • Generate comprehensive solution documentation
  • Adhere to Agile practices throughout the solution development process
  • Design, build, and deploy databases and data stores to support organizational requirements
  • Ensure data quality, consistency and governance across multiple sources
Read More
Arrow Right

Sr. Data Engineer – Clinical Data Foundation

The Sr. Data Engineer is responsible for designing, building, maintaining, analy...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master’s /Bachelor’s degree with 9-12 years of experience in Computer Science, IT or related field
  • Hands on experience with big data technologies and platforms, such as Databricks, Apache Spark (PySpark, SparkSQL), workflow orchestration, performance tuning on big data processing
  • Hands on experience with various Python/R packages for data analysis, feature engineering and machine learning model training
  • Proficiency in data analysis tools (eg. SQL) and experience with data visualization tools
  • Excellent problem-solving skills and the ability to work with large, complex datasets
  • Strong understanding of data governance frameworks, tools, and best practices.
  • Knowledge of data protection regulations and compliance requirements (e.g., GDPR, CCPA)
Job Responsibility
Job Responsibility
  • Design, develop, and maintain data solutions for data generation, collection, and processing
  • Be a key team member that assists in design and development of the data pipeline
  • Create data pipelines and ensure data quality by implementing ETL processes to migrate and deploy data across systems
  • Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions
  • Take ownership of data pipeline projects from inception to deployment, manage scope, timelines, and risks
  • Collaborate with cross-functional teams to understand data requirements and design solutions that meet business needs
  • Develop and maintain data models, data dictionaries, and other documentation to ensure data accuracy and consistency
  • Implement data security and privacy measures to protect sensitive data
  • Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions
  • Collaborate with Data Architects, Business SMEs, and Data Scientists to design and develop end-to-end data pipelines to meet fast paced business needs across geographic regions
Read More
Arrow Right

Data Engineer Sr

Job Summary: Built on a solid, family foundation, we've been building homes acro...
Location
Location
United States , Atlanta
Salary
Salary:
Not provided
beazer.com Logo
Beazer Homes
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Typically requires a Bachelor’s degree with 8+ years experience, a Master’s degree and 6+ years experience, a PhD with 3+ years experience, or an equivalent combination of education & experience.
  • Strong in T-SQL and Python.
  • Expertise with ETL/ELT tools such as SSIS and Azure Data Factory.
  • Knowledge of both on premise and cloud database management systems.
  • Exceled application of data modeling principles and best practices.
  • Knowledge with Big Data concepts such as Multi Parallel Processing and Modern Cloud Data Warehouse Architecture.
  • Excellent communication skills, both verbal and written.
  • Exceptional analytical and problem-solving skills with keen attention to detail.
Job Responsibility
Job Responsibility
  • Build, maintain, and optimize ETL/ELT pipelines for ingesting, transforming, and processing data.
  • Ensure data pipelines are scalable, reliable, and efficient.
  • Design and implement data models, warehouses, and lakehouse architectures.
  • Optimize data storage solutions using relational (SQL) and NoSQL databases.
  • Work with big data technologies like Spark, Hadoop, and Kafka for large-scale data processing.
  • Deploy and manage data solutions on cloud platforms (AWS, Azure, GCP).
  • Improve query performance and storage efficiency.
  • Implement indexing, partitioning, and caching strategies for faster data retrieval.
  • Performs other duties as assigned.
  • Fulltime
Read More
Arrow Right

Sr Data Engineer

As a Senior Data Engineer at Amgen, you will be responsible for managing and opt...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s or Master's degree with 8 - 13 years of experience in Computer Science, IT or related field
  • Hands-on experience in data engineering technologies such as Databricks, PySpark, SparkSQL Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies
  • Proficiency in workflow orchestration, performance tuning on big data processing
  • Strong understanding of AWS services
  • Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures
  • Ability to quickly learn, adapt and apply new technologies
  • Strong problem-solving and analytical skills
  • Excellent communication and teamwork skills
  • Experience with Scaled Agile Framework (SAFe), Agile delivery practices, and DevOps practices
  • Excellent analytical and troubleshooting skills
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable ETL/ELT pipelines to support structured, semi-structured, and unstructured data processing across the Enterprise Data Engineering for Biotech or Pharma functional knowledge of R&D
  • Implement real-time and batch data processing solutions, integrating data from multiple sources into a unified, governed data fabric architecture
  • Optimize big data processing frameworks using Apache Spark, Hadoop, or similar distributed computing technologies to ensure high availability and cost efficiency
  • Work with metadata management and data lineage tracking tools to enable enterprise-wide data discovery and governance
  • Ensure data security, compliance, and role-based access control (RBAC) across data environments
  • Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets
  • Develop CI/CD pipelines for automated data pipeline deployments, version control, and monitoring
  • Implement data virtualization techniques to provide seamless access to data across multiple storage systems
  • Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals
  • Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures
  • Fulltime
Read More
Arrow Right

Sr. Data Engineer

We are seeking a highly skilled Data Engineer with strong backend engineering ex...
Location
Location
United States , Boston
Salary
Salary:
Not provided
solomonpage.com Logo
Solomon Page
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of experience in Data Engineering, Backend Engineering, or related roles
  • Strong hands-on experience with Python development
  • Extensive experience with PostgreSQL and advanced SQL optimization
  • Experience building and maintaining ETL/ELT pipelines and data integration workflows
  • Strong knowledge of data engineering concepts, data modeling, and data architecture
  • Experience working with Parquet file formats and large-scale datasets
  • Hands-on experience with AWS services including EKS, Lambda, and cloud-native architectures
  • Experience developing APIs and microservices using FastAPI or similar frameworks
  • Strong data testing, validation, troubleshooting, and performance tuning skills
  • Excellent organizational skills with the ability to manage timelines, priorities, and status reporting
Job Responsibility
Job Responsibility
  • Design, develop, optimize, and maintain scalable data pipelines and ETL/ELT workflows
  • Refactor and enhance existing data processing frameworks to improve performance and reliability
  • Perform extensive data validation, testing, and quality assurance across data platforms
  • Optimize SQL queries, database structures, and table performance within PostgreSQL environments
  • Implement and support backend services and APIs using Python and FastAPI
  • Develop and maintain data solutions leveraging AWS services including EKS and Lambda
  • Work with Parquet files and large-scale datasets to support analytics and reporting requirements
  • Improve overall data architecture, operational stability, and system performance
  • Maintain detailed task tracking, status reporting, and execution transparency through Excel-based project tracking
  • Proactively identify risks, communicate project status, and manage timelines effectively
What we offer
What we offer
  • medical
  • dental
  • 401(k)
  • direct deposit
  • commuter benefits
Read More
Arrow Right

Sr Data Engineer

Designing, building, maintaining, analyzing, and interpreting data to provide ac...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
amgen.com Logo
Amgen
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s / Master’s degree and 8 to 13 years of Computer Science, IT or related field experience
  • Hands-on experience with big data technologies and platforms, such as Databricks, Apache Spark (PySpark, SparkSQL), workflow orchestration, performance tuning on big data processing
  • Proficiency in data analysis tools (e.g. SQL) and experience with data visualization tools
  • Excellent problem-solving skills and the ability to work with large, complex datasets
  • Strong understanding of data governance frameworks, tools, and best practices
  • Excellent critical-thinking and problem-solving skills
  • Strong communication and collaboration skills
  • Demonstrated awareness of how to function in a team setting
  • Demonstrated presentation skills
Job Responsibility
Job Responsibility
  • Design, develop, and maintain data solutions for data generation, collection, and processing
  • Be a key team member that assists in design and development of the data pipeline
  • Create data pipelines and ensure data quality by implementing ETL processes to migrate and deploy data across systems
  • Contribute to the design, development, and implementation of data pipelines, ETL/ELT processes, and data integration solutions
  • Take ownership of data pipeline projects from inception to deployment, manage scope, timelines, and risks
  • Collaborate with cross-functional teams to understand data requirements and design solutions that meet business needs
  • Develop and maintain data models, data dictionaries, and other documentation to ensure data accuracy and consistency
  • Implement data security and privacy measures to protect sensitive data
  • Leverage cloud platforms (AWS preferred) to build scalable and efficient data solutions
  • Collaborate and communicate effectively with product teams
  • Fulltime
Read More
Arrow Right