CrawlJobs Logo

Level 3 Data Engineer

https://www.randstad.com Logo

Randstad

Location Icon

Location:
India , Bengaluru

Category Icon
Category:
IT - Software Development

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided
Save Job
Save Icon
Apply Position

Job Description:

The purpose of the Level 3 Data Engineer position is to leverage their technical expertise and domain knowledge to design, build, and maintain efficient and robust data solutions. The Data Engineer shall be responsible for the building, testing, and deployment of Data Pipelines on the EDP. The engineer shall ensure that the pipelines are developed and deployed with a secure-by-design approach, delivering robust, thoroughly tested, and maintainable solutions.

Job Responsibility:

  • Build data pipelines using standard patterns with Streamsets, Kestra, dbt, Git
  • Design and implement data storage and processing solutions employing Snowflake
  • Utilize AWS services for cloud-based platform tooling infrastructure including Lambda, ECS, MSK, RDS, EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge
  • Utilize Terraform for AWS and Azure deployments
  • Leverage and integrate APIs for data access and manipulation
  • Write Python scripts for data common processing and automation tasks
  • Leverage Platform APIs and Web Applications to enforce Platform Security
  • Develop with Go, SQL, C#, .net, JavaScript, shell scripts and container platforms like Docker
  • Integrate with timeseries source systems: Honeywell Plant Historian Database, OSI Pi
  • Implement authentication mechanisms including OAuth 2.0, OIDC, Microsoft Entra, Key Pair Authentication, Certificate based authentication, SAML based SSO
  • Create and execute comprehensive test plans
  • Develop unit tests, integration tests, and end-to-end tests
  • Ensure data accuracy and consistency through rigorous testing
  • Leverage automated testing processes
  • Adhere to access processes and procedures to maintain Data Privacy and Security
  • Identify and report potential breaches of Data Information and Systems Processes
  • Monitor and manage platform for optimal performance and uptime
  • Conduct regular maintenance tasks such as updates, patches, and backups
  • Resolve issues or incidents related to the platform
  • Improve platform operations through automation and optimization
  • Implement security best practices throughout pipeline development and deployment
  • Conduct regular security reviews and vulnerability assessments
  • Ensure data encryption, access control, and other security measures are enforced
  • Use credential management platforms like Thycotic Secret Server, AWS Secrets Manager
  • Assist in troubleshooting and resolving intricate technical issues
  • Deliver robust, scalable, and well-documented data pipelines
  • Deliver comprehensive test plans and automation scripts
  • Deliver regular security assessments and compliance reports
  • Deliver technical support and guidance documentation
  • Maintain compliance with security standards, Data Lifecycle Management Process, and Data Privacy standards

Requirements:

  • 10yrs Data Engineering/Data Analyst experience
  • Bachelor’s or Master’s degree in Computer Science, Data Science, Information Technology, or a related field with a focus on data engineering or data analytics
  • Strong proficiency in programming languages such as Python, SQL, Java, or Scala for data processing and analysis
  • Experience with data modeling, ETL processes, data warehousing, data integration, and data pipeline development
  • Proficiency in relational databases (e.g., SQL, PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra)
  • Working knowledge of cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform for data storage and processing
  • Experience with data visualization tools (e.g., Power BI) to create meaningful insights from data
  • Understanding of data quality principles, data governance, and data validation processes
  • Ability to analyze complex data sets, identify trends, patterns, and insights to drive data-driven decision-making
  • Proficiency in troubleshooting data-related issues, identifying root causes, and implementing solutions
  • Familiarity with project management methodologies to contribute effectively to project planning and execution
  • Strong verbal and written communication skills to collaborate with cross-functional teams and communicate technical concepts to non-technical stakeholders
  • Willingness to stay updated with the latest data technologies, tools, and industry trends to enhance data engineering skills
  • Prior experience in data engineering, data analytics, or related roles with a track record of successful data project delivery
  • Ability to make informed, strategic decisions that align technology with business objectives, while balancing short-term and long-term trade-offs
  • Deep understanding of customer needs and how to translate them into effective technical solutions that drive business value
  • Encourages collaboration across teams and stakeholders, breaking down silos and ensuring alignment
  • Strong analytical and problem-solving skills, capable of addressing complex technical challenges and delivering innovative solutions
  • Ability to lead innovation in technology while maintaining an eye on product-market fit and user experience
  • Adaptability in a fast-moving environment, with a mindset focused on delivering high-impact solutions quickly and iteratively
What we offer:
  • Commitment to your ongoing development, including on the job opportunities and formal programs
  • Inclusive parental leave entitlements for both parents
  • Values led culture
  • Flexible work options
  • Generous annual leave, sick leave and casual leave
  • Cultural and religious leave with flexible public holiday opportunities
  • Competitive remuneration package featuring performance based incentives with uncapped Employer Provident Fund

Additional Information:

Job Posted:
May 19, 2026

Expiration:
July 04, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Icon
Randstad - All Job Offers
Job Link Share:
Copy Link Icon
Facebook Icon Twitter Icon LinkedIn Icon
PREMIUM
More languages and countries
+ Unlock 30381 hidden job offers
Languages
English Čeština Deutsch Ελληνικά Español Français +15
Countries
United States United Kingdom India Canada Australia +
See plans
Plans from $2.99 / month

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Level 3 Data Engineer

Test Engineer Level 3

Our customer is seeking a System Administrator to contribute to national securit...
Location
Location
United States , Fort Meade
Salary
Salary:
Not provided
aquila.technology Logo
Aquila Technology
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Eight (8) years’ experience as a TE in programs and contracts of similar scope, type and complexity is required
  • Bachelor’s degree in Math, Science, Engineering, Statistics, Engineering Management or related discipline from an accredited college or university is required
  • Four (4) years of additional TE experience may be substituted for a bachelor’s degree
  • Must have an Active Full Scope Poly to be considered for this position
Job Responsibility
Job Responsibility
  • Analyze system requirements and concept of operations documents, acquisition plans, and system descriptions to develop evaluation and test plans and procedures
  • Prepare for and conduct the data collection and analysis, and report status and results
  • Support the development of Government test and evaluation documentation e.g., Test & Evaluation (T&E) Strategy, Test & Evaluation Master Plan, Event Test Plans, Test Readiness Assessments, and T&E Reports
  • Work with other test organizations to support T&E program integration
  • Provide specialized T&E capabilities tailored to system functions, level, maturity, and performance area
  • Assist with any or all of the Level I tasks as constrained by technical training and previous testing experience
  • Conduct research into and apply knowledge of Government test and evaluation concepts and related engineering, acquisition, and operations activities as needed to support the T&E activities
  • Plan and Execute Performance Analysis
  • Review planned acquisition approaches to support T&E schedule alignment with decision maker needs for T&E support
  • Analyze performance risk areas to focus T&E planning
What we offer
What we offer
  • 401k Retirement Plan
  • Health Care Coverage
  • Health Savings Account
  • Pet Insurance
  • Flexible Spending Account
  • PTO: 0-2 Years – 21 days, 2-8 Years – 26 days, 8-15 Years – 28 days, 16+ Years – 31 days
  • Professional Development Program: reimburses Up to $2k per year for classes, seminars, licensing and certifications
  • Referral Bonuses: Up to $10,000 referral bonus, both you and the person you refer must be employed by for a minimum of 90 days
  • Employee Assistance Program: Offers services to help the well-being of employees and their dependent family members
  • Worldwide Travel Assistance: for travel beyond 100 miles of employees’ residence
  • Fulltime
Read More
Arrow Right

Software Engineer, Data Engineering

Join us in building the future of finance. Our mission is to democratize finance...
Location
Location
Canada , Toronto
Salary
Salary:
124000.00 - 145000.00 CAD / Year
robinhood.com Logo
Robinhood
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of professional experience building end-to-end data pipelines
  • Hands-on software engineering experience, with the ability to write production-level code in Python for user-facing applications, services, or systems (not just data scripting or automation)
  • Expert at building and maintaining large-scale data pipelines using open source frameworks (Spark, Flink, etc)
  • Strong SQL (Presto, Spark SQL, etc) skills
  • Experience solving problems across the data stack (Data Infrastructure, Analytics and Visualization platforms)
  • Expert collaborator with the ability to democratize data through actionable insights and solutions
Job Responsibility
Job Responsibility
  • Help define and build key datasets across all Robinhood product areas. Lead the evolution of these datasets as use cases grow
  • Build scalable data pipelines using Python, Spark and Airflow to move data from different applications into our data lake
  • Partner with upstream engineering teams to enhance data generation patterns
  • Partner with data consumers across Robinhood to understand consumption patterns and design intuitive data models
  • Ideate and contribute to shared data engineering tooling and standards
  • Define and promote data engineering best practices across the company
What we offer
What we offer
  • bonus opportunities
  • equity
  • benefits
  • Fulltime
Read More
Arrow Right

Data Engineer

If you are passionate about big data, want to be part of a great team, and love ...
Location
Location
United States , Fort Worth, Texas
Salary
Salary:
Not provided
koddi.com Logo
Koddi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in CS or other technical, science, or math fields
  • 3+ years of experience as a data or software engineer
  • High level of experience with SQL, with ability to design and optimize objects
  • Experience with Extract, Transform, and Load (ETL)
  • Basic command line knowledge and Git
  • Outstanding work ethic and drive
  • Strong communication and teamwork skills
  • Willingness to learn and utilize emerging technologies
  • Relational Databases: MS SQL Server, Postgres, MySQL
Job Responsibility
Job Responsibility
  • Work within robust data systems and develop custom solutions while consulting with external customers
  • Recommend and implement improvements to data processes and warehouses that improve supportability, usability, and scalability
  • Optimize and refactor existing code
  • Improve the efficiency, scalability, and reliability of applications
Read More
Arrow Right

Data Engineer - II

The Data Engineer will design, develop, and maintain scalable data pipelines and...
Location
Location
India , Pune
Salary
Salary:
Not provided
aticaglobal.com Logo
Atica Global
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in computer science, Engineering, Mathematics, a related field, or equivalent practical experience
  • 3-5 years of experience in data engineering or a similar mid-level role
  • Proficiency in Python and SQL
  • experience with Java is a plus
  • Hands-on experience with AWS, Airbyte, DBT, PostgreSQL, MongoDB, Airflow, and Spark
  • Familiarity with data storage solutions such as PostgreSQL, MongoDB
  • Experience with BigQuery (setup, management and scaling)
  • Strong understanding of data modeling, ETL/ELT processes, and database systems
  • Experience with data extraction, batch processing and data warehousing
  • Excellent problem-solving skills and a keen attention to detail
Job Responsibility
Job Responsibility
  • Design, develop, and maintain scalable data pipelines and ETL/ELT processes using tools like Airflow, Airbyte and PySpark
  • Collaborate with software engineers and analysts to ensure data availability and integrity for various applications
  • Design and implement robust data pipelines to extract, transform, and load (ETL) data from various sources
  • Utilize Airflow for orchestrating complex workflows and managing data pipelines
  • Implement batch processing techniques using Airflow/PySpark to handle large volumes of data efficiently
  • Develop ELT processes to optimize data extraction and transformation within the target data warehouse
  • Leverage AWS services (e.g., S3, RDS, Lambda) for data storage, processing, and orchestration
  • Ensure data security, reliability, and performance when utilizing AWS resources
  • Work closely with developers, analysts, and other stakeholders to understand data requirements and provide the necessary data infrastructure
  • Assist in troubleshooting and optimizing existing data workflows and queries
What we offer
What we offer
  • Competitive salary and benefits package
  • Comprehensive Health Care benefits (best in the country, includes IPD+OPD, covers Employee, Spouse and two children)
  • Growth and advancement opportunities within a rapidly expanding company
  • Fulltime
Read More
Arrow Right

Data Engineer

Barbaricum is seeking a Data Engineer to provide support an emerging capability ...
Location
Location
United States , Omaha
Salary
Salary:
Not provided
barbaricum.com Logo
Barbaricum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Active DoD Top Secret/SCI clearance required
  • 8+ years of demonstrated experience in software engineering
  • Bachelor’s degree in computer science or a related field
  • 8+ years of experience working with AWS big data technologies (S3, EC2) and demonstrate experience in distributed data processing, Data Modeling, ETL Development, and/or Data Warehousing
  • Demonstrated mid-level knowledge of software engineering best practices across the development lifecycle
  • 3+ years of experience using analytical concepts and statistical techniques
  • 8+ years of demonstrated experience across Mathematics, Applied Mathematics, Statistics, Applied Statistics, Machine Learning, Data Science, Operations Research, or Computer Science especially around software engineering and/or designing/implementing machine learning, data mining, advanced analytical algorithms, programming, data science, advanced statistical analysis, artificial intelligence
Job Responsibility
Job Responsibility
  • Design, implement, and operate data management systems for intelligence needs
  • Use Python to automate data workflows
  • Design algorithms databases, and pipelines to access, and optimize data retrieval, storage, use, integration and management by different data regimes and digital systems
  • Work with data users to determine, create, and populate optimal data architectures, structures, and systems
  • and plan, design, and optimize data throughput and query performance
  • Participate in the selection of backend database technologies (e.g. SQL, NoSQL, etc.), its configuration and utilization, and the optimization of the full data pipeline infrastructure to support the actual content, volume, ETL, and periodicity of data to support the intended kinds of queries and analysis to match expected responsiveness
  • Assist and advise the Government with developing, constructing, and maintaining data architectures
  • Research, study, and present technical information, in the form of briefings or written papers, on relevant data engineering methodologies and technologies of interest to or as requested by the Government
  • Align data architecture, acquisition, and processes with intelligence and analytic requirements
  • Prepare data for predictive and prescriptive modeling deploying analytics programs, machine learning and statistical methods to find hidden patterns, discover tasks and processes which can be automated and make recommendations to streamline data processes and visualizations
Read More
Arrow Right

Data Engineer

PulsePoint Data Engineering team plays a key role in our technology company that...
Location
Location
Salary
Salary:
Not provided
pulsepoint.com Logo
PulsePoint
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years of data engineering experience
  • Strong skills in and current experience with SQL and Python
  • Strong recent Spark experience (3+ years)
  • Experience working in on-prem environments
  • Hadoop and Hive experience
  • Proficiency in Linux
  • Strong understanding of RDBMS and query optimization
  • Passion for engineering and computer science around data
  • East Coast U.S. hours 9am-6pm EST
  • Notice period needs to be less than 2 months (or 2 months max)
Job Responsibility
Job Responsibility
  • Design, build, and maintain reliable and scalable enterprise-level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
  • Optimize jobs to utilize Kafka, Hadoop, Presto, Spark, and Kubernetes resources in the most efficient way
  • Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
  • Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
  • Collaborate within a small team with diverse technology backgrounds
  • Provide mentorship and guidance to junior team members
Read More
Arrow Right

Data engineer senior

Within a dynamic, high-level team, you will contribute to both R&D and client pr...
Location
Location
France , Paris
Salary
Salary:
Not provided
artelys.com Logo
Artelys
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Degree from a top engineering school or a high-level university program
  • At least 3 years of experience in designing and developing data-driven solutions with high business impact, particularly in industrial or large-scale environments
  • Excellent command of Python for both application development and data processing, with strong expertise in libraries such as Pandas, Polars, NumPy, and the broader Python Data ecosystem
  • Experience implementing data processing pipelines using tools like Apache Airflow, Databricks, Dask, or flow orchestrators integrated into production environments
  • Contributed to large-scale projects combining data analysis, workflow orchestration, back-end development (REST APIs and/or Messaging), and industrialisation, within a DevOps/DevSecOps-oriented framework
  • Proficient in using Docker for processing encapsulation and deployment
  • Experience with Kubernetes for orchestrating workloads in cloud-native architectures
  • Motivated by practical applications of data in socially valuable sectors such as energy, mobility, or health, and thrives in environments where autonomy, rigour, curiosity, and teamwork are valued
  • Fluency in English and French is required
Job Responsibility
Job Responsibility
  • Design and develop innovative and high-performance software solutions addressing industrial challenges, primarily using the Python language and a microservices architecture
  • Gather user and business needs to design data collection and storage solutions best suited to the presented use cases
  • Develop technical solutions for data collection, cleaning, and processing, then industrialise and automate them
  • Contribute to setting up technical architectures based on Data or even Big Data environments
  • Carry out development work aimed at industrialising and orchestrating computations (statistical and optimisation models) and participate in software testing and qualification
What we offer
What we offer
  • Up to 2 days of remote work per week possible
  • Flexible working hours
  • Offices located in the city center of each city where we are located
  • Fulltime
Read More
Arrow Right

Project Engineer 3

Project Engineer 3 will require the following skillsets: able to perform high-le...
Location
Location
Salary
Salary:
Not provided
atlasps.com Logo
Atlas Professional Services
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of experience as Project Engineer or equivalent
  • Certification current in at least one or more Atlas IT-related technologies
  • Training goal >= 26 hours every 6 months
  • Able to work independently, and consistently follows direction from Project Seniors on Atlas Best Practices
  • Dependable – Able to communicate and has tenacity to work on solution when obstacles present themselves
  • Collaboration with all project team members regarding status, milestones, risks on all assigned projects
  • Excellent organizational and coordination abilities with strong problem-solving skills
  • Excellent client-facing and internal communication skills
  • Excellent written and verbal communication skills
  • Solid organizational skills including attention to detail and multitasking skills.
Job Responsibility
Job Responsibility
  • Performs engineering design evaluations and works to complete projects within budget.
  • Reviews industry policies and procedures to ensure data accuracy, security, and regulatory compliance.
  • Provides technical support for all engineering processes.
  • Takes action to resolve operational problems and minimize delays.
  • Project Engineer 3 can work independently and consistently and follow Atlas Best Practices
  • Dependable – able to communicate and has tenacity to work on solution(s) when obstacles present themselves. Engaging Project escalation team in timely manner.
  • Documentation is essential to all projects as a key to our service delivery success.
  • Collaboration with project team members regarding status, milestones, risks on all projects assigned is required and necessary in order to meet our KPI’s (Key Performance Indicators).
  • Project Engineer 3 has a technical core competency that can be demonstrated and is able to stay current in knowledge of hardware, software, networking, applications, and systems engineering.
  • Company training is provided, both instructor-led and through online professional training tools provided by your supervisor. Learning is part of our culture and required in order to perform job functions, included but not limited to all above duties as a Project Engineer 3.
Read More
Arrow Right