CrawlJobs Logo

Big Data ETL Developer

realign-llc.com Logo

Realign

Location Icon

Location:
Canada , Mississauga

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

130000.00 USD / Year

Job Description:

We are seeking a skilled Big Data ETL Developer to design, develop, and maintain large-scale data pipelines and ETL processes. The candidate will work with big data technologies to process, transform, and manage large volumes of structured and unstructured data.

Job Responsibility:

  • Design and develop ETL pipelines for big data processing
  • Work with big data technologies such as Hadoop, Spark, and Hive
  • Extract, transform, and load data from multiple sources into data platforms
  • Optimize data processing workflows and ETL jobs for performance and scalability
  • Collaborate with data engineers, analysts, and business teams to support data requirements
  • Ensure data quality, integrity, and governance
  • Troubleshoot ETL failures and performance issues
  • Document data pipelines and technical processes

Requirements:

  • Strong experience in Big Data and ETL development
  • Proficiency in SQL, Python, or Scala
  • Hands-on experience with Hadoop ecosystem tools (Spark, Hive, HDFS)
  • Experience with data pipeline orchestration tools
  • Knowledge of data warehousing and big data architecture
  • Strong analytical and problem-solving skills

Nice to have:

  • Experience with cloud data platforms such as AWS, Azure, or GCP
  • Familiarity with Kafka or real-time data streaming technologies
  • Experience with CI/CD pipelines and data engineering best practices

Additional Information:

Job Posted:
March 21, 2026

Employment Type:
Fulltime
Work Type:
Hybrid work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Big Data ETL Developer

Senior Python Big Data Developer

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7 - 12 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
  • Strong expertise in Big Data technologies (Spark, Hadoop, Hive, Impala, Kafka, Scala, Cloudera)
  • Design, develop, and maintain robust and scalable data pipelines using Python, SQL, PySpark, and streaming technologies like Kafka
  • Strong SQL and NoSQL experience (Oracle, MongoDB, PostgreSQL) for data extraction, reconciliation, and transformation
  • Proficiency in Python and Shell scripting for data processing and automation
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Sr. ETL Developers

We are looking for talented individuals to become part of our growing business a...
Location
Location
United States , Scottsdale; Naperville
Salary
Salary:
Not provided
thinkartha.com Logo
Artha Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 6 years of experience in IT Industry
  • At least 6 years of technical experience with databases or data warehousing
  • Experience in Oracle DB, SQL Server, Mysql, No SQL such as Mongo DB with strong skills on SQL and PL/SQL and Java
  • BS/BA in Computer Science or related discipline required
  • Candidates with Talend certifications & experience will be given a strong preference
  • Strong understanding of relational database concepts & technology – data modeling (dimensional/snowflake), SQL, query optimization, data warehouse architecture
  • Experience with setup, configuration, design best practices, performance optimization, team based development approaches with a leading Talend/BI tool
  • Well organized and able to manage multiple projects in a fast paced demanding environment
  • Attention to detail and quality
  • excellent problem solving and communication skills
Read More
Arrow Right

ETL Developer

We are looking for talented individuals to become part of our growing business a...
Location
Location
United States , Scottsdale; Naperville
Salary
Salary:
Not provided
thinkartha.com Logo
Artha Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Overall years of experience: Varies based on the role. A minimum experience of 3 years
  • At least 3 years of technical experience with databases, data warehousing, and BI tools
  • BS/BA in Computer Science or related discipline required
  • Experience with reporting and analytics tools is an advantage
  • Experience in Oracle DB, SQL Server, Mysql, No SQL such as Mongo DB with strong skills on SQL and PL/SQL and Java
  • Candidates with Talend certifications & experience will be given a strong preference
  • Strong understanding of relational database concepts & technology – data modeling (dimensional/snowflake), SQL, query optimization, data warehouse architecture
  • Experience with setup, configuration, design best practices, performance optimization, team-based development approaches with a leading Talend/BI tool
  • Well organized and able to manage multiple projects in a fast paced demanding environment
  • Attention to detail and quality
Read More
Arrow Right

Senior Big Data Engineer

The Big Data Engineer is a senior level position responsible for establishing an...
Location
Location
Canada , Mississauga
Salary
Salary:
94300.00 - 141500.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ Years of Experience in Big Data Engineering (PySpark)
  • Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from multiple sources
  • Big Data Infrastructure: Develop and manage large-scale data processing systems using frameworks like Apache Spark, Hadoop, and Kafka
  • Proficiency in programming languages like Python, or Scala
  • Strong expertise in data processing frameworks such as Apache Spark, Hadoop
  • Expertise in Data Lakehouse technologies (Apache Iceberg, Apache Hudi, Trino)
  • Experience with cloud data platforms like AWS (Glue, EMR, Redshift), Azure (Synapse), or GCP (BigQuery)
  • Expertise in SQL and database technologies (e.g., Oracle, PostgreSQL, etc.)
  • Experience with data orchestration tools like Apache Airflow or Prefect
  • Familiarity with containerization (Docker, Kubernetes) is a plus
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Appropriately assess risk when business decisions are made, demonstrating consideration for the firm's reputation and safeguarding Citigroup, its clients and assets
  • Fulltime
Read More
Arrow Right

Big Data / Scala / Python Engineering Lead

The Applications Development Technology Lead Analyst is a senior level position ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least two years (Over all 10+ hands on Data Engineering experience) of experience building and leading highly complex, technical data engineering teams
  • Lead data engineering team, from sourcing to closing
  • Drive strategic vision for the team and product
  • Experience managing an data focused product, ML platform
  • Hands on experience relevant experience in design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scala
  • Experience managing, hiring and coaching software engineering teams
  • Experience with large-scale distributed web services and the processes around testing, monitoring, and SLAs to ensure high product quality
  • 7 to 10+ years of hands-on experience in big data development, focusing on Apache Spark, Scala, and distributed systems
  • Proficiency in Functional Programming: High proficiency in Scala-based functional programming for developing robust and efficient data processing pipelines
  • Proficiency in Big Data Technologies: Strong experience with Apache Spark, Hadoop ecosystem tools such as Hive, HDFS, and YARN
Job Responsibility
Job Responsibility
  • Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
  • Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
  • Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
  • Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
  • Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
  • Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
  • Fulltime
Read More
Arrow Right

Big Data/Java Application Developer

The Big Data/Java Application Developer is an intermediate level position respon...
Location
Location
Canada , Mississauga
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Hands on relevant experience in Angular, HTML, CSS Java, Spring boot, Oracle, NoSQL OR Design, develop, and optimize scalable distributed data processing pipelines using Apache Spark and Scala.
  • Proficiency in Functional Programming: High proficiency in Scala-based functional programming for developing robust and efficient data processing pipelines.
  • Proficiency in Big Data Technologies: Strong experience with Apache Spark, Hadoop ecosystem tools such as Hive, HDFS, and YARN.
  • Programming and Scripting: Advanced knowledge of Scala and a good understanding of Python for data engineering tasks.
  • Data Modeling and ETL Processes: Solid understanding of data modeling principles and ETL processes in big data environments.
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision.
  • Can exercise independence of judgement and autonomy.
  • Experience managing an data focused product, ML platform and or UI/UX
  • Fulltime
Read More
Arrow Right

Big Data Engineer

We are looking for a Big Data Engineer that will work on the collecting, storing...
Location
Location
United States , St. Louis
Salary
Salary:
Not provided
protocolinfotech.com Logo
Protocol Infotech
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Computer Applications, Science, Engineering, Technology
  • 2-4 Years experience
  • Proficiency with Hadoop v2, MapReduce, HDFS
  • Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
  • Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
  • Experience with Spark
  • Experience with integration of data from multiple data sources
  • Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
  • Knowledge of various ETL techniques and frameworks, such as Flume
  • Experience with various messaging systems, such as Kafka or RabbitMQ
Job Responsibility
Job Responsibility
  • Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
  • Implementing ETL process
  • Monitoring performance and advising any necessary infrastructure changes
  • Defining data retention policies
  • Ability to solve any ongoing issues with operating the cluster
  • Guides the development team in overall application technology design activities
What we offer
What we offer
  • Employee referral program
  • Referral fee of $1,000 will be paid if referred candidate is hired
  • Fulltime
Read More
Arrow Right

Product Control Tech ETL Developer, AVP

This role is part of Global FX Exposure Management Tech team which supports vari...
Location
Location
United Kingdom , Belfast
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in ETL development using AB Initio preferably in Financial/Investment banking domain
  • Strong knowledge of ETL processes, data warehousing, and data modelling concepts
  • Hands-on experience with AB Initio GDE, EME, Co>Operating System, Conduct>It
  • Work with databases (SQL & NoSQL) such as MongoDB
  • Experience working with big data technologies (Hadoop, Spark, Hive) is a plus
  • Strong knowledge of shell scripting and Unix/Linux environments
  • Experience with performance tuning and optimization of ETL jobs
  • Excellent problem-solving, analytical, and communication skills
Job Responsibility
Job Responsibility
  • Understanding Business Requirements and Functional Requirements provided by Business Analysts and working as part of the development team to produce Technical Design Documents and deliver solutions on those requirements
  • Work closely with Cross Functional Teams e.g. Business Analysis, Product Assurance, Platforms and Infrastructure, Business Office, Controls and Production Support
  • Prepare handover documents, perform unit testing, SIT and assist with UAT and Production Implementation as required
  • Identify and proactively resolve issues that could impact system performance, reliability, and usability
  • Write efficient, maintainable, and reusable code following best practices
  • Perform code reviews, debugging, and unit testing to ensure high-quality software delivery
  • Collaborate with DevOps teams to implement CI/CD pipelines using Jenkins, Docker, and Kubernetes
  • Optimize application performance and ensure security best practices are followed
  • Mentor junior developers and participate in agile development processes (Scrum/Kanban)
What we offer
What we offer
  • Generous holiday allowance starting at 27 days plus bank holidays
  • increasing with tenure
  • A discretional annual performance related bonus
  • Private medical insurance packages to suit your personal circumstances
  • Employee Assistance Program
  • Pension Plan
  • Paid Parental Leave
  • Special discounts for employees, family, and friends
  • Access to an array of learning and development resources
  • Fulltime
Read More
Arrow Right