Lead Big Data Developer Job at Wissen (Haveli)

Fullstack Big Data Developer Application Development Technical Lead Analyst Vice President

Discover your future at Citi. Working at Citi is far more than just a job. A car...

Location

Canada , Mississauga

Salary:

120800.00 - 170800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

6+ years of Application development experience
6+ years of experience in full stack development, with a focus on Bigdata and Python/Scala
6+ years experience with big data technologies such as Python, Pyspark, Hadoop, Kafka, etc.
Experience with Core Java/J2EE Application with complete command over OOPs and Design Patterns
Commendable in Data Structures and Algorithms
Worked on Core Application Development of complex size encompassing all areas of Java/J2EE
Thorough knowledge and hands on experience in following technologies Hadoop, Map Reduce Framework, Spark, YARN, Sqoop, Pig , Hue, Unix, Java, Sqoop, Impala, Cassandra on Mesos
Should have implemented or part complex project execution in Big Data Spark eco system, where processing volumes of data thorough understanding of distributed processing and integrated applications
Exposure to ETL and BI tools
Work in an agile environment following through the best practices of agile Scrum

Job Responsibility

Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency
Design, develop, and maintain scalable and robust architecture for the project using Java/Python/Scala and other full stack technologies
Manage big data technologies such as python, pyspark to ensure seamless data integration, storage, and analysis

Fulltime

Lead Big Data Spark Engineer

We are seeking an experienced and highly skilled Big Data Engineer to lead the d...

Location

Canada , Mississauga

Salary:

120800.00 - 170800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

Bachelor's or Master's degree in Computer Science, Engineering, or a related field
8+ years of progressive experience in software development
5+ years focusing on big data technologies
3+ years of experience in a leadership or senior architectural role
Extensive hands-on experience with Scala for big data processing
Demonstrated expertise with Apache Spark (Spark Core, Spark SQL, Spark Streaming)
Strong experience with distributed systems and big data ecosystems (e.g., Hadoop, Kafka, Cassandra, HBase, Delta Lake, Snowflake, Databricks)
Proficiency with cloud platforms (AWS, Azure, GCP) and their big data services
Experience with containerization technologies (Docker, Kubernetes) and CI/CD pipelines
Solid understanding of data warehousing concepts, ETL/ELT processes, and data modeling

Job Responsibility

Lead the architecture, design, and development of high-performance, scalable, and reliable big data processing systems using Scala and Apache Spark
Drive technical vision and strategy for big data initiatives
Evaluate and recommend new technologies and tools
Design, develop, and optimize data pipelines for ingestion, transformation, and storage of massive datasets
Implement robust and efficient data processing jobs using Scala and Spark (batch and streaming)
Ensure data quality, integrity, and security
Promote and enforce best practices in coding, testing, and deployment
Mentor and guide a team of talented big data engineers
Conduct code reviews, provide constructive feedback
Participate in the recruitment and hiring

Fulltime

Lead Java Big Data Engineer Vice President

At Citi, we are at the forefront of financial technology, driven by a belief in ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

10+ years of progressive experience in professional software engineering, with at least 3 years in a technical leadership or architect role
Proven track record of designing and building complex, high-performance, scalable server-side applications using Java
Deep, hands-on experience with the Big Data ecosystem, including mastery of Apache Spark, Hadoop (HDFS), and real-time data streaming with Kafka
Extensive experience with relational databases, data modeling, and data warehousing concepts
Demonstrated experience leading and mentoring technical teams and successfully delivering complex, large-scale data projects from concept to production
Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field

Job Responsibility

Define the end-to-end architectural vision and technical roadmap for migrating from Sybase IQ to a modern Big Data platform, ensuring solutions are scalable, resilient, and secure
Lead the design, development, and deployment of robust, large-scale data processing pipelines using technologies like Apache Spark, Kafka, and distributed data stores
Develop and execute a comprehensive, phased strategy for migrating petabytes of historical and transactional data from Sybase IQ, ensuring data integrity, minimal downtime, and zero business disruption
Oversee the design and development of Java-based microservices that interact with the new data platform, ensuring seamless integration with the broader Oasys application ecosystem
Lead, inspire, and mentor a high-performing team of Java and Big Data engineers. Foster a culture of engineering excellence, innovation, and accountability
Partner with global business leaders, product owners, and other senior technology managers to define requirements, manage expectations, and deliver solutions that drive significant business value
Remain deeply technical and contribute to coding, design, and architectural decisions, leading by example

Fulltime

Pyspark Big Data Senior Developer - Vice President

We are building an A-team of highly skilled and autonomous engineers, and we are...

Location

Canada , Mississauga

Salary:

120800.00 - 170800.00 USD / Year

Citi

Expiration Date

Until further notice

Requirements

6+ years of extensive, hands-on experience as a Senior Big Data Developer, with a strong emphasis on PySpark and the Apache Spark ecosystem, operating as a player/coach
Expert proficiency in Python, with a proven track record of developing robust, scalable, and high-performance PySpark applications for large-scale data processing
Deep understanding and extensive hands-on experience with Apache Spark (Spark Core, Spark SQL, Spark Streaming) and its ecosystem
Experience with distributed computing frameworks such as Hadoop (HDFS, YARN)
Expert proficiency in SQL and extensive experience with data warehousing concepts and technologies (e.g., Hive, Snowflake, Redshift, Databricks SQL)
Proven experience with various data storage formats (e.g., Parquet, ORC, Avro) and data lake solutions (e.g., Delta Lake, Iceberg)
Experience with NoSQL databases (e.g., MongoDB, Cassandra, HBase) is a significant plus
Strong experience with Apache Kafka for building real-time data pipelines and event-driven architectures
Demonstrated experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift, Azure Databricks/Data Factory/Synapse, GCP Dataflow/Dataproc/BigQuery) is highly desirable
Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is a mandatory requirement

Job Responsibility

Operate end-to-end in the design, development, and implementation of robust big data solutions, ensuring optimal performance, scalability, data quality, and security
Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for big data processing and analytical solutions
Act as a player/coach within the team, mentoring junior members and leading by example in the development of efficient and innovative big data architectures
Design, develop, and optimize large-scale data pipelines using PySpark for data ingestion, transformation, and aggregation, always with an eye towards efficiency and domain relevance
Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka
Design and implement sophisticated data warehousing solutions and dimensional models for efficient data storage and retrieval, ensuring alignment with business needs
Work with various distributed data storage technologies, including distributed file systems (e.g., HDFS, S3) and NoSQL databases (e.g., MongoDB, Cassandra), selecting the right tool for the right problem
Implement efficient data processing and storage strategies to optimize the performance and scalability of big data applications, with a strong focus on the 'why' behind the technology choices
Champion best practices in software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines
Demonstrate high autonomy and agency in driving projects forward, making informed decisions, and proactively identifying areas for improvement

Fulltime

Big Data / PySpark Engineering Lead - Vice President

The Applications Development Technology Lead Analyst is a senior level position ...

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering
Experience in Data Engineering, focused on Big Data ecosystems
Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development
Programming Expert level and hand on experience in Python
Familiarity with data formats like Avro, Parquet, CSV, JSON
Hands-on experience in writing SQL queries
Highly experienced with Unix based operating systems and shell scripting
Experience with source code management tools such as Bitbucket, Git etc
Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase)
Experience working with query engines like Trino, Presto, Starburst

Job Responsibility

Design and implement scalable, fault-tolerant batch and real-time data processing pipelines
Develop robust data models and schema designs optimized for both performance and storage efficiency
Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack
Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment
ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino
Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results
Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg)
Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools

Fulltime

Lead Technology Analyst - Big Data

Are you passionate about IT, experienced in software development techniques and ...

Location

India , Bangalore

Salary:

Not provided

Airbus

Expiration Date

Until further notice

Requirements

Engineering graduate/post-graduate
7 - 9 years of experience in developing, delivering AWS based solutions
Full stack developer with capabilities in Backend Java and Frontend technologies
Have maintained DevOps pipelines for applications and have worked on Fast APIs
Knowledge on some of the Spark & Spark-SQL with Python
Experience with writing Javascript, HTML, CSS
Knowledge on some of the following products/tools: Git, Confluence, VersionOne etc
Proven track record working in Agile Scrum and/or Kanban projects
Knowledge of continuous integration frameworks and some of the following products/tools: VersionOne, Klaxoon, etc.
Willing to work in CET / IST timezone based on project need

Job Responsibility

Designing solutions to meet functional and non-functional requirements at scale for business needs
Involvement in the full delivery lifecycle – responsible for designing, implementing, testing, documenting, supporting and operating Foundry based applications
Implementing new tools and enhancing the existing tools & data products built on top of the foundry platform
Refactor and debug pipelines
Participate in knowledge sharing activities to further enhance the knowledge base for the project
This job requires the constant awareness of the compliance risks we face in day-to-day responsibilities
Continuous commitment to act with integrity with each other, with your communities, business partners and suppliers is the foundation of your success and sustainable growth
The commitment to integrity is supported by your adherence to all internal policies and procedures that govern business activities
Compliance with these policies will also protect Airbus reputation and brand, some of our most strategic and important assets

Fulltime

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location

India , Pune

Salary:

Not provided

Citi

Expiration Date

Until further notice

Requirements

Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
Experience of relational databases and using SQL for data querying, transformation and manipulation
Experience of modelling data for analytical consumers
Ability to automate and streamline the build, test and deployment of data pipelines
Experience in cloud native technologies and patterns
A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
Excellent communication and problem-solving skills
An inclination to mentor
an ability to lead and deliver medium sized components independently

Job Responsibility

Developing and supporting scalable, extensible, and highly available data solutions
Deliver on critical business priorities while ensuring alignment with the wider architectural vision
Identify and help address potential risks in the data supply chain
Follow and contribute to technical standards
Design and develop analytical data models

Fulltime

Data Scientist (Big Data) III – Contractor

The Senior Data Scientist (Big Data) will support large-scale data science initi...

Location

United States , Philadelphia

Salary:

Not provided

Robert Half

Expiration Date

Until further notice

Requirements

Strong hands-on experience with PySpark, Python, and R for data analysis and machine learning
Experience working in AWS cloud environments
Proficiency with Databricks for large-scale data processing and analytics
Experience building and supporting large-scale data pipelines and ETL workflows
Demonstrated experience applying statistical and modeling techniques, including: Hypothesis testing, Supervised and unsupervised learning, Forecasting and regression, Dimensionality reduction and clustering
Experience working with relational databases, SQL, and large datasets
Ability to gather, interpret, and translate business requirements into technical solutions
Strong communication skills with the ability to present complex concepts to diverse audiences
Bachelor's degree in Computer Science, Mathematics, Statistics, Engineering, or a related quantitative field, or equivalent practical experience
Typically 5+ years of relevant professional experience in data science, analytics, or related roles

Job Responsibility

Lead complex, cross-functional data science initiatives delivering solutions across multiple technologies and platforms
Design, develop, and deploy data mining, statistical, machine learning, and graph-based algorithms for large-scale data sets
Partner with data engineering teams to ensure proper implementation, performance, and operational use of analytical solutions
Review and assess data science programs and models at an enterprise level to evaluate suitability, performance, and scalability
Build and maintain scalable big-data analytics solutions supporting accurate targeting, forecasting, and advanced insights
Develop and support end-to-end machine learning pipelines, including data preparation, training, testing, validation, and deployment
Establish performance metrics, monitoring, and evaluation procedures for models in production
Translate complex analytical findings into clear, actionable insights for technical and non-technical stakeholders
Provide mentorship and technical guidance to junior team members
Contribute to data strategy, methodology selection, and continuous improvement of analytics capabilities

What we offer

medical
vision
dental
life and disability insurance
401(k) plan

Fulltime

Select Country

Lead Big Data Developer

Job Description

Job Responsibility

Requirements

Nice to have

Looking for more opportunities?