CrawlJobs Logo

Data Engineer - Big Data, Python, Databricks

India, Chennai · Job Posted May 03, 2026
Apply Position
Job Link Share

Job Description

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.

Job Responsibility

  • Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
  • Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
  • Apply fundamental knowledge of programming languages for design specifications
  • Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
  • Serve as advisor or coach to new or lower level analysts
  • Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
  • Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency

Requirements

  • 4 - 6 years of relevant experience in the Financial Service industry
  • Intermediate level experience in Applications Development role
  • Proficient in Big DataTechnologies - Cloudera, Hive, Python, Java/PySpark, Data Bricks
  • Proficient in Data analysis and data modelling
  • Good understanding of ETL concepts
  • Consistently demonstrates clear and concise written and verbal communication
  • Demonstrated problem-solving and decision-making skills
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor's degree/University degree or equivalent experience

Nice to have

Knowledge on relational Database like Oracle, SQL Server is good to have

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Data Engineer - Big Data, Python, Databricks

8 matching positions

Data Engineer (Big Data, Python, Databricks) - Assistant Vice President

The Applications Development Senior Programmer Analyst is an intermediate level ...
Location
Location
India , Chennai, Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5-8 years of relevant handson experience in Big Data technologies like Cloudera, Python, HQL, Java/PySpark
  • Knowledge on Machine Learning, AI would be added advantage
  • Experience in systems analysis, data analysis and programming of software applications
  • Experience in managing and implementing successful projects
  • Working knowledge of consulting/project management techniques/methods
  • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
  • Bachelor’s degree/University degree or equivalent experience
Job Responsibility
Job Responsibility
  • Conduct tasks related to feasibility studies, time and cost estimates, IT planning, risk technology, applications development, model development, and establish and implement new or revised applications systems and programs to meet specific business needs or user areas
  • Monitor and control all phases of development process and analysis, design, construction, testing, and implementation as well as provide user and operational support on applications to business users
  • Utilize in-depth specialty knowledge of applications development to analyze complex problems/issues, provide evaluation of business process, system process, and industry standards, and make evaluative judgement
  • Recommend and develop security measures in post implementation analysis of business usage to ensure successful system design and functionality
  • Consult with users/clients and other technology groups on issues, recommend advanced programming solutions, and install and assist customer exposure systems
  • Ensure essential procedures are followed and help define operating standards and processes
  • Serve as advisor or coach to new or lower level analysts
  • Has the ability to operate with a limited level of direct supervision
  • Can exercise independence of judgement and autonomy
  • Acts as SME to senior stakeholders and /or other team members
  • Fulltime
Read More
Arrow Right

Data Engineer (Big Data, Cloud - AWS, Databricks) - Assistant Vice President

Location
Location
India , Pune
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Scala, Spark/Pyspark is must, Hadoop ( BIG Data ), + AWS,Databricks
  • 8 to 11 years’ experience implementing data-intensive solutions using agile methodologies
  • Experience of relational databases and using SQL for data querying, transformation and manipulation
  • Experience of modelling data for analytical consumers
  • Ability to automate and streamline the build, test and deployment of data pipelines
  • Experience in cloud native technologies and patterns
  • A passion for learning new technologies, and a desire for personal growth, through self-study, formal classes, or on-the-job training
  • Excellent communication and problem-solving skills
  • An inclination to mentor
  • an ability to lead and deliver medium sized components independently
Job Responsibility
Job Responsibility
  • Developing and supporting scalable, extensible, and highly available data solutions
  • Deliver on critical business priorities while ensuring alignment with the wider architectural vision
  • Identify and help address potential risks in the data supply chain
  • Follow and contribute to technical standards
  • Design and develop analytical data models
  • Fulltime
Read More
Arrow Right

Big Data Engineer

We are seeking a talented and experienced Big Data Hadoop Developer to join our ...
Location
Location
India , Chennai
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field
  • 3-4 years of professional experience in Big Data development
  • Proven experience with the Hadoop ecosystem, including HDFS, YARN, Hive, and other related technologies
  • Hands on experience in SQL and shell scripting
  • Strong expertise in Apache Spark for data processing and analysis
  • Proficiency in at least one of the following programming languages: Scala, Python, or PySpark
  • Experience with building and optimizing large-scale data pipelines
  • Familiarity with data warehousing concepts and ETL methodologies
  • Solid understanding of distributed computing principles
  • Excellent problem-solving skills and attention to detail
Job Responsibility
Job Responsibility
  • Design, develop, and maintain robust and scalable ETL processes and data pipelines using Apache Hadoop and Apache Spark
  • Write efficient, clear, and well-documented code primarily in Scala, Python, or PySpark for big data processing
  • Implement data ingestion, transformation, and loading routines from various sources into Hadoop Distributed File System (HDFS) and other big data stores
  • Optimize existing Spark jobs and Hadoop ecosystem components for performance and scalability
  • Collaborate with data architects, data scientists, and other stakeholders to understand data requirements and translate them into technical solutions
  • Ensure data quality, integrity, and security across all big data platforms
  • Participate in code reviews, testing, and deployment of big data applications
  • Troubleshoot and resolve issues in big data environments
  • Stay up-to-date with the latest trends and technologies in the big data ecosystem
  • Fulltime
Read More
Arrow Right

Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology)

Do you love building and pioneering in the technology space? Do you enjoy solvin...
Location
Location
United States , McLean, Virginia; Chicago, Illinois
Salary
Salary:
179400.00 - 225100.00 USD / Year
capitalone.com Logo
Capital One
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s Degree
  • At least 4 years of experience in application development (Internship experience does not apply)
  • At least 2 years of experience in big data technologies
  • At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Job Responsibility
Job Responsibility
  • Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies
  • Work, as a lead developer, with a team of developers with deep experience in data movement, distributed computing, and full stack systems
  • Utilize programming languages like Python, SQL and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Snowflake, Databricks
  • Optimize information system for end-user and downstream application consumers by using sound data design practices
  • Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
  • Collaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowerment
  • Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
What we offer
What we offer
  • Performance based incentive compensation, which may include cash bonus(es) and/or long term incentives (LTI)
  • Comprehensive, competitive, and inclusive set of health, financial and other benefits
  • Fulltime
Read More
Arrow Right

Senior Software Development Engineer, Big Data

The Zillow Group’s Housing Trends Data Engineering team brings innovative analys...
Location
Location
United States
Salary
Salary:
152900.00 - 257100.00 USD / Year
Zillow
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong proficiency in Python programming
  • Hands-on experience with big data ecosystems, architectures and modern data platforms, including AWS cloud data services and Lakehouse/Lakebase platforms
  • Hands-on experience with Databricks is a significant advantage
  • Proven ability to design, build, and orchestrate batch and real-time data pipelines, processing large-scale datasets from diverse sources
  • Practical experience with cutting-edge data engineering technologies such as SparkSQL and Spark Streaming, PostgreSQL, EKS, and Kubernetes
  • Familiarity with Agile/DevOps software development processes and tools such as GitLab and CI/CD
  • A good understanding of data science concepts, fundamental machine learning algorithms/libraries, and development lifecycle is a must-to-have
  • Familiarity with LLM and Agentic AI technologies and engineering patterns is a big advantage
  • Strong ability in performing root cause analysis to address operational issues and identify opportunities for improvement
  • Exceptional communication skills to drive effective customer engagement, collaborate with cross-functional teams, and clearly articulate technical concepts to technical and non-technical stakeholders
Job Responsibility
Job Responsibility
  • Build data systems to link raw data and concrete Housing Trends insights
  • Build and extend our internal forecasting model development framework
  • Develop and maintain scalable data products and deliver pipelines built for speed, accuracy, and consistency that will scale as our stakeholders’ need
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability and performance, etc
  • Collaborate with data scientists to understand their data requirements and build systems/tools that enable efficient model training and experimentation
  • Implement feature stores and ETL pipelines to streamline feature engineering and model development
  • Build and maintain infrastructure for deploying machine learning models into production environments
  • Be a trusted partner for our stakeholders, such as Forecasting, Zestimates, and the ML teams within the AI organization to assist them for data investigation/publication, ML model development and infrastructure support
What we offer
What we offer
  • Eligible for equity awards based on factors such as experience, performance and location
  • Fulltime
Read More
Arrow Right

Senior Software Development Engineer, Big Data

This role will be responsible for developing and supporting data platform infras...
Location
Location
Mexico , Mexico City
Salary
Salary:
Not provided
Zillow
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience in software development with one or more programming languages such as Python, Java, C++, etc.
  • Proficiency in developing and supporting services with cloud infrastructure such as AWS
  • Experience with and ability to debug/troubleshoot container orchestration platforms like Kubernetes
  • Experience with data infrastructure systems like Databricks, Spark, Hive, Airflow, Trino, Kafka, and Flink
  • Experience in setting up and optimizing data pipelines
  • Experience in developing scalable processes for supporting multiple user groups and teams
  • Familiarity and Experience with running on-call operational responsibilities for data platform systems
Job Responsibility
Job Responsibility
  • Developing and supporting data platform infrastructure
  • Collaborating to provide Data Platform Environments that empower internal Zillow Group customers to deliver high-quality data solutions
What we offer
What we offer
  • Competitive base salary and benefits
  • Equity awards based on factors such as experience, performance and location
  • Fulltime
Read More
Arrow Right

Big Data Engineer - AWS

As a full spectrum AWS integrator, we assist hundreds of companies to realize th...
Location
Location
Egypt , Giza
Salary
Salary:
Not provided
rackspace.com Logo
Rackspace
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years design & implementation experience with distributed applications
  • 2+ years of experience in database architectures and data pipeline development
  • Demonstrated knowledge of software development tools and methodologies
  • Presentation skills with a high degree of comfort speaking with executives, IT management, and developers
  • Excellent communication skills with an ability to right level conversations
  • Technical degree required
  • Computer Science or Math background desired
  • Demonstrated ability to adapt to new technologies and learn quickly
  • AWS, GCP And/Or Azure Public Cloud skills and experience
Job Responsibility
Job Responsibility
  • Build complex ETL code
  • Build complex SQL queries using MongoDB, Oracle, SQL Server, MariaDB, MySQL
  • Work on Data and Analytics Tools in the Cloud
  • Develop code using Python, Scala, R languages
  • Work with technologies such as Spark, Databricks, Hadoop, Kafka, etc
  • Build complex Data Engineering workflows
  • Create complex data solutions and build data pipelines
  • Establish credibility and build impactful relationships with our customers to enable them to be cloud advocates
  • Capture and share industry best practices amongst the community
  • Attend and present valuable information at Industry Events
  • Fulltime
Read More
Arrow Right

Senior Software Engineer- Big Data & Java

At PointClickCare our mission is simple: to help providers deliver exceptional c...
Location
Location
United States
Salary
Salary:
158000.00 - 176000.00 CAD / Year
pointclickcare.com Logo
PointClickCare
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience with distributed computing tools like Apache Hudi, Spark, Kafka, Flink, Beam, Trino, DataBricks and other big data technologies
  • Experience with distributed storage systems like ADLS, HDFS, S3, DLT etc.
  • Familiarity with Hadoop, Spark, Databricks or other distributed computing systems
  • Understanding of data partitioning and sharding techniques
  • Knowledge of distributed computing principles and how they apply to large-scale data processing
  • Experience in writing clean code that performs well at scale using languages such as Python, Java etc.
  • Knowledge of relational databases (e.g. Microsoft SQL Server, MySQL)
  • Solid experience writing RESTful API endpoints
  • Absolutely love TDD and have working knowledge of it
  • Proficient in GIT
Job Responsibility
Job Responsibility
  • Identify, prioritize and execute tasks in the software development life cycle
  • Work with business to iterate over software requirements
  • Develop tools and applications by producing clean, efficient code
  • Automate tasks through appropriate tools and scripting
  • Analyze and debug systems
  • Perform validation and verification testing in a test-driven manner
  • Review the work of others, and invite others to review your work
  • Collaborate with internal teams and vendors to fix and improve products
  • Ensure software is up-to-date with latest technologies
  • Work with distributed computing systems like Apache Hudi and Trino for big data processing
What we offer
What we offer
  • Benefits starting from Day 1!
  • Retirement Plan Matching
  • Flexible Paid Time Off
  • Wellness Support Programs and Resources
  • Parental & Caregiver Leaves
  • Fertility & Adoption Support
  • Continuous Development Support Program
  • Employee Assistance Program
  • Allyship and Inclusion Communities
  • Employee Recognition … and more!
  • Fulltime
Read More
Arrow Right