CrawlJobs Logo

Full Stack Data Engineer

India, Pune · Job Posted May 27, 2026
Apply Position
Job Link Share

Job Description

We are building an A-team of highly skilled, autonomous, and AI-first engineers, and we are looking for an ambitious Full Stack Data Engineer to join our focused squads in Pune. This role is designed for a hands-on engineer who is passionate about leveraging data, proficient in building end-to-end data solutions, and deeply committed to using AI tools to maximize productivity. The ideal candidate will be instrumental in designing, developing, and optimizing robust data pipelines, from ingestion to consumption, using Python, PySpark, and other big data technologies. We seek an individual with strong domain understanding who can contribute to our AI-first culture and help shape the future of our data platforms.

Job Responsibility

  • Operate end-to-end in the design, development, and implementation of full-stack data solutions, ensuring optimal performance, scalability, data quality, and security across the data lifecycle.
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering high communication and low coordination overhead, to translate complex business requirements into technical specifications for data engineering solutions.
  • Develop, maintain, and optimize data ingestion, processing, and transformation pipelines using Python and PySpark for large-scale datasets.
  • Implement data storage solutions using big data technologies such as Hive, distributed file systems (e.g., HDFS, S3), and potentially NoSQL databases.
  • Design and implement data models and schemas optimized for analytics and reporting, ensuring data integrity and accessibility.
  • Work with data consumers (e.g., analysts, data scientists) to understand their needs and provide efficient access to processed data, potentially involving reporting tools like Tableau.
  • Implement and manage real-time data streaming and event-driven architectures using technologies like Apache Kafka.
  • Champion best practices in data engineering and software development, including rigorous code reviews, implementing comprehensive testing, and supporting continuous integration and continuous deployment (CI/CD) pipelines.
  • Demonstrate high autonomy and agency in driving data projects forward, making informed technical decisions, and proactively identifying areas for data quality and efficiency improvements.
  • Proactively leverage and contribute to the development of AI-powered development tools, including internal Citi AI tools like Copilot, Claude Code, Codex, and Antigravity, to significantly enhance productivity, code quality, and accelerate development cycles.
  • Participate in technical discussions and contribute to the evolution of our big data technology stack, always seeking innovative approaches to data challenges.
  • Troubleshoot and resolve complex technical issues within data pipelines and big data environments, demonstrating strong analytical and problem-solving skills.

Requirements

  • Experience: 4-5 years of hands-on experience as a Data Engineer, with a strong focus on building end-to-end data solutions and big data technologies.
  • Expert proficiency in Python, with proven experience in developing scalable data processing applications.
  • Strong understanding and hands-on experience with Apache Spark, particularly PySpark, for large-scale data processing.
  • Solid experience with Hive for data warehousing and querying large datasets.
  • Familiarity with distributed computing fundamentals and components like HDFS.
  • Proficiency in SQL and experience with data warehousing concepts.
  • Experience with data storage formats (e.g., Parquet, ORC, Avro) and cloud-based data lake solutions (e.g., S3).
  • Experience with Apache Kafka for building real-time data pipelines and event-driven architectures.
  • Proven effectiveness with AI coding tools (e.g., Claude Code, Codex, Antigravity) is expected
  • a strong willingness to adopt and maximize their usage is essential.
  • An "AI-first thinker" mindset, demonstrating how to leverage and integrate AI tools into the development workflow for continuous improvement.
  • Strong ability to articulate the functional domain being worked in, understanding the business context, and explaining the "why" behind the technical data solutions.
  • Strong understanding of data structures, algorithms, and performance optimization techniques for large-scale data processing.
  • Experience with RESTful API design and development for data ingestion or exposure points.
  • Expert proficiency with version control systems, especially Git.
  • Exceptional problem-solving, analytical, and debugging skills in complex, distributed data environments.
  • Superior communication and interpersonal skills, with the ability to work effectively and autonomously within small, high-performing teams, and to collaborate with various stakeholders.
  • Demonstrated high autonomy and agency in tackling complex challenges and delivering impactful data solutions.
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field is required. Equivalent practical experience with a demonstrable track record of excellence will also be considered.

Nice to have

  • Knowledge of data visualization tools like Tableau is beneficial but not mandatory.
  • Familiarity with containerization technologies (e.g., Docker, Kubernetes) for deploying data applications is a plus.

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Full Stack Data Engineer

8 matching positions

Full Stack Data Engineer

We are seeking a highly skilled Full Stack Data Engineer who thrives in building...
Location
Location
United States , Charlotte
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong experience building data pipelines (ELT/ETL) in modern environments
  • Hands-on experience with Snowflake
  • Advanced Python and SQL skills
  • Experience designing data models and warehouse schemas
  • Familiarity with CI/CD and DevOps practices
  • Ability to work independently and own solutions end-to-end
Job Responsibility
Job Responsibility
  • Design, build, and maintain scalable ELT pipelines and workflows
  • Develop and optimize data models and warehouse structures in Snowflake
  • Build full stack data applications and backend services
  • Write clean, efficient Python and SQL code
  • Develop reusable data frameworks and components
  • Implement automated testing for data quality and reliability
  • Build and maintain CI/CD pipelines (GitHub-based)
  • Create reporting and visualization solutions (Power BI or similar)
  • Monitor production systems and troubleshoot data issues proactively
What we offer
What we offer
  • Medical, vision, dental, and life and disability insurance
  • Company 401(k) plan
Read More
Arrow Right

Full-Stack Data Engineer – Data & ML Automation (Databricks)

We are seeking a Fullstack Data Engineer who can operate at the intersection of ...
Location
Location
India , Pune
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience with CLI tools, scripts, and utilities for automating data platform workflows
  • Experience with Databricks APIs, Terraform, Databricks SDK
  • Experience designing integration tests, end-to-end pipeline tests, validation frameworks for Databricks ETL/ELT pipelines and ML inference workflows
  • Experience building internal applications using React, Streamlit, or similar frameworks
  • Experience with spec-driven development, coding agents and automation patterns, CI/CD workflows for data/ML systems
Job Responsibility
Job Responsibility
  • Develop CLI tools, scripts, and utilities to automate repetitive workflows across the data platform
  • Automate Databricks workflows, job deployments, environment provisioning, and MLOps operations using Databricks APIs, Terraform, Databricks SDK
  • Design and implement integration tests, end-to-end pipeline tests, validation frameworks for Databricks ETL/ELT pipelines and ML inference workflows
  • Improve reliability, observability, and overall engineering productivity across the data & ML team
  • Build quick internal applications using React, Streamlit, or similar frameworks to visualize data flows, provide model inference demos, enable operational or configuration controls
  • Develop internal productivity and monitoring dashboards
  • Apply best practices around spec-driven development, coding agents and automation patterns, CI/CD workflows for data/ML systems
  • Fulltime
Read More
Arrow Right

Senior / Staff Full-Stack Data Engineer – Databricks

At Codvo, we are committed to building scalable, future-ready data platforms tha...
Location
Location
India , Pune
Salary
Salary:
Not provided
Codvo AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong hands-on experience with Databricks, Apache Spark, and Delta Lake
  • Proven experience building and operating production-grade data pipelines
  • Experience operationalizing machine learning models and inference pipelines
  • Strong understanding of data reliability, observability, and monitoring practices
  • Experience with CI/CD, DevOps, and MLOps workflows
  • Experience working with cloud platforms (AWS or Azure)
  • Familiarity with Unity Catalog and enterprise data governance concepts
  • Experience with spec-driven development and coding agents
Job Responsibility
Job Responsibility
  • Design, build, and maintain ETL/ELT pipelines on Databricks using Spark, Delta Lake, and Databricks Workflows
  • Build and operate batch and real-time data pipelines for ingestion, transformation, and orchestration
  • Operationalize machine learning inference pipelines authored by data scientists (batch and real-time)
  • Ensure consistency between model training and inference environments
  • Implement data quality checks, validation rules, monitoring, alerting, and automated recovery
  • Collaborate with data scientists to productionize models and optimize inference performance and cost
  • Implement CI/CD, DevOps, and MLOps best practices for data pipelines and ML workflows
  • Optimize compute, storage, and job configurations for performance and cost efficiency
  • Implement and manage enterprise data governance using Unity Catalog (schemas, lineage, ownership, documentation)
  • Work with Databricks infrastructure and platform configurations
  • Fulltime
Read More
Arrow Right

Lead Python Full Stack Data Engineer

We are assembling an A-team of highly skilled, autonomous, and visionary enginee...
Location
Location
Canada , Mississauga
Salary
Salary:
120800.00 - 170800.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 6+ years of progressive, hands-on experience as a Senior/Lead Data Engineer
  • Expert-level proficiency in Python
  • Deep expertise in developing highly optimized, scalable, and production-grade PySpark applications
  • Deep architectural understanding and extensive hands-on experience with the entire Apache Spark ecosystem (Spark Core, Spark SQL, Spark Streaming, Spark MLlib)
  • Advanced proficiency with Hive for enterprise data warehousing
  • Expert knowledge of distributed computing fundamentals, HDFS, and other components of the Hadoop ecosystem
  • Master-level proficiency in SQL, complex query optimization, and advanced data warehousing concepts
  • Extensive experience with various data storage formats (e.g., Parquet, ORC, Avro) and leading data lake solutions (e.g., Delta Lake, Iceberg)
  • Proven experience with enterprise-grade NoSQL databases (e.g., Cassandra, MongoDB, HBase)
  • Expert-level experience with Apache Kafka
Job Responsibility
Job Responsibility
  • Lead and Architect end-to-end data solutions
  • Drive Strategic Initiatives within small, co-located squads
  • Act as a Player/Coach
  • Design, Develop, and Optimize highly efficient and resilient data ingestion, processing, and transformation pipelines using advanced Python and PySpark techniques
  • Architect and Implement sophisticated data storage solutions leveraging a diverse set of big data technologies
  • Champion Data Modeling and Governance
  • Strategically Engage with data consumers, data scientists, and business stakeholders
  • Lead the Implementation of real-time data streaming and complex event-driven architectures
  • Enforce and Evolve Best Practices in data engineering and software development
  • Exhibit High Autonomy and Agency
  • Fulltime
Read More
Arrow Right

Python Full Stack Data Engineer - Assistant Vice President

We are assembling an A-team of highly skilled, autonomous, and AI-first engineer...
Location
Location
Canada , Mississauga
Salary
Salary:
94300.00 - 141500.00 USD / Year
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Experience: 4+ years of progressive, hands-on experience as a Data Engineer, with a proven track record of delivering complex, large-scale data solutions
  • Expert-level proficiency in Python, with deep expertise in developing highly optimized, scalable, and production-grade PySpark applications for mission-critical data processing
  • Deep understanding and extensive hands-on experience with the entire Apache Spark ecosystem (Spark Core, Spark SQL, Spark Streaming)
  • Advanced proficiency with Hive for enterprise data warehousing, including optimization techniques for large and complex queries
  • Expert knowledge of distributed computing fundamentals, HDFS, and other components of the Hadoop ecosystem
  • Proficiency in SQL, complex query optimization, and advanced data warehousing concepts (e.g., dimensional modeling, data vault, data lakes)
  • Extensive experience with various data storage formats (e.g., Parquet, ORC, Avro) and leading data lake solutions (e.g., Delta Lake, Iceberg)
  • Proven experience with enterprise-grade NoSQL databases (e.g., Cassandra, MongoDB, HBase) and understanding of their architectural trade-offs
  • Expert-level experience with Apache Kafka, including design and implementation of high-throughput, low-latency real-time data pipelines and event-driven architectures
  • Extensive experience with big data services on major cloud platforms (e.g., AWS EMR/Glue/Redshift/Kinesis, Azure Databricks/Data Factory/Synapse/Event Hubs, GCP Dataflow/Dataproc/BigQuery/Pub/Sub), including cloud-native architectural patterns
Job Responsibility
Job Responsibility
  • Operate end-to-end in the design, development, and implementation of full-stack data solutions, ensuring optimal performance, scalability, data quality, security, and compliance across the data lifecycle
  • Collaborate closely within small, co-located squads (4-7 person teams), fostering an environment of high communication and minimal coordination overhead, to deliver impactful data products
  • Develop, maintain, and optimize highly efficient and resilient data ingestion, processing, and transformation pipelines using advanced Python and PySpark techniques for large-scale datasets
  • Implement sophisticated data storage solutions leveraging a diverse set of big data technologies including Hive, distributed file systems (e.g., HDFS, S3), and enterprise-grade NoSQL databases (e.g., Cassandra, MongoDB)
  • Design and implement scalable data models and schemas that support advanced analytics, machine learning, and critical reporting needs, ensuring data integrity, accessibility, and discoverability
  • Engage effectively with data consumers, data scientists, and business stakeholders to deeply understand their requirements, translating them into robust data solutions and providing expert guidance on data utilization and interpretation
  • Implement real-time data streaming and complex event-driven architectures using technologies like Apache Kafka, ensuring low-latency data availability for critical business functions
  • Adhere to and contribute to best practices in data engineering and software development, participating in rigorous code reviews, implementing comprehensive automated testing strategies, and supporting robust CI/CD pipelines within a DevOps culture
  • Exhibit High Autonomy and Agency, taking ownership of technical challenges, making well-reasoned architectural decisions, and proactively identifying and implementing continuous improvements across the data landscape
  • Innovate with AI-Powered Development, actively leveraging, integrating, and contributing to AI coding tools (e.g., internal Citi AI tools, Copilot, Claude Code, Codex, Antigravity) to significantly enhance productivity, code quality, and development velocity, and inspiring others to do the same
  • Fulltime
Read More
Arrow Right

Senior Full-Stack Data Engineer

We are Metyis, a forward-thinking, global company that develops and delivers sol...
Location
Location
Portugal , Porto
Salary
Salary:
Not provided
metyis.com Logo
Metyis
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Academic degree in computer science, software engineering, machine learning engineering, or related field
  • At least 5 years of professional software development experience using languages such as Python, Spark, SQL, Scala, or Rust
  • Strong knowledge of cloud technology such as Microsoft Azure, or GCP
  • Strong knowledge of all-in-one analytics platforms such as Databricks, or Microsoft Fabric
  • Strong knowledge of data pipeline orchestration platforms such as Azure Data Factory, or Control-M
  • Experience in using data warehouse and data lake solutions such das SAP BW and Azure Data Lake
  • Experience with test driven development as well as with testing & data quality frameworks such as pytest, or great expectations in Python
  • Experience with SQL & No-SQL databases such as Azure SQL, or MongoDB
  • Experience with RESTful APIs and microservice architecture
  • Experience with GIT version control
Job Responsibility
Job Responsibility
  • Work on a company-wide batch & streaming data processing framework that can be viewed as a software application as part of an IT-Analytics team of data engineers
  • Design, develop, test, and deploy software & data pipelines using various batch & streaming technologies, and automate & monitor CI/CD/CT pipelines for an Azure based Data Platform
  • Optimize existing code for performance, reliability, and scalability
  • Debug and troubleshoot issues and provide technical support as needed
  • Follow best practices and standards for coding, documentation, testing, and security
  • Mentor and provide technical guidance to other data professionals, fostering a culture of knowledge sharing and continuous learning
  • Be proactive and be willing to work closely with Data Science, DevOps/MLOps, Cloud Infrastructure, or IT-Security colleagues
  • Research and evaluate new technologies and trends to improve existing software or create new solutions
What we offer
What we offer
  • Develop your professional career working with one of the major brands in the fashion industry
  • Opportunity to accelerate the pace of digitalization & eCommerce growth through advanced technology, business intelligence, and analytics
  • Driving high-impact insights enhances decision-making across the entire organization
  • Driving brand equity and digital sales through enhanced digital experiences
  • Interaction with senior business and eCommerce leaders regularly to drive their business toward impactful change
  • Become part of a fast-growing international and diverse team
Read More
Arrow Right

Software Engineer, Full Stack (Data Input)

Benchling is seeking a Software Engineer to join our Data Input team, one of two...
Location
Location
United States , San Francisco
Salary
Salary:
165113.00 - 223388.00 USD / Year
benchling.com Logo
Benchling
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience in a fulltime software engineering role
  • Build software with a product-first approach
  • Experience developing both user-facing & backend experiences in a web application
  • Experience with React, or similar javascript-based frontend application frameworks
  • Experience leading large, sophisticated, long-term initiatives, while also breaking down this work into smaller, iterative projects
  • Enjoy a high-degree of ownership in key areas of the software product you’re building
  • Are interested in learning more about life science (prior knowledge is not required
  • desire to learn is a must)
  • Willing to work onsite in our SF office 3 days a week
Job Responsibility
Job Responsibility
  • Own large-scale projects end-to-end, from ideation with cross-functional partners, through to rollout to end-users
  • Engineer across our stack, from designing and implementing backend models & APIs to crafting rich frontend components and architecture
  • Collaborate closely with product managers, designers, and other teams to build the best product possible for our users
  • Create entirely net-new products, while also investing in taking existing products to feature completeness
  • Be a core team member, mentoring engineers on the team, while also leading technical & architectural discussions among engineers inside & outside the team
  • Help to architect a platform of Data Input components, to build once, and leverage many times over, throughout Benchling
What we offer
What we offer
  • Broad range of medical, dental, and vision plans for employees and their dependents
  • Fertility healthcare and family-forming benefits
  • Four months of fully paid parental leave
  • 401(k) + Employer Match
  • Commuter benefits for in-office employees and a generous home office set up stipend for remote employees
  • Mental health benefits, including therapy and coaching, for employees and their dependents
  • Monthly Wellness stipend
  • Learning and development stipend
  • Generous and flexible vacation
  • Company-wide Winter holiday shutdown
  • Fulltime
Read More
Arrow Right

Senior Full-Stack Engineer (Data-Aware)

Sunscrapers is a technology consultancy that empowers finance and healthcare lea...
Location
Location
Poland , Warsaw
Salary
Salary:
Not provided
sunscrapers.com Logo
Sunscrapers sp. z o.o.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • At least 5+ years of professional experience as a software engineer (backend/full-stack)
  • Undergraduate or graduate degree in Computer Science, Engineering, Mathematics, or similar
  • Strong proficiency in Next.js / React for frontend development
  • Solid backend development skills in Python (FastAPI, Flask, or Django)
  • Experience working with data in any capacity - SQL databases, data processing, analytics, working alongside data teams, or building data-driven applications
  • Proven experience building secure, scalable web applications
  • Experience with AWS stack and services, proficiency in using Docker
  • Experience with infrastructure-as-code tools, like Terraform
  • Excellent command in spoken and written English, at least C1
  • Creative problem-solving skills and excellent technical documentation
Job Responsibility
Job Responsibility
  • Owning and evolving Next.js/React frontend with authenticated flows and secure session handling
  • Designing and implementing secure, scalable access patterns (OAuth2/OIDC, authorization boundaries)
  • Building and maintaining FastAPI backend services for agent-driven workflows and data integrations
  • Developing PoCs using latest technologies, experimenting with third party integrations
  • Delivering production grade applications once PoCs are validated
  • Creating solutions that enable data scientists and business analysts to be self-sufficient as much as possible
  • Finding new ways how to leverage Gen AI applications and underlying vector and graph data storages
  • Working with data warehouses, databases, and building data flows for fetching and aggregation
  • Contributing to infrastructure-as-code and AWS-based systems
  • Documenting design decisions before implementation
What we offer
What we offer
  • Working alongside a talented team of software engineers who are changing the image of Poland abroad
  • Culture of teamwork, professional development and knowledge sharing
  • Flexible working hours and remote work possibility
  • Comfortable office in central Warsaw, equipped with all the necessary tools for conquering the universe (Macbook Pro/Dell, external screen, ergonomic chairs)
  • Fulltime
Read More
Arrow Right