CrawlJobs Logo

Python/Pyspark Engineer

signifytechnology.com Logo

Signify Technology

Location Icon

Location:
Slovakia , Bratislava

Category Icon

Job Type Icon

Contract Type:
B2B

Salary Icon

Salary:

Not provided

Job Responsibility:

  • Development of a modern Lakehouse architecture based on Azure Datalake using Python and the PySpark framework for implementing business services in the field of insurance
  • implementation of business functions that will allow you to run accounting processes and generate data to meet reporting requirements
  • designing, developing, automating and supporting backend applications that combine data elements from multiple domains and systems
  • cooperation with: other engineers, analysts, product owners and stakeholders to deliver value-added solutions that meet business needs and expectations
  • team lead engineer to create a target architecture for products within the team's scope
  • design of data transformation and data flow services and active participation in coding
  • presentation and communication of ideas and proposals to various stakeholders for the purpose of evaluation and brainstorming
  • implementation of software engineering practices to ensure the quality, performance and sustainability of applications
  • performing peer code reviews

Requirements:

  • min. 4-year demonstrable project experience in the field of software Python engineering
  • language SQL for searching and manipulating data
  • framework PySpark or equivalent for creating and optimizing complex data pipelines
  • Scrum/Agile development methodologies
  • working in a global distributed team in a multicultural environment
  • ability to clearly express technical topics to a non-technical audience
  • active knowledge of English at a communicative level (min. B2-C1)
  • min. Bachelor's or equivalent degree in computer science, data science or a similar discipline

Nice to have:

  • environment Azure Cloud (Databricks, Data Factory, Data Lakes)
  • relational databases (Oracle, PostgreSQL)
  • integration technologies (REST/SOAP API)
  • working in Azure DevOps
  • projects in the insurance or financial sector

Additional Information:

Job Posted:
December 12, 2025

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Python/Pyspark Engineer

Associate MLOps Analyst

The Associate MLOps Analyst will be a key member of Circle K's Data & Analytics ...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree required, preferably with a quantitative focus (Statistics, Business Analytics, Data Science, Math, Economics, etc.)
  • Master’s degree preferred (MBA/MS Computer Science/M.Tech Computer Science, etc.)
  • 1-2 years of relevant working experience in MLOps
  • Knowledge of core computer science concepts such as common data structures and algorithms, OOPs
  • Programming languages (R, Python, PySpark, etc.)
  • Big data technologies & framework (AWS, Azure, GCP, Hadoop, Spark, etc.)
  • Enterprise reporting systems, relational (MySQL, Microsoft SQL Server etc.), non-relational (MongoDB, DynamoDB) database management systems and Data Engineering tools
  • Exposure to ETL tools and version controlling
  • Experience in building and maintaining CI/CD pipelines for ML models
  • Understanding of machine-learning, information retrieval or recommendation systems
Job Responsibility
Job Responsibility
  • Collaborate with data scientists to deploy ML models into production environments
  • Implement and maintain CI/CD pipelines for machine learning workflows
  • Use version control tools (e.g., Git) and ML lifecycle management tools (e.g., MLflow) for model tracking, versioning, and management
  • Design, build as well as optimize applications containerization and orchestration with Docker and Kubernetes and cloud platforms like AWS or Azure
  • Automating pipelines using understanding of Apache Spark and ETL tools like Informatica PowerCenter, Informatica BDM or DEI, Stream Sets and Apache Airflow
  • Implement model monitoring and alerting systems to track model performance, accuracy, and data drift in production environments
  • Work closely with data scientists to ensure that models are production-ready
  • Collaborate with Data Engineering and Tech teams to ensure infrastructure is optimized for scaling ML applications
  • Optimize ML pipelines for performance and cost-effectiveness
  • Help the Data teams leverage best practices to implement Enterprise level solutions
Read More
Arrow Right

Python/Pyspark Engineer

Location
Location
United States , Jersey City
Salary
Salary:
115000.00 USD / Year
realign-llc.com Logo
Realign
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong proficiency in Python, packaging, dependency management, and virtual environments
  • Hands-on experience with PySpark, including Spark performance tuning (partitioning, caching, broadcast joins, memory optimization)
  • Expertise in data ingestion (batch/stream), schema management, and robust error-handling/retry logic
  • Solid unit and integration testing practices, including data quality validations
  • Experience with CI/CD pipelines (Azure DevOps/Jenkins), Git branching strategies, and artifact versioning
  • Working experience with Cloudera/Hadoop (HDFS, Spark, Hive/Impala) and Databricks (Delta Lake, clusters, jobs, notebooks)
  • Knowledge of observability techniques: structured logging, metrics, tracing, and debugging in distributed systems
  • Secure coding practices including secrets management, PII protection, and compliance-aware development
  • Strong documentation discipline for frameworks, reusable components, and best-practice patterns
  • Effective collaboration with Cloud Architects and Data Ops to ensure stable and supportable pipelines
Job Responsibility
Job Responsibility
  • Design and develop scalable Python/PySpark ingestion and transformation pipelines
  • Implement schema evolution logic, validation frameworks, and resilient error-handling mechanisms
  • Optimize Spark jobs for performance, cost efficiency, and production readiness
  • Integrate all jobs into automated CI/CD pipelines, ensuring versioning and release governance
  • Work closely with Ops teams to ensure proper monitoring, logging, and operational supportability
  • Participate in Agile ceremonies, sprint planning, code reviews, and demo sessions
  • Fulltime
Read More
Arrow Right

Lead Data Engineer

We’re looking for a highly skilled Lead Data Engineer to provide technical leade...
Location
Location
United Kingdom
Salary
Salary:
70000.00 GBP / Year
zebrapeople.com Logo
Zebra People
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Strong consultancy background across ETL/ELT, data warehousing and big‑data solutions
  • hands‑on delivery experience
  • proven leadership/mentoring in an onshore–offshore model
  • Proficiency with ETL tools and cloud data services (AWS preferred — e.g., Glue
  • with exposure to Azure Data Factory)
  • experience implementing CI/CD for data
  • knowledge of Informatica tools is beneficial
  • Experience working with major databases such as Postgres, Redshift, Oracle, SQL Server or Snowflake
  • strong SQL capability
  • working knowledge of Python/PySpark
Job Responsibility
Job Responsibility
  • Provide technical leadership across data engineering, delivery execution and project governance
  • Interpret detailed technical requirements and deliverables across multiple projects
  • Act as subject‑matter expert in data‑engineering technologies
  • Contribute to effective solution design
  • Mentor, support and guide developers
  • Balance involvement across various initiatives to provide direction, technical leadership and oversight
  • Fulltime
Read More
Arrow Right

Senior Data Engineer II

We are seeking a highly skilled Senior Data Engineer with 5+ years of experience...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
inxiteout.ai Logo
InXiteOut
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of experience with SQL
  • 4+ years of hands-on experience in building data pipelines using Python/PySpark
  • 4+ years of experience with the Azure ETL stack
  • Strong understanding of: Data modeling
  • Distributed computing
  • Data warehousing concepts
  • Experience with code versioning tools (Git)
  • Familiarity with Agile tools such as Azure DevOps or Jira
  • Bachelor’s degree in: B.Sc / BCA / B.Tech / B.E (any specialization)
  • Strong verbal and written communication skills
Job Responsibility
Job Responsibility
  • Design, develop, and maintain robust data pipelines using Python/PySpark
  • Build and optimize ETL workflows for large-scale data processing
  • Work with Azure services such as: Azure Blob Storage
  • Azure Data Lake
  • Azure Data Factory
  • Azure Synapse Analytics
  • Ensure high performance and reliability of data systems
  • Collaborate with cross-functional teams to understand data requirements
  • Implement best practices for data governance, security, and quality
  • Use version control tools like Git and manage tasks via Azure DevOps or Jira
  • Fulltime
Read More
Arrow Right

Data Engineer

This role involves designing, building, and optimizing data ingestion, transform...
Location
Location
United States , Radnor
Salary
Salary:
120000.00 - 150000.00 USD / Year
bhsg.com Logo
Beacon Hill
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of professional data engineering experience
  • Strong hands‑on expertise with: Azure Databricks (Spark/PySpark), Azure Data Factory (pipelines, data flows, orchestration), Azure Data Lake Storage, SQL and Python/PySpark scripting
  • Experience building scalable, reliable ETL/ELT solutions in cloud environments
  • Familiarity with CI/CD, version control, and DevOps workflows for data solutions
Job Responsibility
Job Responsibility
  • Designing, building, and optimizing data ingestion, transformation, and delivery pipelines that support enterprise analytics, reporting, and operational data needs
  • Fulltime
Read More
Arrow Right

Data Engineer

We are seeking a Data Engineer with strong experience in Azure cloud technologie...
Location
Location
United States , Radnor
Salary
Salary:
120000.00 - 150000.00 USD / Year
bhsg.com Logo
Beacon Hill
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of professional data engineering experience
  • Strong hands‑on expertise with: Azure Databricks (Spark/PySpark), Azure Data Factory (pipelines, data flows, orchestration), Azure Data Lake Storage, SQL and Python/PySpark scripting
  • Experience building scalable, reliable ETL/ELT solutions in cloud environments
  • Familiarity with CI/CD, version control, and DevOps workflows for data solutions
Job Responsibility
Job Responsibility
  • Designing, building, and optimizing data ingestion, transformation, and delivery pipelines that support enterprise analytics, reporting, and operational data needs
  • Fulltime
Read More
Arrow Right

Graduate Data Engineer

As a Graduate Data Engineer, you will build and maintain scalable data pipelines...
Location
Location
United Kingdom , Marlow
Salary
Salary:
Not provided
srgtalent.com Logo
SRG
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Degree in Computer Science, Engineering, Mathematics, or similar, or similar work experience
  • Up to 2 years of experience building data pipelines at work or through internships
  • Can write clear and reliable Python/PySpark code
  • Familiar with popular analytics tools (like pandas, numpy, matplotlib), big data frameworks (like Spark), and cloud services (like Palantir, AWS, Azure, or Google Cloud)
  • Deep understanding of data models, relational and non-relational databases, and how they are used to organize, store, and retrieve data efficiently for analytics and machine learning
  • Knowledge about software engineering methods, including DevOps, DataOps, or MLOps is a plus
  • Master's degree in engineering (such as AI/ML, Data Systems, Computer Science, Mathematics, Biotechnology, Physics), or minimum 2 years of relevant technology experience
  • Experience with Generative AI (GenAI) and agentic systems will be considered a strong plus
  • Have a proactive and adaptable mindset: willing to take initiative, learn new skills, and contribute to different aspects of a project as needed to drive solutions from start to finish, even beyond the formal job description
  • Show a strong ability to thrive in situations of ambiguity, taking initiative to create clarity for yourself and the team, and proactively driving progress even when details are uncertain or evolving
Job Responsibility
Job Responsibility
  • Build and maintain data pipelines, leveraging PySpark and/or Typescript within Foundry, to transform raw data into reliable, usable datasets
  • Assist in preparing and optimizing data pipelines to support machine learning and AI model development, ensuring datasets are clean, well-structured, and readily usable by Data Science teams
  • Support the integration and management of feature engineering processes and model outputs into Foundry's data ecosystem, helping enable scalable deployment and monitoring of AI/ML solutions
  • Engaged in gathering and translating stakeholder requirements for key data models and reporting, with a focus on Palantir Foundry workflows and tools
  • Participate in developing and refining dashboards and reports in Foundry to visualize key metrics and insights
  • Collaborate with Product, Engineering, and GTM teams to align data architecture and solutions, learning to support scalable, self-serve analytics across the organization
  • Have some prompt engineering experience with large language models, including writing and evaluating complex multi-step prompts
  • Continuously develop your understanding of the company's data landscape, including Palantir Foundry's ontology-driven approach and best practices for data management
Read More
Arrow Right

Senior Data Engineer

Sr Data Engineer. SR DE-I: Highly skilled Data Engineer with minimum 5+ years of...
Location
Location
India , Kolkata
Salary
Salary:
Not provided
inxiteout.ai Logo
InXiteOut
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 5+ years of relevant experience in SQL, PySpark, ETL, Data Lakes and Azure Tech Stack
  • 3+ years of experience in building data Pipelines with Python/PySpark
  • 4+ years of experience in Azure ETL stack (eg. Blog Storage, Data Lake, Data Factory, Synapse)
  • 4+ years of experience with SQL
  • Proficient understanding of code versioning tools such as Git and PM tool like Jira
  • Excellent verbal and written communication skills
  • UG: B.Sc in Any Specialization, BCA in Any Specialization, B.Tech/B.E. in Any Specialization
  • A good internet connection is a must
  • Fulltime
Read More
Arrow Right