CrawlJobs Logo

Python/Pyspark Engineer

signifytechnology.com Logo

Signify Technology

Location Icon

Location:
Slovakia , Bratislava

Category Icon

Job Type Icon

Contract Type:
B2B

Salary Icon

Salary:

Not provided

Job Responsibility:

  • Development of a modern Lakehouse architecture based on Azure Datalake using Python and the PySpark framework for implementing business services in the field of insurance
  • implementation of business functions that will allow you to run accounting processes and generate data to meet reporting requirements
  • designing, developing, automating and supporting backend applications that combine data elements from multiple domains and systems
  • cooperation with: other engineers, analysts, product owners and stakeholders to deliver value-added solutions that meet business needs and expectations
  • team lead engineer to create a target architecture for products within the team's scope
  • design of data transformation and data flow services and active participation in coding
  • presentation and communication of ideas and proposals to various stakeholders for the purpose of evaluation and brainstorming
  • implementation of software engineering practices to ensure the quality, performance and sustainability of applications
  • performing peer code reviews

Requirements:

  • min. 4-year demonstrable project experience in the field of software Python engineering
  • language SQL for searching and manipulating data
  • framework PySpark or equivalent for creating and optimizing complex data pipelines
  • Scrum/Agile development methodologies
  • working in a global distributed team in a multicultural environment
  • ability to clearly express technical topics to a non-technical audience
  • active knowledge of English at a communicative level (min. B2-C1)
  • min. Bachelor's or equivalent degree in computer science, data science or a similar discipline

Nice to have:

  • environment Azure Cloud (Databricks, Data Factory, Data Lakes)
  • relational databases (Oracle, PostgreSQL)
  • integration technologies (REST/SOAP API)
  • working in Azure DevOps
  • projects in the insurance or financial sector

Additional Information:

Job Posted:
December 12, 2025

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Python/Pyspark Engineer

Associate MLOps Analyst

The Associate MLOps Analyst will be a key member of Circle K's Data & Analytics ...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor’s degree required, preferably with a quantitative focus (Statistics, Business Analytics, Data Science, Math, Economics, etc.)
  • Master’s degree preferred (MBA/MS Computer Science/M.Tech Computer Science, etc.)
  • 1-2 years of relevant working experience in MLOps
  • Knowledge of core computer science concepts such as common data structures and algorithms, OOPs
  • Programming languages (R, Python, PySpark, etc.)
  • Big data technologies & framework (AWS, Azure, GCP, Hadoop, Spark, etc.)
  • Enterprise reporting systems, relational (MySQL, Microsoft SQL Server etc.), non-relational (MongoDB, DynamoDB) database management systems and Data Engineering tools
  • Exposure to ETL tools and version controlling
  • Experience in building and maintaining CI/CD pipelines for ML models
  • Understanding of machine-learning, information retrieval or recommendation systems
Job Responsibility
Job Responsibility
  • Collaborate with data scientists to deploy ML models into production environments
  • Implement and maintain CI/CD pipelines for machine learning workflows
  • Use version control tools (e.g., Git) and ML lifecycle management tools (e.g., MLflow) for model tracking, versioning, and management
  • Design, build as well as optimize applications containerization and orchestration with Docker and Kubernetes and cloud platforms like AWS or Azure
  • Automating pipelines using understanding of Apache Spark and ETL tools like Informatica PowerCenter, Informatica BDM or DEI, Stream Sets and Apache Airflow
  • Implement model monitoring and alerting systems to track model performance, accuracy, and data drift in production environments
  • Work closely with data scientists to ensure that models are production-ready
  • Collaborate with Data Engineering and Tech teams to ensure infrastructure is optimized for scaling ML applications
  • Optimize ML pipelines for performance and cost-effectiveness
  • Help the Data teams leverage best practices to implement Enterprise level solutions
Read More
Arrow Right

Data Engineer

This role involves designing, building, and optimizing data ingestion, transform...
Location
Location
United States , Radnor
Salary
Salary:
120000.00 - 150000.00 USD / Year
bhsg.com Logo
Beacon Hill
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of professional data engineering experience
  • Strong hands‑on expertise with: Azure Databricks (Spark/PySpark), Azure Data Factory (pipelines, data flows, orchestration), Azure Data Lake Storage, SQL and Python/PySpark scripting
  • Experience building scalable, reliable ETL/ELT solutions in cloud environments
  • Familiarity with CI/CD, version control, and DevOps workflows for data solutions
Job Responsibility
Job Responsibility
  • Designing, building, and optimizing data ingestion, transformation, and delivery pipelines that support enterprise analytics, reporting, and operational data needs
  • Fulltime
Read More
Arrow Right

Data Engineer

We are seeking a Data Engineer with strong experience in Azure cloud technologie...
Location
Location
United States , Radnor
Salary
Salary:
120000.00 - 150000.00 USD / Year
bhsg.com Logo
Beacon Hill
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of professional data engineering experience
  • Strong hands‑on expertise with: Azure Databricks (Spark/PySpark), Azure Data Factory (pipelines, data flows, orchestration), Azure Data Lake Storage, SQL and Python/PySpark scripting
  • Experience building scalable, reliable ETL/ELT solutions in cloud environments
  • Familiarity with CI/CD, version control, and DevOps workflows for data solutions
Job Responsibility
Job Responsibility
  • Designing, building, and optimizing data ingestion, transformation, and delivery pipelines that support enterprise analytics, reporting, and operational data needs
  • Fulltime
Read More
Arrow Right

Graduate Data Engineer

As a Graduate Data Engineer, you will build and maintain scalable data pipelines...
Location
Location
United Kingdom , Marlow
Salary
Salary:
Not provided
srgtalent.com Logo
SRG
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Degree in Computer Science, Engineering, Mathematics, or similar, or similar work experience
  • Up to 2 years of experience building data pipelines at work or through internships
  • Can write clear and reliable Python/PySpark code
  • Familiar with popular analytics tools (like pandas, numpy, matplotlib), big data frameworks (like Spark), and cloud services (like Palantir, AWS, Azure, or Google Cloud)
  • Deep understanding of data models, relational and non-relational databases, and how they are used to organize, store, and retrieve data efficiently for analytics and machine learning
  • Knowledge about software engineering methods, including DevOps, DataOps, or MLOps is a plus
  • Master's degree in engineering (such as AI/ML, Data Systems, Computer Science, Mathematics, Biotechnology, Physics), or minimum 2 years of relevant technology experience
  • Experience with Generative AI (GenAI) and agentic systems will be considered a strong plus
  • Have a proactive and adaptable mindset: willing to take initiative, learn new skills, and contribute to different aspects of a project as needed to drive solutions from start to finish, even beyond the formal job description
  • Show a strong ability to thrive in situations of ambiguity, taking initiative to create clarity for yourself and the team, and proactively driving progress even when details are uncertain or evolving
Job Responsibility
Job Responsibility
  • Build and maintain data pipelines, leveraging PySpark and/or Typescript within Foundry, to transform raw data into reliable, usable datasets
  • Assist in preparing and optimizing data pipelines to support machine learning and AI model development, ensuring datasets are clean, well-structured, and readily usable by Data Science teams
  • Support the integration and management of feature engineering processes and model outputs into Foundry's data ecosystem, helping enable scalable deployment and monitoring of AI/ML solutions
  • Engaged in gathering and translating stakeholder requirements for key data models and reporting, with a focus on Palantir Foundry workflows and tools
  • Participate in developing and refining dashboards and reports in Foundry to visualize key metrics and insights
  • Collaborate with Product, Engineering, and GTM teams to align data architecture and solutions, learning to support scalable, self-serve analytics across the organization
  • Have some prompt engineering experience with large language models, including writing and evaluating complex multi-step prompts
  • Continuously develop your understanding of the company's data landscape, including Palantir Foundry's ontology-driven approach and best practices for data management
Read More
Arrow Right

Senior Data Engineer

Sr Data Engineer. SR DE-I: Highly skilled Data Engineer with minimum 5+ years of...
Location
Location
India , Kolkata
Salary
Salary:
Not provided
inxiteout.ai Logo
InXiteOut
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 5+ years of relevant experience in SQL, PySpark, ETL, Data Lakes and Azure Tech Stack
  • 3+ years of experience in building data Pipelines with Python/PySpark
  • 4+ years of experience in Azure ETL stack (eg. Blog Storage, Data Lake, Data Factory, Synapse)
  • 4+ years of experience with SQL
  • Proficient understanding of code versioning tools such as Git and PM tool like Jira
  • Excellent verbal and written communication skills
  • UG: B.Sc in Any Specialization, BCA in Any Specialization, B.Tech/B.E. in Any Specialization
  • A good internet connection is a must
  • Fulltime
Read More
Arrow Right

Data Architect

We are seeking a talented and experienced Data Architect to join our team. The D...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full-Time bachelor’s or master’s degree in engineering/technology, computer science, information technology, or related fields
  • 10+ years of total experience in data modeling and database design
  • experience in Retail domain will be added advantage
  • 8+ years of experience in data engineering development and support
  • 3+ years of experience in leading technical team of data engineers and BI engineers
  • Proficiency in data modeling tools such as Erwin, ER/Studio, or similar tools
  • Strong knowledge of Azure cloud infrastructure and development using SQL/Python/PySpark using ADF, Synapse and Databricks
  • Hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Azure Databricks, Blob Storage, Python/PySpark, Logic Apps, Key Vault, and Azure functions
  • Strong communication, interpersonal, collaboration skills along with leadership capabilities
  • Ability to work effectively in a fast-paced, dynamic environment as cloud SME
Job Responsibility
Job Responsibility
  • Collaborate with solution architect, data engineers, business stakeholders, business analysts, and DQ testers to ensure data management and data governance framework is defined as critical components
  • Design and develop data models using industry-standard modeling techniques and tools
  • Perform data profiling, data lineage and analysis to understand data quality, structure, and relationships
  • Optimize data models for performance, scalability, and usability by creating optimal data storage layer
  • Define and enforce data modeling standards, best practices, and guidelines
  • Participate in data governance initiatives to ensure compliance with data management policies and standards
  • Work closely with database administrators and developers to implement data models in relational and non-relational database systems
  • Conduct data model reviews and provide recommendations for improvements
  • Stay updated on emerging trends and technologies in data modeling and data management
  • Conduct continuous audits of data management system performance and refine where necessary
  • Fulltime
Read More
Arrow Right

Data Architect

We are seeking a talented and experienced Data Architect/ Modeller to join our t...
Location
Location
India , Gurugram
Salary
Salary:
Not provided
https://www.circlek.com Logo
Circle K
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full-Time bachelor’s or master’s degree in engineering/technology, computer science, information technology, or related fields
  • 10+ years of total experience in data modeling and database design and experience in Retail domain will be added advantage
  • 8+ years of experience in data engineering development and support
  • 3+ years of experience in leading technical team of data engineers and BI engineers
  • Proficiency in data modeling tools such as Erwin, ER/Studio, or similar tools
  • Strong knowledge of Azure cloud infrastructure and development using SQL/Python/PySpark using ADF, Synapse and Databricks
  • Hands-on experience with Azure Data Factory, Azure Synapse Analytics, Azure Analysis Services, Azure Databricks, Blob Storage, Python/PySpark, Logic Apps, Key Vault, and Azure functions
  • Strong communication, interpersonal, collaboration skills along with leadership capabilities
  • Ability to work effectively in a fast-paced, dynamic environment as cloud SME
  • Act as single point of contact for all kinds of data management related queries to make data decisions
Job Responsibility
Job Responsibility
  • Collaborate with solution architect, data engineers, business stakeholders, business analysts, and DQ testers to ensure data management and data governance framework is defined as critical components
  • Design and develop data models using industry-standard modeling techniques and tools
  • Perform data profiling, data lineage and analysis to understand data quality, structure, and relationships
  • Optimize data models for performance, scalability, and usability by creating optimal data storage layer
  • Define and enforce data modeling standards, best practices, and guidelines
  • Participate in data governance initiatives to ensure compliance with data management policies and standards
  • Work closely with database administrators and developers to implement data models in relational and non-relational database systems
  • Conduct data model reviews and provide recommendations for improvements
  • Stay updated on emerging trends and technologies in data modeling and data management
  • Fulltime
Read More
Arrow Right
New

Data & AI Engineer

As a Data & AI Engineer at NTT DATA, you will design and implement scalable data...
Location
Location
Belgium , Diegem
Salary
Salary:
Not provided
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3–7+ years in data/AI engineering (or equivalent impact) delivering production systems on Azure
  • Python (production code, packaging, testing)
  • PySpark
  • SQL
  • Databricks (Repos/Workflows, Delta Lake, Unity Catalog, MLflow)
  • Azure AI Foundry (projects, prompt/eval flows, model endpoints)
  • Azure AI Search (index design, vector embeddings, skillsets/indexers, semantic search)
  • Azure OpenAI (prompt design, grounding, safety filters, function/tool calling, evaluation)
  • Terraform (AzureRM + Databricks providers), modular patterns, pipelines, approvals
  • Git & CI/CD (Azure DevOps or GitHub), code reviews, trunk‑based or GitFlow
Job Responsibility
Job Responsibility
  • Build reliable batch/stream pipelines in Databricks (Python/PySpark, SQL) with Delta Lake and Unity Catalog
  • Implement best practices for code quality, testing, documentation, lineage, and cost‑efficient performance
  • Design and deliver GenAI solutions using Azure AI Foundry (projects, prompt flow/evaluation), Azure OpenAI (chat/completions, tool/function calling), and Azure AI Search (indexing, vector search, semantic ranking) with robust RAG patterns
  • Instrument evaluation, grounding, safety checks, and quality metrics for AI features
  • Use MLflow for experiment tracking, model packaging, and deployment
  • standardize environments and feature stores where relevant
  • Build CI/CD for data and AI workloads (e.g., Azure DevOps/GitHub Actions) and implement monitoring & observability (logs, metrics, drift)
  • Provision Azure and Databricks resources with Terraform (modular design, workspaces/state, policies, service principals, Key Vault)
  • Apply RBAC, secrets management, data masking, and governance (Unity Catalog / Purview) with privacy‑by‑design and compliance best practices
  • Partner with product owners, architects, and SMEs to translate use‑cases into simple, secure, cost‑aware solutions
  • Fulltime
Read More
Arrow Right