CrawlJobs Logo

PySpark Module Lead

https://www.soprasteria.com Logo

Sopra Steria

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

PySpark Module Lead position at Sopra Steria, a major Tech player in Europe. The role is within the Engineering, Development, Applications department.

Requirements:

  • Good experience in applying best coding practices
  • Good understanding and ability to develop simple to medium applications
  • Strong learnability and analytical skills
  • Strong team player with excellent communication skills
  • Mandatory skills: AWS, PySpark, PL SQL, Java
  • AWS Skills are on S3, Lambda, Cloudwatch, Dynamo DB, SQS, SNS, CloudFormation, Stack, Serverless
  • Total Experience Expected: 04-06 years
  • B.Tech, MCA or equivalent degree

Nice to have:

Desirable skills: Pytest, Junit, Maven, Basic understanding of DevOps concepts

What we offer:
  • Committed to fighting against all forms of discrimination
  • Fosters a work environment that is inclusive and respectful of all differences
  • All positions are open to people with disabilities

Additional Information:

Job Posted:
January 31, 2026

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for PySpark Module Lead

Pyspark Module Lead

Sopra Steria is seeking a PySpark Module Lead based in Bengaluru, India. The pos...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Python
  • PySpark
  • SQL
  • JavaScript
  • Proficient with Skywise platform and tools e.g. Contour, Code-Workbook, Code, Slate and Ontology
Job Responsibility
Job Responsibility
  • Designing and building solutions for Airbus on foundry
  • Data engineering and integration
  • Data analysis and visualization
  • Application development within Skywise Framework
  • Working closely with Product Owners and core Airbus business functions to understand processes, capture requirements, and solve business problems
  • Full delivery lifecycle for designing, implementing, testing, documenting, and supporting applications
  • CI/CD pipeline development and automated deployments
  • Troubleshooting application issues
  • Providing support on developed tools
What we offer
What we offer
  • Inclusive and respectful work environment
  • Open to people with disabilities
  • Fulltime
Read More
Arrow Right

Pyspark Module Lead

We are seeking a highly skilled and motivated Data Engineer to join our dynamic ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop
  • Proven expertise in designing and deploying data pipelines
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholders
Job Responsibility
Job Responsibility
  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines
  • Utilize PySpark for data processing, transformation, and preparation for model training
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing
  • Implement and manage ETL workflows using Streamsets for data ingestion and transformation
  • Design and construct pipelines to deliver high-quality training and inference datasets
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities
  • Optimize and fine-tune pipelines for performance, scalability, and reliability
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management
  • Implement Spark architecture and optimize Spark jobs for scalable data processing
What we offer
What we offer
  • All positions are open to people with disabilities
  • Commitment to fighting against all forms of discrimination
  • Inclusive and respectful work environment
Read More
Arrow Right

Senior Software Engineer

The Senior Software Engineer will lead a team of data engineers to design, build...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
gomillenniumsoft.com Logo
MillenniumSoft Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in developing data solutions
  • Proven experience in designing and implementing large-scale data pipelines
  • Hands-on expertise with AWS cloud technologies and distributed systems
  • Strong programming skills in PySpark, Python, and SQL
  • Data pipeline development using AWS tools (Step Functions, Airflow, etc.)
  • Distributed systems (S3, ADLS), and modern file/table formats (Parquet, Hudi)
  • ETL tools, orchestration technologies, and database design
  • Knowledge of regulatory standards like SOX, PCI, and HIPAA
  • Bachelor’s or Master’s in Computer Science, Information Systems, or related fields
  • AWS certification preferred
Job Responsibility
Job Responsibility
  • Design and implement scalable data pipelines for structured and unstructured data
  • Lead module architecture design for optimal performance, reliability, and maintainability
  • Mentor team members and ensure alignment with business goals
  • Collaborate with cross-functional teams to enable real-time and batch data processing
  • Ensure adherence to data security, privacy, and governance standards
  • Create and maintain technical documentation for architectures and decisions
  • Conduct code reviews and establish best practices
  • Develop monitoring systems for data pipeline reliability
  • Drive adoption of new technologies and workflows
  • Fulltime
Read More
Arrow Right
New

Software Engineer

You will lead the development and delivery of web-based solutions that directly ...
Location
Location
France , Bidart
Salary
Salary:
Not provided
sophiagenetics.com Logo
SOPHiA GENETICS
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2-3 years’ experience in the software engineering field
  • Bachelor’s degree in Computer Science or Engineering or equivalent professional experience
  • demonstrated experience in developing reliable web-based services and have a firm grasp on the underlying challenges of releasing web components to production
  • solid proficiency of front-end framework: React, Next JS, Node, JSX
  • proficiency in Java for back-end development
  • solid understanding of backend technologies and microservices architecture
  • experience with databases like MySQL, MariaDB, and DuckDB
  • experience with AI use in SDLC
  • familiarity with Linux environments and GitLab for version control and CI/CD pipelines
  • good knowledge of software engineering best-practices, Agile, CI/CD, testing frameworks, TDD, BDD
Job Responsibility
Job Responsibility
  • Develop and optimize web-based services and components, leveraging modern frameworks (React, Next.js, Node.js) and robust backend technologies (Java, microservices) to deliver seamless user experiences
  • Own and drive the full lifecycle of complex, high-performance software systems - from design and implementation to testing, deployment, and maintenance - ensuring reliability and scalability
  • Enhance system performance through database optimization (MySQL, MariaDB, DuckDB) and efficient query design, ensuring data integrity and fast processing
  • Leverage AI and modern tools to enhance SDLC, observability, monitoring, and system intelligence, driving data-driven decision-making
  • Orchestrate cross-functional projects by collaborating with Product, QA, Architecture, and other teams to design interoperable systems and deliver high-impact features
  • Lead code reviews and mentor junior engineers, fostering a culture of quality, innovation, and continuous improvement
  • Resolve complex technical issues with efficiency, minimizing downtime and ensuring smooth operations
  • Streamline SDLC processes, integrating Agile, CI/CD, and DevOps tools (Docker, Kubernetes, GitLab) to accelerate delivery and improve workflow automation
  • Collaborate with stakeholders (PMO, Product, QA, Architecture) to align technical execution with product vision and user needs
  • Bridge technical execution with business goals by deeply understanding the genomics and digital healthcare domain, ensuring solutions align with user needs and regulatory requirements
What we offer
What we offer
  • Health benefits for you and your family covered by 80% employer contributions
  • Life Insurance and pensions contribution
  • SWILE meal vouchers and home office allowances
  • Fulltime
Read More
Arrow Right
New

Regional finance associate

We are seeking a high-potential Finance Associate to be groomed for future manag...
Location
Location
Malaysia , Kuala Lumpur
Salary
Salary:
6000.00 - 9000.00 MYR / Month
https://www.randstad.com Logo
Randstad
Expiration Date
May 02, 2026
Flip Icon
Requirements
Requirements
  • Degree in Accounting, Business Administration, or an equivalent professional qualification
  • 3 – 5 years of accounting experience
  • External audit exposure in fund accounting, private equity, or fund management (preference)
  • Strong foundation in Singapore Financial Reporting Standards (FRS) and IRAS regulations (significant advantage)
  • Proficiency in Microsoft Office
  • Experience with NetSuite or similar ERP systems (bonus)
  • Proactive problem-solver with a passion for process improvement
  • Ability to prioritize and perform under tight timelines with minimal supervision
  • Excellent communication skills and high attention to detail
Job Responsibility
Job Responsibility
  • Independently manage the full set of accounts for assigned entities, encompassing monthly closing, reconciliations, and supporting schedules
  • Execute core accounting functions including cash management, payment operations, and payroll processing
  • Determine Net Asset Values (NAVs), calculate fund performance, and ensure the accuracy of fund accounting records
  • Prepare and analyze financial and operational reports to facilitate management decision-making
  • Lead the coordination of annual external audits across multiple entities and funds, ensuring all schedules and documentation are complete and accurate
  • Prepare and submit quarterly GST filings and manage local tax matters with relevant authorities
  • Assist in the preparation of statutory forms, reports, and surveys required by the Monetary Authority of Singapore (MAS)
  • Ensure compliance with statutory regulations and internal financial controls
  • Manage relationships with professional service providers, including fund administrators, banks, tax agents, corporate secretaries, and auditors
  • Support the distribution of returns to existing investors, including the drafting of necessary documentation
Read More
Arrow Right
New

Assistant Front Office Manager

Assists the Front Office Manager in administering front office functions and sup...
Location
Location
United States , Maui
Salary
Salary:
72000.00 - 95000.00 USD / Year
https://www.marriott.com Logo
Marriott Bonvoy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • High school diploma or GED
  • 2 years experience in the guest services, front desk, or related professional area
  • OR 2-year degree from an accredited university in Hotel and Restaurant Management, Hospitality, Business Administration, or related major
  • no work experience required
Job Responsibility
Job Responsibility
  • Assists the Front Office Manager in administering front office functions and supervising staff on a daily basis
  • Utilizes interpersonal and communication skills to lead, influence, and encourage others
  • advocates sound financial/business decision making
  • demonstrates honesty/integrity
  • leads by example
  • Encourages and builds mutual trust, respect, and cooperation among team members
  • Supervises and manages employees
  • Managing all day-to-day operations
  • Understanding employee positions well enough to perform duties in employees' absence
  • Ensures employee recognition is taking place on all shifts
  • Fulltime
Read More
Arrow Right
New

VP Sales - Insurtech

An innovative and fast-growing insurtech operating at the forefront of digital t...
Location
Location
United Kingdom , London
Salary
Salary:
120000.00 GBP / Year
calibrecandidates.com Logo
Calibre Candidates
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven track record of SaaS sales success within Insurance and London Market clients
  • Strong focus within Property & Casualty (P&C) insurance
  • Entrepreneurial mindset with a proactive, self-starting approach
  • Demonstrated ability to build and leverage a strong industry network
  • Credible communicator with the ability to engage C-suite stakeholders
  • Commercially astute with a strategic approach to business development
Job Responsibility
Job Responsibility
  • Drive new business sales across London Market (insurers, brokers, MGAs)
  • Build and execute a strategic sales plan aligned with company growth targets
  • Develop and maintain a strong network of senior industry contacts
  • Identify and convert new opportunities within Property & Casualty (P&C) lines
  • Collaborate with product and leadership teams to refine market proposition
  • Represent the business at industry events, conferences, and regular client meetings
  • Lead complex sales cycles from origination through to close
  • Fulltime
Read More
Arrow Right
New

Finance Manager

You will be working for a dynamic and growing property development company commi...
Location
Location
Malaysia , Kuala Lumpur
Salary
Salary:
10000.00 - 13000.00 MYR / Month
https://www.randstad.com Logo
Randstad
Expiration Date
May 03, 2026
Flip Icon
Requirements
Requirements
  • Bachelor’s degree in Finance, Accounting, or related field
  • Minimum 5 years experience in finance or accounting roles
  • leadership experience preferred
  • Strong knowledge of Malaysian financial regulations and tax laws
Job Responsibility
Job Responsibility
  • Prepare and analyze financial statements, budgets, forecasts, and reports
  • Manage accounting operations including month-end and year-end closing
  • Ensure compliance with statutory requirements, financial regulations, and tax filings
  • Lead financial planning and analysis to support decision making
What we offer
What we offer
  • Competitive salary and performance bonus
  • Medical and healthcare benefits
  • EPF & SOCSO contributions
  • Learning and development opportunities
  • Supportive and collaborative work environment
Read More
Arrow Right