CrawlJobs Logo

PySpark Module Lead

https://www.soprasteria.com Logo

Sopra Steria

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Not provided

Salary Icon

Salary:

Not provided

Job Description:

PySpark Module Lead position at Sopra Steria, a major Tech player in Europe. The role is within the Engineering, Development, Applications department.

Requirements:

  • Good experience in applying best coding practices
  • Good understanding and ability to develop simple to medium applications
  • Strong learnability and analytical skills
  • Strong team player with excellent communication skills
  • Mandatory skills: AWS, PySpark, PL SQL, Java
  • AWS Skills are on S3, Lambda, Cloudwatch, Dynamo DB, SQS, SNS, CloudFormation, Stack, Serverless
  • Total Experience Expected: 04-06 years
  • B.Tech, MCA or equivalent degree

Nice to have:

Desirable skills: Pytest, Junit, Maven, Basic understanding of DevOps concepts

What we offer:
  • Committed to fighting against all forms of discrimination
  • Fosters a work environment that is inclusive and respectful of all differences
  • All positions are open to people with disabilities

Additional Information:

Job Posted:
January 31, 2026

Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for PySpark Module Lead

Pyspark Module Lead

Sopra Steria is seeking a PySpark Module Lead based in Bengaluru, India. The pos...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Python
  • PySpark
  • SQL
  • JavaScript
  • Proficient with Skywise platform and tools e.g. Contour, Code-Workbook, Code, Slate and Ontology
Job Responsibility
Job Responsibility
  • Designing and building solutions for Airbus on foundry
  • Data engineering and integration
  • Data analysis and visualization
  • Application development within Skywise Framework
  • Working closely with Product Owners and core Airbus business functions to understand processes, capture requirements, and solve business problems
  • Full delivery lifecycle for designing, implementing, testing, documenting, and supporting applications
  • CI/CD pipeline development and automated deployments
  • Troubleshooting application issues
  • Providing support on developed tools
What we offer
What we offer
  • Inclusive and respectful work environment
  • Open to people with disabilities
  • Fulltime
Read More
Arrow Right

Pyspark Module Lead

We are seeking a highly skilled and motivated Data Engineer to join our dynamic ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop
  • Proven expertise in designing and deploying data pipelines
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholders
Job Responsibility
Job Responsibility
  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines
  • Utilize PySpark for data processing, transformation, and preparation for model training
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing
  • Implement and manage ETL workflows using Streamsets for data ingestion and transformation
  • Design and construct pipelines to deliver high-quality training and inference datasets
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities
  • Optimize and fine-tune pipelines for performance, scalability, and reliability
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management
  • Implement Spark architecture and optimize Spark jobs for scalable data processing
What we offer
What we offer
  • All positions are open to people with disabilities
  • Commitment to fighting against all forms of discrimination
  • Inclusive and respectful work environment
Read More
Arrow Right

Senior Software Engineer

The Senior Software Engineer will lead a team of data engineers to design, build...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
gomillenniumsoft.com Logo
MillenniumSoft Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in developing data solutions
  • Proven experience in designing and implementing large-scale data pipelines
  • Hands-on expertise with AWS cloud technologies and distributed systems
  • Strong programming skills in PySpark, Python, and SQL
  • Data pipeline development using AWS tools (Step Functions, Airflow, etc.)
  • Distributed systems (S3, ADLS), and modern file/table formats (Parquet, Hudi)
  • ETL tools, orchestration technologies, and database design
  • Knowledge of regulatory standards like SOX, PCI, and HIPAA
  • Bachelor’s or Master’s in Computer Science, Information Systems, or related fields
  • AWS certification preferred
Job Responsibility
Job Responsibility
  • Design and implement scalable data pipelines for structured and unstructured data
  • Lead module architecture design for optimal performance, reliability, and maintainability
  • Mentor team members and ensure alignment with business goals
  • Collaborate with cross-functional teams to enable real-time and batch data processing
  • Ensure adherence to data security, privacy, and governance standards
  • Create and maintain technical documentation for architectures and decisions
  • Conduct code reviews and establish best practices
  • Develop monitoring systems for data pipeline reliability
  • Drive adoption of new technologies and workflows
  • Fulltime
Read More
Arrow Right
New

Primary Teaching Assistant

Are you passionate about shaping young minds? As a Teaching Assistant in primary...
Location
Location
United Kingdom , Caterham
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
May 22, 2026
Flip Icon
Requirements
Requirements
  • communication
  • teaching assistant experience
  • passionate
  • positive
  • flexible
  • hold two work references
  • UK work eligibility
Job Responsibility
Job Responsibility
  • providing crucial support to the class teacher
  • working with a child one-on-one or in small groups
  • preparing the classroom and aiding in the learning process
What we offer
What we offer
  • Convenient Mobility: Ideal for drivers due to diverse school locations
  • Versatile Exposure: Work across various schools for diverse experiences
  • Weekly Pay: Enjoy hassle-free weekly payments with fantastic daily rates
  • Effortless Hours: Submit your hours seamlessly through our time sheet app
  • Rewards Await: Earn £300 through our Refer a Friend scheme
  • Dedicated Support: Benefit from a consultant deeply connected with supported schools
  • Personalised On boarding: Receive tailored assistance for compliance
  • Compliance Assistance: Support for DBS on the update service or help to obtain one
  • career
  • work-life
  • Fulltime
Read More
Arrow Right
New

Primary Teaching Assistant

Are you passionate about shaping young minds? As a Teaching Assistant in primary...
Location
Location
United Kingdom , Caterham
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
May 21, 2026
Flip Icon
Requirements
Requirements
  • Non Teaching
  • communication
  • teaching assistant experience
Job Responsibility
Job Responsibility
  • Providing crucial support to the class teacher
  • Working with a child one-on-one or in small groups
  • Preparing the classroom
  • Aiding in the learning process
What we offer
What we offer
  • Convenient Mobility: Ideal for drivers due to diverse school locations
  • Versatile Exposure: Work across various schools for diverse experiences
  • Weekly Pay: Enjoy hassle-free weekly payments with fantastic daily rates
  • Effortless Hours: Submit your hours seamlessly through our time sheet app
  • Rewards Await: Earn £300 through our Refer a Friend scheme
  • Dedicated Support: Benefit from a consultant deeply connected with supported schools
  • Personalised On boarding: Receive tailored assistance for compliance
  • Essential Traits: Be passionate, positive, flexible, and hold two work references
  • Compliance Assistance: Support for DBS on the update service or help to obtain one, and UK work eligibility
  • Fulltime
Read More
Arrow Right
New

Long term supply teacher

Transform lives through education as a caring and dedicated teacher, making a la...
Location
Location
United Kingdom , Caterham
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • QTS/PGCE qualification required
  • comprehensive working history and two recent working references
  • DBS on the update service or readiness to obtain one
  • eligibility to work in the UK
  • passion and positivity for teaching
  • good communication skills
  • key stage 1 experience
Job Responsibility
Job Responsibility
  • Full teaching responsibilities within your placement
  • marking, planning, and teaching throughout extended periods
  • payment to scale for placements surpassing 6 weeks
  • full-time availability dedicated to teaching
What we offer
What we offer
  • Refer a friend bonus scheme
  • weekly pay with competitive daily rates
  • simple hour submissions through user-friendly app
  • £300 Refer a Friend bonus
  • dedicated consultant
  • personalised assistance for compliance and seamless onboarding
  • career
  • work-life
  • Fulltime
Read More
Arrow Right
New

Supply Sen Teaching Assistant

Are you passionate about shaping young minds? As a Teaching Assistant in primary...
Location
Location
United Kingdom , Caterham
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
May 21, 2026
Flip Icon
Requirements
Requirements
  • Non Teaching
  • 121 teaching experience
  • communication
  • SEN experience
  • teaching assistant experience
What we offer
What we offer
  • Career
  • Work-Life balance
  • Refer a friend bonus scheme
  • Dedicated support
  • Personalised onboarding
  • Compliance assistance
  • Fulltime
Read More
Arrow Right
New

Primary Teaching Assistant

Are you passionate about shaping young minds? As a Teaching Assistant in primary...
Location
Location
United Kingdom , Caterham
Salary
Salary:
Not provided
https://www.randstad.com Logo
Randstad
Expiration Date
May 21, 2026
Flip Icon
Requirements
Requirements
  • communication
  • teaching assistant experience
  • Non Teaching
  • passionate
  • positive
  • flexible
  • hold two work references
  • UK work eligibility
Job Responsibility
Job Responsibility
  • providing crucial support to the class teacher
  • working with a child one-on-one or in small groups
  • preparing the classroom and aiding in the learning process
What we offer
What we offer
  • Refer a friend bonus scheme
  • Convenient Mobility
  • Versatile Exposure
  • Weekly Pay
  • Effortless Hours
  • Rewards Await £300 through Refer a Friend scheme
  • Dedicated Support
  • Personalised On boarding
  • Compliance Assistance
  • training and education
  • Fulltime
Read More
Arrow Right