CrawlJobs Logo

Pyspark Module Lead

https://www.soprasteria.com Logo

Sopra Steria

Location Icon

Location:
India , Bengaluru

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

Not provided

Job Description:

Sopra Steria is seeking a PySpark Module Lead based in Bengaluru, India. The position involves designing and building data solutions for Airbus, leveraging the Skywise platform and tools, and working along Product Owners and business functions. The role requires a strong understanding of Python, PySpark, SQL, and JavaScript, and proficiency with Skywise tools such as Contour, Code-Workbook, and Ontology.

Job Responsibility:

  • Designing and building solutions for Airbus on foundry
  • Data engineering and integration
  • Data analysis and visualization
  • Application development within Skywise Framework
  • Working closely with Product Owners and core Airbus business functions to understand processes, capture requirements, and solve business problems
  • Full delivery lifecycle for designing, implementing, testing, documenting, and supporting applications
  • CI/CD pipeline development and automated deployments
  • Troubleshooting application issues
  • Providing support on developed tools

Requirements:

  • Python
  • PySpark
  • SQL
  • JavaScript
  • Proficient with Skywise platform and tools e.g. Contour, Code-Workbook, Code, Slate and Ontology

Nice to have:

  • Knowledge of airline and MRO operations
  • AWS Cloud Services like EC2, RDS and Redshift
  • Agile/Scrum
  • SAFe
  • Palantir tools like Slate, Workshop
What we offer:
  • Inclusive and respectful work environment
  • Open to people with disabilities

Additional Information:

Job Posted:
May 05, 2025

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Pyspark Module Lead

Pyspark Module Lead

We are seeking a highly skilled and motivated Data Engineer to join our dynamic ...
Location
Location
India , Noida
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Advanced SQL (Window functions), Spark Architecture, Pyspark or Scala with Spark, Hadoop
  • Proven expertise in designing and deploying data pipelines
  • Strong problem-solving skills and ability to work effectively in a collaborative team environment
  • Excellent communication skills and ability to translate technical concepts to non-technical stakeholders
Job Responsibility
Job Responsibility
  • Work in tandem with Data Scientists to design, develop, and implement machine learning pipelines
  • Utilize PySpark for data processing, transformation, and preparation for model training
  • Leverage AWS EMR and S3 for scalable and efficient data storage and processing
  • Implement and manage ETL workflows using Streamsets for data ingestion and transformation
  • Design and construct pipelines to deliver high-quality training and inference datasets
  • Collaborate with cross-functional teams to ensure smooth deployment and real-time/near real-time inferencing capabilities
  • Optimize and fine-tune pipelines for performance, scalability, and reliability
  • Ensure IAM policies and permissions are appropriately configured for secure data access and management
  • Implement Spark architecture and optimize Spark jobs for scalable data processing
What we offer
What we offer
  • All positions are open to people with disabilities
  • Commitment to fighting against all forms of discrimination
  • Inclusive and respectful work environment
Read More
Arrow Right

PySpark Module Lead

PySpark Module Lead position at Sopra Steria, a major Tech player in Europe. The...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
https://www.soprasteria.com Logo
Sopra Steria
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Good experience in applying best coding practices
  • Good understanding and ability to develop simple to medium applications
  • Strong learnability and analytical skills
  • Strong team player with excellent communication skills
  • Mandatory skills: AWS, PySpark, PL SQL, Java
  • AWS Skills are on S3, Lambda, Cloudwatch, Dynamo DB, SQS, SNS, CloudFormation, Stack, Serverless
  • Total Experience Expected: 04-06 years
  • B.Tech, MCA or equivalent degree
What we offer
What we offer
  • Committed to fighting against all forms of discrimination
  • Fosters a work environment that is inclusive and respectful of all differences
  • All positions are open to people with disabilities
Read More
Arrow Right

Senior Software Engineer

The Senior Software Engineer will lead a team of data engineers to design, build...
Location
Location
United States , San Diego
Salary
Salary:
Not provided
gomillenniumsoft.com Logo
MillenniumSoft Inc
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 10+ years of experience in developing data solutions
  • Proven experience in designing and implementing large-scale data pipelines
  • Hands-on expertise with AWS cloud technologies and distributed systems
  • Strong programming skills in PySpark, Python, and SQL
  • Data pipeline development using AWS tools (Step Functions, Airflow, etc.)
  • Distributed systems (S3, ADLS), and modern file/table formats (Parquet, Hudi)
  • ETL tools, orchestration technologies, and database design
  • Knowledge of regulatory standards like SOX, PCI, and HIPAA
  • Bachelor’s or Master’s in Computer Science, Information Systems, or related fields
  • AWS certification preferred
Job Responsibility
Job Responsibility
  • Design and implement scalable data pipelines for structured and unstructured data
  • Lead module architecture design for optimal performance, reliability, and maintainability
  • Mentor team members and ensure alignment with business goals
  • Collaborate with cross-functional teams to enable real-time and batch data processing
  • Ensure adherence to data security, privacy, and governance standards
  • Create and maintain technical documentation for architectures and decisions
  • Conduct code reviews and establish best practices
  • Develop monitoring systems for data pipeline reliability
  • Drive adoption of new technologies and workflows
  • Fulltime
Read More
Arrow Right

Recruitment Consultant

Recruitment Consultant – Further & Higher Education – The Protocol Group – Notti...
Location
Location
United Kingdom , Nottingham
Salary
Salary:
26000.00 - 29000.00 GBP / Year
affinityworkforce.com Logo
AFFINITY WORKFORCE
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in recruitment, ideally within the Further or Higher Education / public sector
  • Strong business development skills with a track record of winning new clients
  • Excellent communication and relationship-building abilities
  • A proactive, target-driven mindset with a passion for education
  • Knowledge of FE funding, safeguarding, and compliance is a plus
Job Responsibility
Job Responsibility
  • Develop and execute a business development strategy to grow the Further & Higher Education client portfolio
  • Build strong relationships with hiring managers and decision-makers across colleges and training providers
  • Source, attract, and place high-quality candidates in roles ranging from lecturers to support staff
  • Stay up to date with sector trends, funding changes, and compliance requirements
  • Collaborate with the wider recruitment team to ensure seamless service delivery
What we offer
What we offer
  • Competitive salary with uncapped commission structure
  • Flexible working arrangements
  • Ongoing training and professional development
  • Supportive team culture with clear progression pathways
  • Opportunity to make a real impact in the education sector
  • Fulltime
Read More
Arrow Right

Workday Testing Analyst

We are looking for a skilled Workday Testing Analyst to join our team in King of...
Location
Location
United States , King of Prussia, Pennsylvania
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience working with Workday, particularly in creating test data and developing test scripts
  • Familiarity with Workday purge functionality and data retention processes is highly desirable
  • Strong documentation and organizational skills with a focus on accuracy
  • Exceptional attention to detail, ensuring compliance and data integrity
  • Effective communication and collaboration skills to work efficiently in cross-functional teams
  • Ability to thrive in a fast-paced, structured environment while managing multiple priorities
  • Knowledge of system performance, scalability, and security testing
  • Experience in tracking and reporting test progress and addressing issues proactively
Job Responsibility
Job Responsibility
  • Develop and implement comprehensive test plans and procedures for Workday integrations and purge functionality
  • Create detailed test scripts and generate test data to ensure accurate validation of system processes
  • Oversee all testing activities, including execution, documentation, and tracking of issues
  • Validate data retention processes and purge logic within Workday to ensure compliance with organizational policies
  • Conduct usability testing to evaluate system performance, scalability, reliability, and security
  • Document and monitor testing progress and results, providing detailed reports to stakeholders
  • Collaborate closely with configuration and compliance teams to ensure thorough testing coverage
  • Participate in project meetings and deliver status updates to managers and team leads
  • Escalate any testing-related issues promptly to ensure timely resolution
  • Maintain a structured and organized approach to testing tasks, adhering to deadlines and project goals
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right

Car Delivery Driver

As a Car Delivery Driver, you’ll be the face of Domino’s in your local community...
Location
Location
United Kingdom , Carluke
Salary
Salary:
10.00 - 12.21 GBP / Hour
jobs.360resourcing.co.uk Logo
360 Resourcing Solutions
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Full, valid UK or International driving licence
  • Own a reliable, roadworthy car with valid MOT, tax, and insurance
  • Smartphone for delivery app use
  • Good knowledge of the local area
  • Friendly, reliable, and able to work independently
  • Able to work flexible hours, including evenings and weekends
Job Responsibility
Job Responsibility
  • Deliver pizzas and menu items to customers promptly and safely
  • Provide excellent customer service at every doorstep
  • Operate your vehicle in accordance with road safety regulations
  • Use the Domino’s Delivery App on your smartphone to manage orders
  • Support the in-store team during quieter delivery periods (e.g. cleaning, restocking, folding boxes)
  • Represent Domino’s positively in the community
What we offer
What we offer
  • Competitive hourly pay + per-delivery payment + tips
  • 28 days paid holiday per year (includes BH, pro rata for part time)
  • Flexible working hours to suit your lifestyle
  • Staff discount on our delicious food
  • Occasional Business Use insurance provided whilst out driving (store can provide more details)
  • Staff meals (conditions apply)
  • Company pension scheme (where eligible)
  • Family Leave policies in place
  • Paid training and clear career progression pathway with linked pay increases
  • Supportive, inclusive, and fun team environment
  • Parttime
Read More
Arrow Right

Senior Director Cloud Architecture

The Senior Director of Cloud Architecture will lead the cloud strategy and overs...
Location
Location
United States , Austin
Salary
Salary:
120000.00 - 220000.00 USD / Year
nttdata.com Logo
NTT DATA
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Must be a US citizen or Green card holder to proceed with applying
  • 5+ years of managerial experience in cloud architecture, engineering, or IT leadership
  • deep expertise in cloud technologies and architecture
  • 5+ years of experience in cloud architecture with deep expertise in at least two public cloud providers (Azure, AWS, GCP, and/or OCI)
  • 5+ years’ hands-on experience with cloud platforms (e.g., AWS EC2, S3, RDS, Kubernetes, Lambda) and cloud architecture
  • 5+ years supporting the State of Texas or comparable sized Public Sector organizations
  • Must have experience designing referenced architectures and public cloud deployment patterns used across large enterprise environments
Job Responsibility
Job Responsibility
  • Strategic Leadership: Oversee and guide every element of cloud systems analysis and strategic planning, covering both Infrastructure as a Service (IaaS) and Platform as a Service (PaaS)
  • Define the vision, mission, and objectives for the CCoE, including cloud adoption, migration, and innovation initiatives
  • Architecture & Design: Develop and communicate cloud architecture strategies aligned with enterprise objectives and client delivery needs
  • design secure, scalable, and resilient cloud architectures
  • Governance & Compliance: Oversee governance, audit standards, controls, and processes to monitor and optimize cloud infrastructure
  • ensure adherence to frameworks like CIS, HIPAA, and NIST
  • Review and approve cloud-native tool usage, standardize platform components, and automate deployment processes
  • Technology Oversight: Maintain deep knowledge of cloud products, technologies, and best practices
  • evaluate and implement solutions to meet organizational needs
  • Identify opportunities to modernize systems and integrate new cloud technologies
What we offer
What we offer
  • medical, dental, and vision insurance
  • flexible spending or health savings account
  • life and AD&D insurance
  • short- and long-term disability coverage
  • paid time off
  • employee assistance
  • participation in a 401k program with company match
  • additional voluntary or legally required benefits
  • Fulltime
Read More
Arrow Right

Full Charge Bookkeeper

We are looking for a skilled Full Charge Bookkeeper to join our team in Boca Rat...
Location
Location
United States , Boca Raton
Salary
Salary:
Not provided
https://www.roberthalf.com Logo
Robert Half
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience as a Full Charge Bookkeeper with expertise in managing complex accounts
  • Proficiency in QuickBooks Online and payroll systems, such as ADP
  • Strong knowledge of accounts payable, accounts receivable, and bank reconciliations
  • Ability to handle high transaction volumes efficiently and accurately
  • Excellent organizational skills and attention to detail
  • Strong communication skills to work effectively with management and team members
  • A proactive approach to identifying and resolving financial discrepancies
Job Responsibility
Job Responsibility
  • Manage and maintain comprehensive bookkeeping functions for two convenience store locations
  • Utilize QuickBooks Online and payroll software to handle all financial transactions and processes
  • Oversee accounts payable and receivable, ensuring timely and accurate processing
  • Perform regular bank reconciliations to ensure account accuracy and resolve discrepancies
  • Monitor and manage approximately 6,700 transactions annually, averaging 25 per day
  • Ensure automated systems are functioning correctly and optimize workflows where necessary
  • Collaborate with management to provide financial insights and reports
  • Maintain compliance with financial regulations and company policies
  • Troubleshoot and resolve any issues related to payroll systems and financial software
  • Support business operations with detailed record-keeping and clear communication about financial performance
What we offer
What we offer
  • medical, vision, dental, and life and disability insurance
  • eligible to enroll in our company 401(k) plan
Read More
Arrow Right