This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
The Software Engineer shall possess extensive expertise in dataflow design, data transport mechanisms, and Apache Spark based distributed processing. In this role, the Software Engineer shall be responsible for designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable, and reliable processing of the organization’s analytics workloads.
Job Responsibility:
Designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable, and reliable processing of the organization’s analytics workloads
Requirements:
Master’s degree in computer science or related discipline plus five (5) years of experience as a SWE
Bachelor’s degree in computer science or related discipline plus seven (7) years of experience as a SWE
Nine (9) years of experience as a SWE
Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Recent software development experience using Python and Java
Experience using Apache Airflow (DAG design, scheduling, operators, sensors)
Experience with Distributed Big Data processing engines including Apache Spark
Familiar with SQL technologies such as MySQL, MariaDB, and PostgreSQL
Experience using Jupyter Notebook
Experience with data wrangling and preprocessing using tools such as pandas, NumPy
Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML
Familiarity with data quality concepts, data validation, and anomaly detection
Experience with Git Source Control System
Nice to have:
Familiar with HPC Job Scheduling tools including Slurm
Experience using the Atlassian Tool Suite (JIRA, Confluence)