This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
This role is categorized as hybrid. This means the successful candidate is expected to report to Austin, TX or Warren, MI three times per week, at minimum [or other frequency dictated by the business if more than 3 days]. General Motors uses telematics data as a cornerstone of its software-defined vehicle strategy, enhancing safety, security, and product development to create an exceptional customer experience. Analytical insights from this data lead to improvements in operational efficiency, maintenance, and vehicle performance as well as address the need for intelligent (ADAS), sustainable transportation (EV). Our analytical datastore team owns the platform that streams and stores this data from millions of vehicles around the globe. We seek a polyglot engineer with a strong background in data and software, experienced in designing and maintaining large-scale systems. This technical leadership role requires a passion for quality, efficiency, and reliability. As a senior data engineer, you'll be building the platform to enable thousands of use cases that require telematics data. You will play a key role in our team's success, supporting hundreds of internal analytics users and enhancing the experience for millions of retail and fleet customers. This is a high-visibility role!
Job Responsibility:
Develop a scalable data platform to support continuing increases in data volume and complexity
Develop and maintain data pipelines to ingest, process, and curate large volumes of data using Spark, Kafka, Event Hubs, Azure Data Lake Storage, or similar technologies
Support the infrastructure required for optimal ingestion, transformation, and storing of data
Collaborate with data scientists to support the needs of advanced ML development
Design and implement rigorous data observability, validation, governance, and quality checks
Mentor and support junior engineers by providing guidance, coaching and educational opportunities
Simplify, simplify, simplify… and document your work!
Requirements:
Bachelor's degree in Computer Science, Software Engineering, or a related field
5+ years of experience in data engineering including Python or Scala, SQL, and relational/non-relational data storage
3+ years of experience in distributed, petabyte-scale data processing using Spark as well as container orchestration technologies like Kubernetes
Strong with performance tuning strategies such as partitioning, clustering, caching, and various other serialization techniques
Strong with infrastructure as code (IaC) and CI/CD best practices
Proven cloud experience and familiarity with at least one cloud platform (Azure preferred)
Strong background in any of the following: data architecture, data governance, metadata management, data quality, observability, or MLOps
Knowledge in schema design and data contracts
Excellent teamwork and leadership skills to collaborate and influence across product, program, and engineering
Nice to have:
Master's degree in Computer Science, Software Engineering, or a related field
Experience in processing and storing various file formats, including video, audio and image
Experience with tools such as Databricks or Snowflake