This list contains only the countries for which job offers have been published in the selected language (e.g., in the French version, only job offers written in French are displayed, and in the English version, only those in English).
As a Staff Engineer on the Twilio Segment Data platform/ pipelines team, you’ll build and scale systems that process several hundred thousands of data points per second. You will lead the development of high-scale ingestion and data processing systems You'll be designing, operating and maintaining complex distributed systems, ensuring reliability, performance, and cost-efficiency while querying petabytes of data for our customer data platform (CDP).
Job Responsibility
Design and deliver robust, high-scale routing experiences for the Data platform/ pipelines team for Twilio Segment.
Ship features that opt for high availability and throughput with eventual consistency
Collaborate with engineering and product leads, as well as teams across Twilio Segment
Support the reliability and security of the platform
Build and optimize globally available and highly scalable distributed systems
Be able to act as a team Tech Lead as needed
Mentor other engineers on the team in technical architecture and design
Partner with application teams to deliver end to end customer success.
Requirements
Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
Hands-on experience with high-scale messaging/streaming systems (several thousand events/sec) and processing engines ( 1M+ events/sec).
8+ years of experience writing production-grade code in a modern programming language
Strong theoretical fundamentals and hands-on experience designing and implementing highly available and performant fault-tolerant distributed systems.
Experience programming in one or more of the following: Go, Java, Scala, or similar languages
Well-versed in concurrent programming, along with a solid grasp of Linux systems and networking concepts.
Experience operating large-scale, distributed systems on top of cloud infrastructure such as Amazon Web Services (AWS) or Google Cloud Platform (GCP)
Experience in message passing systems (e.g., Kafka, AWS Kinesis) and/or modern stream processing systems (e.g., Spark, Flink).
Have hands-on experience with container orchestration frameworks (e.g. Kubernetes, EKS, ECS)
Leverage best-in-class development productivity practices including AI tooling.
Experience shipping services (products) following CI/CD development paradigm.
Deep understanding of architectural patterns of high-scale web applications (e.g., well-designed APIs, high volume data pipelines, efficient algorithms)
Ideally domain expertise in the Modern Data stack with experience in developing cloud-based data solution components and architecture covering data ingestion, data processing and data storage
Have a track record of successfully leading teams, large projects, or owned and built an important, complex system end to end, delivered iteratively.
Excellent written and verbal technical communication skills to convey complex technical concepts effectively.
Nice to have
Proficiency with datastores, distributed caching, and container technologies (Docker/Kubernetes).
Experience with modern CI/CD pipelines and observability tools (Datadog, Grafana, Prometheus).
Familiarity with AI/ML technologies.
Experience managing and operating AWS infrastructure
Experience working in a geographically distributed team.