CrawlJobs Logo

Software Engineer II (Backend + Data pipelines)

United States; Canada; Mexico, San Francisco 103500.00 - 196000.00 USD / Year · Job Posted February 18, 2026
Apply Position
Job Link Share

Job Description

We’re seeking a Software Engineer II with strong backend development experience and a passion for solving complex data challenges at scale. In this role, you’ll design, build, and optimize distributed systems that extract, enrich, and process metadata for a wide range of content. You’ll work closely with ML engineers, product managers, and cross-functional partners to integrate machine learning models and LLM-based services into production pipelines and deliver impactful, high-performance solutions. This role offers the opportunity to work on cutting-edge generative AI and metadata enrichment problems at a truly global scale.

Job Responsibility

  • Design and build scalable systems to extract, enrich, and process metadata from millions of documents, images, and audio content
  • Leverage LLMs to integrate capabilities like summarization, classification, extraction, and enrichment into metadata pipelines
  • Collaborate with cross-functional teams, including ML engineers and product managers, to deliver scalable, efficient, and reliable metadata solutions
  • Optimize and refactor existing systems for performance, scalability, and reliability
  • Ensure data accuracy, integrity, and quality through automated validation and monitoring
  • Participate in code reviews, ensuring best practices are followed and maintaining high-quality standards in the codebase
  • Manage and maintain data pipelines, security and infrastructure

Requirements

  • 5+ years of professional software engineering experience
  • Proficiency in Python, Scala, Ruby, or similar languages
  • Experience designing and building distributed systems at scale
  • Hands-on experience building, deploying, and optimizing solutions using ECS, EKS, or AWS Lambda
  • Experience with infrastructure-as-code tools like Terraform (or similar)
  • Experience working with a public cloud provider (AWS, Azure, or Google Cloud)
  • Familiarity with data processing frameworks like Spark or Databricks for large-scale workloads
  • Proven ability to test, profile, and optimize systems for performance, scalability, and reliability
  • Bachelor’s degree in Computer Science or equivalent professional experience

Nice to have

Experience working with LLMs or integrating ML models into production systems

What we offer

  • Healthcare Insurance Coverage (Medical/Dental/Vision): 100% paid for employees
  • 12 weeks paid parental leave
  • Short-term/long-term disability plans
  • 401k/RSP matching
  • Onboarding stipend for home office peripherals + accessories
  • Learning & Development allowance
  • Learning & Development programs
  • Quarterly stipend for Wellness, WiFi, etc.
  • Mental Health support & resources
  • Free subscription to the Scribd Inc. suite of products
  • Referral Bonuses
  • Book Benefit
  • Sabbaticals
  • Company-wide events
  • Team engagement budgets
  • Vacation & Personal Days
  • Paid Holidays (+ winter break)
  • Flexible Sick Time
  • Volunteer Day
  • Company-wide Employee Resource Groups and programs that foster an inclusive and diverse workplace
  • Access to AI Tools: We provide free access to best-in-class AI tools, empowering you to boost productivity, streamline workflows, and accelerate bold innovation

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Similar Jobs for

Software Engineer II (Backend + Data pipelines)

8 matching positions

Software Engineer II - Data

As an Engineer on the Data Intelligence team, you will be dealing with large-sca...
Location
Location
India , Bangalore
Salary
Salary:
Not provided
uber.com Logo
Uber
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of Data engineering experience
  • Demonstrated experience of working with large data volumes and backend services
  • Good working knowledge of SQL (mandatory) and any other languages (Java, Scala, Python)
  • Working Experience of ETL, Data pipelines, Data Lake, Data Modeling fundamentals
  • Good problem-solving and analytical skills
  • Good team player and collaboration skills
Job Responsibility
Job Responsibility
  • Responsible for defining the Source of Truth (SOT) and dataset design for multiple Uber teams
  • Identify unified data models, collaborating with Data Science teams
  • Streamline data processing of the original event sources and consolidate them in the source of truth event logs
  • Build and maintain real-time/batch data pipelines that can consolidate and clean up usage analytics
  • Build systems that monitor data losses from the different sources and improve the data quality
  • Own the data quality and reliability of the Tier-1 & Tier-2 datasets, including maintaining their SLAs, TTL, and consumption
  • Devise strategies to consolidate and compensate for the data losses by correlating different sources
  • Solve challenging data problems with cutting-edge design and algorithms
Read More
Arrow Right

Senior Software Engineer II - Backend - AI Search

AI is one of the fastest growing product areas in Seismic. We believe that AI, p...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
seismic.com Logo
Seismic
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7+ years of experience in software engineering and a proven track record of building and scaling microservices and working with data retrieval systems
  • 5+ Experience with C# and .NET, unit testing, object-oriented programming, and web services
  • 3+ Experience with Python, with the ability to work concurrently on Python and .NET repositories
  • 3+ Experience with Redis, including expertise in managing large-scale Redis clusters
  • 2+ Experience with PostgreSQL, including maintaining and performing tuning
  • Proficient in Test Driven Development (TDD) with hands-on experience using xUnit and Postman to develop automation test scripts
  • Experience with Infrastructure as Code (Terraform, Pulumi, etc.)
  • Experience with Event driven architectures with tools like Kafka
  • Experienced in container technologies such as Docker and proficient in microservice frameworks like Kubernetes (K8s)
  • Experienced in Continuous Integration and Continuous Deployment (CI/CD) with expertise in developing Jenkins pipelines using Scala
Job Responsibility
Job Responsibility
  • Design, develop, and maintain backend systems and services for search functionality, ensuring high performance, scalability, and reliability
  • Implement and optimize search and AI-driven semantic algorithms, indexing, and information retrieval techniques to enhance search accuracy and efficiency
  • Collaborate with data scientists, AI engineers, and product teams to integrate AI-driven search capabilities across the Seismic platform
  • Monitor and optimize search performance, addressing bottlenecks and ensuring low-latency query responses
  • Provide technical guidance and mentorship to junior engineers, promoting best practices in search backend development
  • Work closely with cross-functional and geographically distributed teams, including product managers, frontend engineers, and UX designers, to deliver seamless and intuitive search experiences
  • Stay updated with the latest trends and advancements in search technologies, conducting research and experimentation to drive innovation
  • Fulltime
Read More
Arrow Right

Software Engineer II - Backend - Search

Join us at Seismic, a cutting-edge technology company leading the way in the Saa...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
seismic.com Logo
Seismic
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of experience in software engineering, with experience contributing to backend systems, microservices, or data-driven applications
  • Experience with C# and .NET, including object-oriented programming, unit testing, and web services
  • Experience working with PostgreSQL or similar relational databases, including writing queries and understanding basic performance considerations
  • Familiarity with container technologies such as Docker and microservice-based architectures
  • Working knowledge of automated testing practices (e.g., xUnit, Postman)
  • Familiarity with CI/CD concepts and modern development pipelines
  • Familiarity with Python
  • Exposure to cloud platforms such as AWS, Google Cloud Platform (GCP), or Microsoft Azure
  • Basic familiarity with cloud-native services for data storage, compute, and monitoring
  • Experience working on SaaS applications or cloud-hosted services
Job Responsibility
Job Responsibility
  • Contribute to the design, development, and maintenance of backend systems and services supporting search functionality, ensuring performance, scalability, and reliability
  • Assist in implementing search and/or AI-related features, including indexing, retrieval, and ranking logic, to improve search accuracy and efficiency
  • Collaborate with engineers, AI partners, and product teams to integrate search and AI-driven capabilities across the Seismic platform
  • Participate in monitoring and performance tuning efforts, identifying routine bottlenecks and applying guided improvements to ensure acceptable query latency
  • Work closely with cross-functional and geographically distributed teams, including product managers, frontend engineers, and UX designers, to support seamless search experiences
  • Learn and apply new tools, technologies, and best practices related to search, backend development, and AI systems
  • Fulltime
Read More
Arrow Right

Software Engineer II - Finance Data & Experiences

Ready to shape the future of how Microsoft operates its ~$250B+ business portfol...
Location
Location
United States , Redmond
Salary
Salary:
100600.00 - 199000.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 5+ years of experience building modern web applications using Angular (15+) or React 18, with hands‑on experience in .NET, REST APIs, and SQL/Azure SQL
  • Experience with AI‑assisted development tools (e.g., GitHub Copilot or similar)
  • Experience with Azure cloud services and deploying applications in cloud‑native environments
  • Familiarity with CI/CD pipelines, DevOps practices, and automated testing
  • Experience with performance optimization, monitoring, and debugging in production
  • Collaboration skills and experience working in agile / scrum teams
Job Responsibility
Job Responsibility
  • Design and develop end‑to‑end features across front‑end, backend services, and data layers for web applications
  • Build responsive UI components using modern frameworks (e.g., Angular/React) and integrate them with backend APIs
  • Develop and maintain backend services and APIs using secure, scalable patterns and cloud‑native services
  • Own features from development to production, including testing, deployment, monitoring, and bug fixes
  • Write clean, maintainable, production‑quality code following security, performance, accessibility, and coding standards
  • Collaborate closely with Project Managers, designers, and engineers to translate requirements into working full‑stack solutions
  • Participate in code reviews, debugging, and continuous improvement, strengthening CI/CD, test coverage, and system reliability
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation
  • Fulltime
Read More
Arrow Right
New

Software Engineer II

Security represents the most critical priorities for our customers in a world aw...
Location
Location
United States , Redmond
Salary
Salary:
102100.00 - 202200.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Design, build, and scale backend services and data pipelines that power device and security intelligence
  • Contribute to the development of high-quality, durable data foundations used across Microsoft Security Bring together fragmented device and security signals from multiple systems into a consistent, reliable view
  • Define and implement data models, pipelines, and system contracts across large-scale distributed systems
  • Contribute to architectural decisions and tradeoffs that balance correctness, scalability, latency, and complexity
  • Own or contribute to key platform components, supporting design, delivery, and operational excellence
  • Embody our culture and values
  • Fulltime
Read More
Arrow Right
New

Data Engineer II

Data Engineering is the production engine behind Global Skilling's data-driven d...
Location
Location
United States , Multiple Locations
Salary
Salary:
102100.00 - 202200.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 1+ year(s) experience in business analytics, data science, software development, data modeling, or data engineering
  • OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 2+ years of experience in business analytics, data science, software development, data modeling, or data engineering
  • OR equivalent experience
  • Ability to meet Microsoft, customer, and/or government security screening requirements may be required for this role. These requirements include, but are not limited to, the following specialized security screenings: Microsoft Cloud Background Check — this position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Job Responsibility
Job Responsibility
  • Build and maintain production ETL pipelines ingesting data from 1st-party, 2nd-party, partner, and 3rd-party sources
  • Design and implement data-sharing patterns and access controls (RBAC) across organizational boundaries
  • Contribute to data mesh patterns, including data contracts, domain-owned datasets, and product-style publishing
  • Develop and maintain data governance solutions, including lineage, cataloging, data quality, and compliance
  • Design, build, and operate REST APIs and backend services that expose WISE datasets as data products
  • Define and manage API contracts, including schema design, versioning, and lifecycle management
  • Build serving layers supporting low-latency, request-response data access
  • Design and deliver Power BI semantic models, curated datasets, and reporting solutions
  • Build and support self-serve analytics experiences, including discoverability and performance optimization
  • Instrument pipelines and services through observability practices (metrics, logs, traces)
  • Fulltime
Read More
Arrow Right

Software Engineer II

Join us at Seismic, a cutting-edge technology company leading the way in the Saa...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
seismic.com Logo
Seismic
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 2+ years of professional experience in data engineering or a backend-heavy software engineering role
  • Expert-level Python coding skills (Must Have)
  • Deep, hands-on experience (Must Have) with LangChain or LangGraph
  • Proven experience (Must Have) implementing and tuning Vector Databases for high-volume RAG pipelines
  • Strong understanding of traditional data modeling, ETL/ELT processes, and working with SQL/NoSQL databases
  • Solid grasp of embedding models, tokenization, and modern information retrieval techniques
  • Thrive in fast-paced environments and enjoy staying updated with the rapidly evolving landscape of GenAI and search technologies
Job Responsibility
Job Responsibility
  • Architect AI Data Pipelines: Design and maintain robust data ingestion and transformation pipelines tailored for LLM training, fine-tuning, and Retrieval-Augmented Generation (RAG)
  • Build Agentic Workflows: Utilize LangGraph to develop complex, state-managed AI agents and cyclical workflows that enhance automated user interactions
  • Optimize RAG Systems: Architect the retrieval layer of our AI applications, implementing efficient document embedding strategies and semantic search
  • Manage Vector Infrastructure: Implement and optimize Vector Databases (e.g., Pinecone, Weaviate, or Milvus) to ensure high-performance data retrieval and storage
  • Scale Data Models: Create scalable data schemas that support both structured and unstructured data, ensuring seamless integration with our AI services
  • Performance Engineering: Identify and resolve latency bottlenecks in data retrieval and embedding generation to ensure real-time AI responsiveness
  • Collaborate Cross-Functionally: Partner with AI Researchers and Product Managers to transition experimental AI prototypes into production-ready data products
  • Fulltime
Read More
Arrow Right

Software Engineer II - Full Stack

Hurix is hiring a Software Engineer II – Full Stack for a confidential client. T...
Location
Location
Canada
Salary
Salary:
Not provided
hurix.com Logo
HurixDigital
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of experience in product or enterprise software development
  • Hands-on experience across the full software stack, including frontend, backend/API, databases, cloud infrastructure, and deployment workflows
  • Experience building full stack features using Angular or React, backend APIs using Python, Golang, or Node.js, and cloud infrastructure using AWS or Azure
  • Experience implementing accessible features in web applications
  • Strong technical abilities in software development, testing, monitoring, performance tuning, and deployment pipelines
  • Understanding of application security principles, data privacy considerations, and secure coding practices
  • Strong technical communication, collaboration, and problem-solving skills
  • Passion for technology and commitment to staying current with modern engineering practices and emerging capabilities
Job Responsibility
Job Responsibility
  • Design, develop, and maintain full stack web applications and platform features that are reliable, maintainable, and scalable
  • Build frontend features using modern frameworks such as Angular or React
  • Develop backend services and APIs using technologies such as Python, Node.js, or Golang
  • Work with relational databases such as MySQL and PostgreSQL, and NoSQL databases such as DynamoDB or similar platforms
  • Develop and maintain software solutions that align with accessibility standards, including WCAG 2.2 AA
  • Integrate accessibility best practices into code, testing, and design processes
  • Collaborate closely with product managers, data scientists, and engineering teams to deliver high-quality solutions
  • Explore opportunities to integrate AI capabilities into applications where they add clear value to the user experience
  • Implement application security best practices and support compliance with relevant data privacy and protection requirements
  • Contribute to CI/CD processes to enable smoother and more reliable releases
Read More
Arrow Right