CrawlJobs Logo
Arena Intelligence, Inc. Logo Arena Intelligence, Inc. · IT - Software Development

Senior Software Engineer, Data Infrastructure

United States, Bay Area · Job Posted February 20, 2026
Apply Position
Job Link Share

Job Description

LMArena is seeking a Software Engineer to join our team and build the data pipelines and infrastructure that powers real-world AI evaluation. You'll play a crucial role in designing and building the data pipelines that process and analyze tens of millions user vote data, directly impacting how we understand and evaluate AI model performance. This role is ideal for someone who thrives in fast-moving environments and interested in building products to ensure accurate and fair evaluation of human preferences across different models, which will shape the direction of future AI development. As an early member of our data engineering team, you'll partner closely with researchers, engineers, and product leadership to retrieve valuable data and insights from human votes and feedback. You'll help us move fast while staying rigorous, improving data quality, scaling our infrastructure to new levels, and deepening our ability to compare frontier models and predict human preferences.

Job Responsibility

  • Design and build robust data pipelines to ingest, process, and transform user vote data to features essential for model performance evaluation
  • Collaborate with researchers and product leadership to understand product goals and necessary data
  • Design and implement solutions to generate result dashboards and reports, providing useful information for the public, model providers, and researchers
  • Ensure the integrity, data quality, and reliability of the pipelines
  • Scale our data infrastructure to accommodate increasing data volumes and evolving analytical needs

Requirements

  • 5+ years of experience in software engineering, with a dedicated focus on data engineering and big data technologies
  • Proficiency in SQL and at least one programming language commonly used for data analysis (Python (preferred), Scala, R)
  • Hands-on experience with data processing and pipeline frameworks (Apache Spark, Ray Data, etc.) and at least one popular big data analytics platform (Databricks, Snowflake)
  • Demonstrated experience in designing, implementing, optimizing, and debugging production data pipelines

Nice to have

  • Prior work in data analytics or datalake platforms
  • Experience in advanced data analysis tools, such as Delta lake, streaming tables
  • Exposure to machine learning is a plus

What we offer

  • Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs
  • The opportunity to work on cutting-edge AI with a small, mission-driven team
  • A culture that values transparency, trust, and community impact
Arena Intelligence, Inc. - All Job Offers

Looking for more opportunities?

Search for other job offers that match your skills and interests.

Suggested Tags: Frontend Developer Secretary Talent Manager Salesforce Administrator Data Analyst
Similar Jobs for

Senior Software Engineer, Data Infrastructure

8 matching positions

Senior Software Engineer, Data Infrastructure & AI

Fullstory Anywhere is one of Fullstory's three primary product verticals, and it...
Location
Location
United States , Atlanta
Salary
Salary:
160000.00 - 170000.00 USD / Year
fullstory.com Logo
Fullstory
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Significant experience building and operating high-throughput data pipelines (batch and/or streaming) in a major cloud platform, including work with cloud data warehouses like BigQuery, Snowflake, or Databricks.
  • Proficiency in Go, Python, Java or a similar language.
  • Hands-on experience with data transformation tooling such as dbt, with a strong understanding of data modeling and pipeline observability.
  • Familiarity with LLM integration patterns and evaluation approaches (e.g., LangSmith, Vertex AI, or comparable frameworks), or demonstrated ability to ramp quickly in applied AI.
  • A track record of owning major system areas end-to-end: driving architectural decisions, maintaining production health, and improving reliability over time.
Job Responsibility
Job Responsibility
  • Maintain, extend, and scale Go microservices that transform and deliver Fullstory session data into customer warehouses and power the team's MCP server that enables AI agent integrations.
  • Develop and maintain dbt models and pipeline orchestration to ensure timely, fault-tolerant data migrations across hundreds of customer destinations.
  • Define evaluation frameworks for LLM outputs using tools like Langsmith and Vertex AI, ensuring AI-powered customer agents produce accurate, useful results.
  • Investigate and resolve production incidents across the data pipeline, implementing systemic fixes that prevent entire classes of failure from recurring.
  • Write technical design documents that drive consensus on architectural changes, proactively surfacing scaling bottlenecks, edge cases, and cross-team dependencies.
  • Demonstrate sound technical judgment by de-risking work through spikes, taking on tech debt deliberately, and knowing when to escalate versus dig in.
What we offer
What we offer
  • Flexibility and Connection
  • flexible PTO policy
  • annual company-wide closure
  • Benefits
  • paid parental leave
  • Bereavement leave, including miscarriage/pregnancy loss
  • Learning opportunities
  • annual learning subsidy
  • Productivity support
  • monthly productivity stipend
  • Fulltime
Read More
Arrow Right

Senior .NET Software Engineer (Data Infrastructure)

At the core of Bentley's global infrastructure solutions lies a critical data se...
Location
Location
Lithuania , Vilnius; Kaunas
Salary
Salary:
4000.00 EUR / Month
bentley.com Logo
Bentley Systems
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • A Bachelor’s degree in Computer Science, Software Engineering, or a related field
  • At least 5 years of proven experience in software development with C#, .NET Core, and a strong understanding of OOP, data structures, and test frameworks
  • Expert-level, hands-on experience with major object storage platforms (Azure Blob Storage, Google Cloud Storage, AWS S3). This must include deep knowledge of storage usage optimization, lifecycle policies, and designing cost-efficient data access patterns
  • Solid experience with Docker and Kubernetes for deploying and managing containerized applications
  • Proven ability to write clean, maintainable, testable, and secure code, with an intuitive understanding of the long-term impact of architectural decisions
  • A self-motivated and proactive mindset, with the ability to work effectively as an individual contributor and as part of a high-performing team in an Agile/Scrum environment
  • Strong verbal and written communication skills in English
Job Responsibility
Job Responsibility
  • Architecting for Scale & Stability: Design and develop robust, event-driven cloud services and core components, with a primary focus on stability, performance, and long-term maintainability
  • Modernizing Our Storage Solutions: Implement and optimize solutions using the latest cloud object storage technologies (Google Cloud storage, Azure Blob, AWS S3, etc.) to enhance performance and cost-efficiency
  • Hands-On Implementation: Use the latest .NET development tools to turn complex architectural designs into high-quality, production-ready software
  • Championing DevOps & Automation: Develop and utilize fully automated CI/CD pipelines to deliver both application and infrastructure changes seamlessly and safely into production
  • Driving Technical Excellence: Mentor and share your deep expertise with colleagues, elevating the team's technical capabilities
  • Ensuring System Health: Support the existing code base, troubleshoot complex production issues, and collaborate across teams to ensure end-to-end service reliability
What we offer
What we offer
  • A great Team and culture
  • An exciting career as an integral part of a world-leading software company
  • An attractive salary and benefits package
  • A commitment to inclusion, belonging and colleague wellbeing
  • Training and professional development opportunities (certifications programs, conferences etc.)
  • Additional annual leave days and extra paid days for different occasions (marriage, moving day, bereavement leave etc.)
  • Health insurance package and accidents insurance 24/7
  • Referral program with bonuses
  • Extra paid day for volunteering in the organization of your choice
  • Ability to work from office or hybrid from home
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Platform, AI Infrastructure

We are building a large-scale, productized data platform that powers critical in...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter.
  • Strong programming experience in Python
  • Experience building and operating large-scale distributed systems
  • Hands-on experience with: Backend services or APIs (e.g., FastAPI, Flask, or similar)
  • Cloud-based infrastructure (Azure, AWS, or GCP)
  • Monitoring and observability systems (metrics, logging, alerting)
  • Experience designing systems with reliability, scalability, and operational clarity in mind
  • Proven ability to own and deliver production systems end-to-end
  • Ability to break down ambiguous problems, ask the right questions, and execute effectively
Job Responsibility
Job Responsibility
  • Design, build, and operate core components of a distributed data platform, including: Orchestration systems (e.g., Airflow or equivalent)
  • Backend services and APIs (Python/FastAPI or similar)
  • Monitoring, alerting, and reliability systems
  • Own the end-to-end lifecycle of platform components - from design through deployment, scaling, and maintenance
  • Ensure systems meet requirements for availability, performance, and data reliability at large scale
  • Define and enforce standardized patterns for infrastructure, deployment, and observability across the platform
  • Partner with data engineering teams to enable efficient, reliable data processing workflows
  • Diagnose and resolve complex issues in distributed systems, including performance bottlenecks and failure modes
  • Contribute to infrastructure-as-code and deployment systems to support reproducibility and operational excellence
  • Drive continuous improvements in system robustness, cost efficiency, and operational clarity
What we offer
What we offer
  • Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
  • Fulltime
Read More
Arrow Right

Senior Software Engineer and Principal Software Engineer - Power Point AI Team

The PowerPoint team is embarking on an exciting new chapter - evolving a product...
Location
Location
United States , Redmond
Salary
Salary:
119800.00 - 234700.00 USD / Year
https://www.microsoft.com/ Logo
Microsoft Corporation
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • Ability to meet Microsoft, customer and/or government security screening requirements are required for this role
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience
  • 8+ years of experience in backend service engineering, including work on high-scale infrastructures
  • Proficiency in one or more systems programming languages such as C#, C++
  • 1+ years of experience in software engineering, designing and developing systems (and APIs) that deploy and integrate with AI models
  • 2+ years of experience working with rich telemetry, making data driven decisions, and carrying out rapid experimentation
  • 2+ years of experience building software for scale, performance, and reliability
  • Academic or industry experience with building, finetuning, deploying or building eval-driven systems utilizing the models (any category)
Job Responsibility
Job Responsibility
  • Lead design and delivery of complex, scalable AI features ensuring resilience and exceptional user experience
  • Drive technical strategy and architecture decisions across multiple services, influencing partner teams and aligning with compliance and security requirements
  • Champion modern engineering practices, including AI-driven approaches, automation, and cloud-native patterns, across the full development lifecycle
  • Mentor and guide engineers, fostering technical excellence and continuous improvement in security, reliability, and performance
  • Collaborate cross-org to solve challenging technical problems, streamline processes, and reduce operational costs while improving live-site health
  • Design and implement scalable backend services optimized for machine learning workflows and large language model integration
  • Develop and maintain evaluation-driven systems that leverage text and multimodal inputs (e.g., images) to power visual-creation experiences
  • Build and optimize APIs and infrastructure to support high-performance model inference and experimentation at scale
  • Collaborate with product, ML, and design teams to integrate models into user-facing features, ensuring seamless functionality and performance
  • Conduct model evaluations and experiments, analyze results, and iterate on improvements to enhance accuracy and user experience
  • Fulltime
Read More
Arrow Right

Software Engineer - Data Infrastructure

As a Data Infrastructure Engineer at Luma, you will play a critical role in buil...
Location
Location
United States; United Kingdom , Palo Alto; London
Salary
Salary:
Not provided
lumalabs.ai Logo
Luma AI
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proficiency in Python (or similar languages with willingness to learn Python) and experience with large-scale, high-throughput data infrastructure
  • Familiarity with distributed computing frameworks (e.g., Ray, Spark, Beam)
  • Ability to design and optimize data pipelines for ML research and internal teams
  • Strong problem-solving skills and understanding of data engineering at scale
  • Collaborative, product-focused mindset
  • comfortable in fast-paced environments
  • Experience sourcing, integrating, and optimizing data from diverse and large datasets
  • Comfortable working in a fast-paced, product-focused environment with a strong execution mindset
  • Open to candidates across seniority levels, from mid-level individual contributors to senior engineers and managers.
Job Responsibility
Job Responsibility
  • Build and maintain scalable data infrastructure for high-throughput machine learning workflows
  • Collaborate with ML researchers and product teams to ensure data systems meet evolving needs
  • Develop and optimize large-scale data pipelines and batch processing jobs
  • Contribute to the architecture and implementation of reliable, high-performance data platforms
  • Integrate open-source tools and continuously improve data infrastructure through monitoring and tuning
  • Participate in cross-functional projects to improve data reliability, scalability, and operational excellence
  • Support the evaluation and adoption of new programming languages and frameworks relevant to data infrastructure
  • Engage in continuous improvement of data infrastructure through monitoring, troubleshooting, and performance tuning
  • Collaborate with research & engineering teams to help define and refine best practices for data infrastructure development
  • Fulltime
Read More
Arrow Right

Senior Software Engineer, Data

ABOUT US: The Ads Analytics team is the backbone of our advertising intelligence...
Location
Location
Austria , Vienna
Salary
Salary:
Not provided
sportradar.com Logo
Sportradar
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering experience with proven track record of leading complex data projects from conception to delivery
  • Exceptional communication skills and experience working in cross-functional teams with analysts, product managers, and business stakeholders
  • AWS & Data Engineering: Very strong hands-on experience with AWS services (S3, Lambda, Glue, Athena, Redshift, EMR, etc.) and proficiency with Apache Spark for large-scale data processing
  • Backend Development: Strong experience with Python for building data processing services and APIs, plus expert-level SQL for data processing and analytics
  • Infrastructure & DevOps: Hands-on experience with Docker, Terraform, and CI/CD pipelines with automation best practices for data systems
  • Clean Code Advocate: Strong commitment to writing clean, maintainable, well-documented code with comprehensive testing and deep knowledge of analytics/reporting requirements
  • Data Architecture: Experience designing scalable data architectures, data modeling, and optimizing data processing workflows
  • Dashboard Development: Experience creating and managing analytics dashboards in bi tools (Tableau, Qlik Sense, Quicksuite, Power BI) and data visualization solutions to present complex insights to stakeholders
  • GenAI Tools (beneficial): Experience working with contextual and prompt engineering with GitHub Copilot, Claude, or similar AI assistants
Job Responsibility
Job Responsibility
  • Scale & Performance Engineering: Processing and analyzing terabytes of advertising data with sub-second query performance while building and maintaining robust ETL pipelines using Spark and AWS services to handle massive data volumes daily
  • Data Pipeline Architecture & Development: Designing and building scalable data processing systems, developing backend APIs and microservices (Python or Go), architecting data flows that support both batch and real-time analytics requirements, and managing user-facing dashboards that visualize complex data insights
  • Infrastructure & Data Quality Operations: Implementing robust monitoring and alerting systems to detect data quality issues, managing AWS infrastructure using Terraform, implementing CI/CD best practices, and maintaining high coding standards across data processing systems
  • Cross-Functional Leadership & Collaboration: Leading large-scale data projects from requirements gathering through delivery, bridging technical implementation with business requirements, mentoring team members, and presenting technical concepts to stakeholders while challenging requirements constructively
  • End-to-End Data System Ownership: Taking complete ownership of complex data engineering projects while ensuring high availability and accuracy for both internal stakeholders and external clients, championing clean code principles, and serving as a knowledge leader who supports delivering the right data solutions
What we offer
What we offer
  • A collaborative environment with colleagues from all over the world (Engineering offices in Europe, Asia and US)
  • Ability to shape your own workday and career via a clearly defined professional and personal development plan
  • Opportunity to work with senior leadership, develop yourself and build your career within an inspiring and fast-growing company and digital sports environment
  • A vibrant and inclusive community, including Women in Tech and Pride groups which welcome all participants
  • A company culture that promotes social aspects, sports, physical exercise and fun
  • Innovative and cross-team challenges like ShipIt, office sports tournaments in Darts and Table Tennis and unique beer brewing competitions
  • Competitive salary and benefits (e.g. retirement pension and insurance plan)
  • Sportradar takes over the full costs of € 365.- for the Öffi-Ticket (Jahreskarte) for you
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data (Python, AWS)

Location
Location
Austria , Vienna
Salary
Salary:
Not provided
sportradar.com Logo
Sportradar
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of data engineering experience with proven track record of leading complex data projects from conception to delivery
  • Exceptional communication skills and experience working in cross-functional teams with analysts, product managers, and business stakeholders
  • AWS & Data Engineering: Very strong hands-on experience with AWS services (S3, Lambda, Glue, Athena, Redshift, EMR, etc.) and proficiency with Apache Spark for large-scale data processing
  • Backend Development: Strong experience with Python for building data processing services and APIs, plus expert-level SQL for data processing and analytics
  • Infrastructure & DevOps: Hands-on experience with Docker, Terraform, and CI/CD pipelines with automation best practices for data systems
  • Clean Code Advocate: Strong commitment to writing clean, maintainable, well-documented code with comprehensive testing and deep knowledge of analytics/reporting requirements
  • Data Architecture: Experience designing scalable data architectures, data modeling, and optimizing data processing workflows
  • Dashboard Development: Experience creating and managing analytics dashboards in bi tools (Tableau, Qlik Sense, Quicksuite, Power BI) and data visualization solutions to present complex insights to stakeholders
  • GenAI Tools (beneficial): Experience working with contextual and prompt engineering with GitHub Copilot, Claude, or similar AI assistants
Job Responsibility
Job Responsibility
  • Scale & Performance Engineering: Processing and analyzing terabytes of advertising data with sub-second query performance while building and maintaining robust ETL pipelines using Spark and AWS services to handle massive data volumes daily
  • Data Pipeline Architecture & Development: Designing and building scalable data processing systems, developing backend APIs and microservices (Python or Go), architecting data flows that support both batch and real-time analytics requirements, and managing user-facing dashboards that visualize complex data insights
  • Infrastructure & Data Quality Operations: Implementing robust monitoring and alerting systems to detect data quality issues, managing AWS infrastructure using Terraform, implementing CI/CD best practices, and maintaining high coding standards across data processing systems
  • Cross-Functional Leadership & Collaboration: Leading large-scale data projects from requirements gathering through delivery, bridging technical implementation with business requirements, mentoring team members, and presenting technical concepts to stakeholders while challenging requirements constructively
  • End-to-End Data System Ownership: Taking complete ownership of complex data engineering projects while ensuring high availability and accuracy for both internal stakeholders and external clients, championing clean code principles, and serving as a knowledge leader who supports delivering the right data solutions
What we offer
What we offer
  • Collaborative environment with colleagues from all over the world (Engineering offices in Europe, Asia and US)
  • Ability to shape your own workday and career via a clearly defined professional and personal development plan
  • Opportunity to work with senior leadership, develop yourself and build your career within an inspiring and fast-growing company and digital sports environment
  • Vibrant and inclusive community, including Women in Tech and Pride groups which welcome all participants
  • Company culture that promotes social aspects, sports, physical exercise and fun
  • Innovative and cross-team challenges like ShipIt, office sports tournaments in Darts and Table Tennis and unique beer brewing competitions
  • Competitive salary and benefits (e.g. retirement pension and insurance plan)
  • Sportradar takes over the full costs of € 365.- for the Öffi-Ticket (Jahreskarte) for you
  • Fulltime
Read More
Arrow Right

Senior Software Engineer - Data Products

We are seeking a highly skilled Senior Software Engineer with deep expertise in ...
Location
Location
India , Bengaluru
Salary
Salary:
Not provided
roku.com Logo
Roku
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Bachelor's degree in computer science, Engineering, or a related field (or equivalent experience).
  • 8+ years of experience in software and/or data engineering with expertise in big data technologies such as Apache Spark, Apache Airflow.
  • Expertise with at least one of the following Apache Druid, StarRocks, and Trino.
  • Strong understanding of SOLID principles and distributed systems architecture.
  • Proven experience in distributed data processing, data warehousing, and real-time data pipelines.
  • Advanced SQL skills, with expertise in query optimization for large datasets.
  • Exceptional problem-solving abilities and the capacity to work independently or collaboratively.
  • Excellent verbal and written communication skills.
  • Experience with cloud platforms such as AWS, GCP, or Azure, and containerization tools like Docker and Kubernetes. (preferred)
  • Familiarity with additional big data technologies, including Hadoop and Kafka.
Job Responsibility
Job Responsibility
  • Design and build APIs and backend services using Spring Boot to support data products and analytics workflows.
  • Write clean, maintainable, and efficient code, ensuring adherence to best practices through code reviews.
  • Design, develop, and maintain data pipelines and ETL workflows using Apache Spark and Apache Airflow.
  • Optimize data storage, retrieval, and processing systems to ensure reliability, scalability, and performance.
  • Develop and fine-tune complex queries and analytics solutions using Druid, Trino, and StarRocks for large-scale datasets.
  • Monitor, troubleshoot, and improve data systems to minimize downtime and maximize efficiency.
  • Partner with data scientists, software engineers, and other teams to deliver integrated, high-quality solutions.
  • Provide technical guidance and mentorship to junior engineers, promoting best practices in software and data engineering.
What we offer
What we offer
  • Global access to mental health and financial wellness support and resources.
  • Local benefits including statutory and voluntary benefits which may include healthcare (medical, dental, and vision), life, accident, disability, commuter, and retirement options (401(k)/pension).
  • Time off in accordance with local leave policies.
  • Fulltime
Read More
Arrow Right