CrawlJobs Logo

Manager, Content Engineering — AI Content Understanding

meta.com Logo

Meta

Location Icon

Location:
United States , Menlo Park

Category Icon

Job Type Icon

Contract Type:
Employment contract

Salary Icon

Salary:

162000.00 - 227000.00 USD / Year

Job Description:

Product Content Engineering is a horizontal function supporting initiatives across Meta's Family of Apps. We partner closely with product and technical teams to solve problems by providing content-centered solutions, setting standards of quality, and building the frameworks that ensure AI-powered experiences actually work for people. We're looking for a people manager to lead a team within PCE's AI Content Understanding group. This team is responsible for executing evaluations and improving the quality of Meta's content understanding models — the AI systems that understand what content is about and how it should be ranked, recommended, and moderated. The work spans content quality, AI evaluation, and the search and recommendation systems that power Meta's products — building the frameworks, rubrics, and pipelines that hold AI outputs to a high standard. Your team assesses model behavior, identifies where it falls short, and works cross-functionally with engineering, product, research, and data science teams to make it better. Reporting to the AI CU lead, you'll manage a team of Content Engineers and contingent workers, owning the day-to-day execution of human evaluations, golden dataset creation, auto-eval calibration, and cross-functional delivery with engineering, data science, and product partners. You'll drive operational rigor while navigating a fast-moving environment where evaluation frameworks, model capabilities, and team processes are evolving simultaneously.

Job Responsibility:

  • Manage and develop a team of Content Engineers and contingent workers, setting clear goals, providing regular feedback, and supporting career growth
  • Own the execution of continuous CU model evaluations — coordinating sprint planning, reviewer assignments, QA processes, and delivery timelines across multiple concurrent workstreams
  • Drive the creation and maintenance of golden datasets that serve as ground truth for model benchmarking and auto-eval calibration
  • Partner with engineering, data science, and product teams to translate evaluation insights into actionable recommendations for model improvement and prompt optimization
  • Lead the team's contribution to LLM-as-a-Judge (auto-eval) initiatives — ensuring human evaluation data is used to calibrate, validate, and improve automated evaluation systems
  • Define and maintain evaluation guidelines, rubrics, and quality standards in partnership with Lead Content Engineers, ensuring consistency across reviewers and use cases
  • Build repeatable operational processes for evaluation sprints, including reviewer training, calibration sessions, and escalation workflows
  • Manage CW workforce planning — hiring, onboarding, allocation across workstreams, and performance management
  • Synthesize evaluation results into structured reports and present findings to cross-functional leadership, including engineering leads and lead product stakeholders
  • Identify and mitigate operational risks — staffing gaps, timeline conflicts, quality regressions — before they impact delivery

Requirements:

  • 8+ years of experience in content strategy, content operations, AI evaluation, or a related field
  • 2+ years of people management experience, including hiring, developing, and performance-managing direct reports
  • Experience managing cross-functional programs with engineering, data science, and product partners in fast-paced environments
  • 1+ years working with generative AI products, AI evaluation, prompt engineering, annotation, and/or content labeling and analysis
  • Experience designing or operationalizing evaluation frameworks, annotation guidelines, or quality rubrics for AI/ML systems
  • Demonstrated ability to manage multiple concurrent workstreams with competing priorities and tight deadlines
  • Proven analytical skills with experience interpreting evaluation data and communicating findings to technical and non-technical audiences
  • Track record of building team operational processes and quality standards from the ground up or during periods of significant change

Nice to have:

  • Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
  • Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
  • Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
  • Experience managing contingent worker (CW) teams, vendor relationships, or scaled annotation operations
  • Familiarity with AI evaluation methods such as human eval, LLM-as-judge, model benchmarking, A/B testing, or red-teaming
  • Experience with Python, SQL, or other tools for data analysis and evaluation automation
  • Background in content understanding, search quality, recommendation systems, or trust and safety
  • Experience managing through organizational transitions, including shifts from manual to automated workflows
  • BA or BS in Computer Science, Data Science, Linguistics, or related field
What we offer:
  • bonus
  • equity
  • benefits

Additional Information:

Job Posted:
May 05, 2026

Employment Type:
Fulltime
Work Type:
On-site work
Job Link Share:

Looking for more opportunities? Search for other job offers that match your skills and interests.

Briefcase Icon

Similar Jobs for Manager, Content Engineering — AI Content Understanding

AI Engineer

AI Engineer position at Inetum, a European leader in digital services, focusing ...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 3 to 5 years of experience in AI solutions deployment in enterprise environments
  • Understanding of how LLMs (GPT-4, Gemini, Claude, Llama, Mistral AI) work, their capabilities, and limitations
  • Familiarity with model architecture, tokenization, context windows, and prompt formatting
  • Crafting effective prompts for various tasks (text generation, summarization, Q&A, code generation, images and sound manipulations)
  • Techniques for prompt chaining, few-shot and zero-shot learning, and multi-turn conversations
  • Knowledge of prompt templates, system instructions, and role-based prompting
  • Understanding the concept of AI agents: autonomous entities that perceive, reason, and act to achieve goals
  • Familiarity with multi-agent systems, agent orchestration, and agentic workflows (e.g., using frameworks like Lang Chain, Crew AI, Auto Gen)
  • Ability to design, prompt, and coordinate groups of AI agents for collaborative or competitive tasks
  • Knowledge of agent communication, delegation, and task decomposition
Job Responsibility
Job Responsibility
  • Develop, refine, and optimize prompts for LLMs (GPT-4, Gemini, Claude, Llama, Mistral) to support a variety of tasks such as text generation, summarization, Q&A, and code generation
  • Design and implement prompt strategies for multi-turn conversations, prompt chaining, and role-based instructions
  • Build and coordinate groups of AI agents (multi-agent systems) for collaborative or competitive tasks using frameworks such as Lang Chain, Crew AI, or Auto Gen
  • Upgrade existing prompts while releases and solutions set evolve
  • Evaluate and improve the effectiveness of prompts and agent workflows through iterative testing, A/B experimentation, and performance analysis
  • Collaborate with cross-functional teams (developers, data scientists, product managers) to integrate prompt engineering and agentic workflows into enterprise solutions
  • Ensure compliance with data privacy, security, and regulatory standards (GDPR, NIS2) in all prompt and agent designs
  • Document prompt strategies, agent architectures, and best practices for internal knowledge sharing and training
  • Ensure provided solutions are ready for production, documented and monitored ensuring consistent delivery and quality
What we offer
What we offer
  • Full access to foreign language learning platform
  • Personalized access to tech learning platforms
  • Tailored workshops and trainings to sustain your growth
  • Medical Insurance
  • Meal tickets
  • Monthly budget to allocate on flexible benefit platform
  • Access to 7 Card services
  • Wellbeing activities and gatherings
  • Fulltime
Read More
Arrow Right

AI Engineer

AI Engineer position at Inetum, a European leader in digital services, focusing ...
Location
Location
Romania , Bucharest
Salary
Salary:
Not provided
https://www.inetum.com Logo
Inetum
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Minimum 3 to 5 years of experience in AI solutions deployment in enterprise environments
  • Understanding of how LLMs (GPT-4, Gemini, Claude, Llama, Mistral AI) work, their capabilities, and limitations
  • Familiarity with model architecture, tokenization, context windows, and prompt formatting
  • Crafting effective prompts for various tasks (text generation, summarization, Q&A, code generation, images and sound manipulations)
  • Techniques for prompt chaining, few-shot and zero-shot learning, and multi-turn conversations
  • Knowledge of prompt templates, system instructions, and role-based prompting
  • Understanding the concept of AI agents: autonomous entities that perceive, reason, and act to achieve goals
  • Familiarity with multi-agent systems, agent orchestration, and agentic workflows (e.g., using frameworks like Lang Chain, Crew AI, Auto Gen)
  • Ability to design, prompt, and coordinate groups of AI agents for collaborative or competitive tasks
  • Knowledge of agent communication, delegation, and task decomposition
Job Responsibility
Job Responsibility
  • Develop, refine, and optimize prompts for LLMs (GPT-4, Gemini, Claude, Llama, Mistral) to support a variety of tasks such as text generation, summarization, Q&A, and code generation
  • Design and implement prompt strategies for multi-turn conversations, prompt chaining, and role-based instructions
  • Build and coordinate groups of AI agents (multi-agent systems) for collaborative or competitive tasks using frameworks such as Lang Chain, Crew AI, or Auto Gen
  • Upgrade existing prompts while releases and solutions set evolve
  • Evaluate and improve the effectiveness of prompts and agent workflows through iterative testing, A/B experimentation, and performance analysis
  • Collaborate with cross-functional teams (developers, data scientists, product managers) to integrate prompt engineering and agentic workflows into enterprise solutions
  • Ensure compliance with data privacy, security, and regulatory standards (GDPR, NIS2) in all prompt and agent designs
  • Document prompt strategies, agent architectures, and best practices for internal knowledge sharing and training
  • Ensure provided solutions are ready for production, documented and monitored ensuring consistent delivery and quality
What we offer
What we offer
  • Full access to foreign language learning platform
  • Personalized access to tech learning platforms
  • Tailored workshops and trainings to sustain your growth
  • Medical Insurance
  • Meal tickets
  • Monthly budget to allocate on flexible benefit platform
  • Access to 7 Card services
  • Wellbeing activities and gatherings
  • Fulltime
Read More
Arrow Right

Sr. Manager, Software Engineering (Search)

As a Senior Engineering Manager – Search, you will lead and inspire a talented t...
Location
Location
India , Hyderabad
Salary
Salary:
Not provided
highspot.com Logo
Highspot
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 3+ years of experience managing engineering teams, with a proven record of developing and scaling backend or search systems
  • 7+ years of total software development experience with cloud-native SaaS platforms
  • Strong background in search & recommendation technologies such as Lucene, Solr, Opensearch, Elasticsearch, RAG, or similar frameworks
  • Deep understanding of enterprise search architecture, schema design, and relevance tuning
  • Proven success building REST APIs, distributed systems, and integrating services using AWS or similar cloud platforms
  • Experience with object-oriented and functional programming languages, such as JavaScript/TypeScript, Python, or Ruby
  • Familiarity with machine learning and AI concepts for ranking, personalization, or content recommendations
  • Track record of attracting and developing diverse talent, fostering a collaborative and inclusive culture
  • Strong leadership, communication, and stakeholder management skills able to balance technical depth with strategic decision-making
Job Responsibility
Job Responsibility
  • Lead, mentor, and grow a team of search and backend engineers focused on high-impact, scalable search solutions
  • Own the technical vision for search architecture combining traditional and vector based, including relevance, ranking models, and distributed indexing systems
  • Drive execution excellence — set goals, manage delivery timelines, and ensure consistent progress against engineering objectives
  • Collaborate with Product and Data Science to translate customer and business needs into measurable search and content recommendation improvements
  • Optimize and scale our enterprise search stack (Lucene, Solr, ZooKeeper, or similar technologies) to support massive data volumes
  • Oversee the design and delivery of highly available distributed services and RESTful APIs integrated into Highspot’s platform
  • Partner with DevOps to ensure reliability, observability, and performance across multiple data centers
  • Champion AI-driven enhancements to improve personalization, ranking, and search recommendations
  • Foster a culture of quality, inclusion, and accountability, emphasizing mentorship, continuous learning, and technical excellence
  • Partner cross-functionally to ensure alignment between platform strategy and product outcomes, including stakeholder communication and risk management
  • Fulltime
Read More
Arrow Right

Lead Content Designer, Confluence AI

As a Lead Content Designer in the AI and Horizontals organization, you'll help s...
Location
Location
United States , San Francisco; Mountain View; Seattle
Salary
Salary:
145200.00 - 233300.00 USD / Year
https://www.atlassian.com Logo
Atlassian
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 8+ years if experience in a content design role, designing complex or enterprise-level software
  • Demonstrated experience designing for AI (or related) features
  • Experience creating and delivering content strategy which impacts multiple teams or departments
  • Experience delivering end-to-end content design outcomes from ideation through to delivery, both within product and for supporting documentation
  • Passion for storytelling and experience bringing design direction to life and influencing cross-functional stakeholders
  • Experience coaching and leading other content designers by providing critique and supporting effective design operations to generate higher quality outcomes for users
  • Facilitation and leadership skills - you can work dynamically across multiple teams and align cross-functional partners around clear goals and design solutions
  • Experience balancing customer needs and business goals
  • Experience in design systems, identifying and creating common content patterns and frameworks requires to support a range of customer use cases
  • The ability to work in design tools like Figma to collaborate with product designers on projects.
Job Responsibility
Job Responsibility
  • Lead the strategic vision and content design outcomes, providing clear direction for Confluence AI experiences
  • Apply your systems thinking and ability to understand complex journeys to drive better content design outcomes for users
  • Be a content design craft leader to coach other designers and ensure consistent and cohesive experiences across Confluence, including leading design reviews
  • Have expertise in content strategy, content operations and effective communications which elevates your influence and impact
  • Lead and deliver design projects end-to-end, working across various teams and stakeholders and tying work back to customer value and outcomes
  • Drive research and identify success metrics, ensuring design solutions continue to deliver customer value
  • Collaborate with some of the industry’s smartest (and nicest) people. You’ll work with Engineers, Product Managers, Products Designers, Researchers, Marketing Experts ad Analysts to advocate for the user experience in our products and ship high quality work.
What we offer
What we offer
  • health coverage
  • paid volunteer days
  • wellness resources
  • Fulltime
Read More
Arrow Right

Product Manager, AI Data Pipelines & Developer Community

As a Product Manager for AI Data Pipelines & Developer Community, you will be in...
Location
Location
United States , Campbell
Salary
Salary:
Not provided
komprise.com Logo
Komprise, Inc.
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Proven experience in product management, particularly with a focus on data and AI-driven products
  • Strong understanding of technical data architectures, machine learning models, and AI applications in a business context
  • Demonstrated success in growing technical product adoption and developer communities
  • Experience working with large datasets, data pipelines, and data platforms
  • Excellent communication and interpersonal skills, with the ability to convey complex ideas to both technical and non-technical audiences
  • Proven ability to think strategically and execute methodically, translating complex problems into actionable solutions
  • A Bachelor’s degree in Computer Science, Data Science, Business, or a related field is preferred
Job Responsibility
Job Responsibility
  • Define and Drive Product Strategy: Develop and communicate a clear vision and roadmap for the AI data pipeline platform and associated developer tools, ensuring alignment with business goals and market needs
  • Build and Optimize AI Data Pipelines: Work closely with our product team and IT, end-users, data engineers and data scientists from our customers to build, optimize, and maintain scalable data pipelines that allow them to enrich, classify, curate data for AI agents for augmentation and inferencing
  • Translate User Needs into Product Requirements: Gather and prioritize user requirements, translating them into detailed product specifications and coordinating development efforts with engineering and data science teams
  • Champion Developer Experience: Understand the needs of the developer community, both internal and external, and develop tools, resources, and programs that enhance their experience building with our AI data platform. Investigate participation in partner developer communities and open-source communities
  • Grow and Engage the Developer Community: Lead efforts to foster a vibrant developer community, potentially through events, forums, technical content, and strategic outreach
  • Measure and Analyze Product Performance: Define and track key metrics to measure the success and impact of both the AI data platform and community engagement efforts, using data-driven insights to iterate and improve
  • Cross-Functional Collaboration: Partner effectively with cross-functional teams, including engineering, marketing, sales, and customer success, to ensure product vision alignment and drive successful execution
  • Stay Ahead of the Curve: Monitor industry trends, emerging technologies, and competitive offerings related to AI, data infrastructure, and developer ecosystems
Read More
Arrow Right

UI/UX Design Manager, AI-First

We are seeking a visionary and results-oriented AI-First UI/UX Design Manager to...
Location
Location
United Arab Emirates , Dubai
Salary
Salary:
25000.00 - 35000.00 AED / Month
puffy.com Logo
Puffy
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 7–10 years in web design for e-commerce
  • 4+ years leading, mentoring, and managing the operations of a design team
  • A portfolio that shows shipped PDP/PLP/Checkout improvements tied to clear business metrics
  • Expert in Figma (components, variables, Dev Mode)
  • strong visual/motion craft and responsive design sensibility
  • Proven experience implementing GenAI tools into a professional design pipeline, with a clear understanding of prompt engineering, quality control, and brand governance
  • Familiarity with AI-shaped UX patterns: conversational search, assistant UI, recommendation modules, and their measurement
  • Comfort collaborating on experiments (A/B vs multi-armed bandits) with Growth
  • understands trade-offs between optimization speed and causal certainty
  • Working knowledge of WCAG 2.2 and SEO fundamentals for structured content and schema on e-commerce sites
Job Responsibility
Job Responsibility
  • AI-Powered Creative Operations: Use tools like Figma AI and Adobe Firefly to rapidly explore page layouts, hero concepts, PDP modules, and image/copy variants — always with human QA and brand guardrails
  • Design for AI search & assistants: Create web experiences that work with conversational shopping (natural-language search, guided finders, Q&A on PDPs)
  • Partner with engineering to implement semantic/LLM search UIs and chat patterns inspired by market leaders
  • Personalization & recommendations (design side): Own the UI patterns (placements, card density, explainer microcopy, “why this” affordances) for recommendations on-site
  • Collaborate on rules/guardrails
  • validate with staged rollouts
  • AI-driven experimentation (design practice): When surfaces are evergreen (banners, placements, recommendation modules), partner with Growth to use bandit allocation for faster wins
  • keep classic A/B for high-stakes causal changes
  • Drive Efficiency: Develop, implement, and enforce an AI-first workflow for the design team
  • Systematically target and eliminate workflow bottlenecks
What we offer
What we offer
  • Continuous learning
  • 10% monthly bonus
  • Premium insurance
  • Achievement recognition
  • Free snacks and lunches
  • Generous annual leave
  • AI tools and tech stack
  • 18+ nationalities
  • Social events & activities
  • No income tax
  • Fulltime
Read More
Arrow Right

Senior Generative AI Engineer

The Citi Innovation Lab is a leader in creating new ideas, innovative technology...
Location
Location
Israel , Tel Aviv
Salary
Salary:
Not provided
https://www.citi.com/ Logo
Citi
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • Hands-on experience with transformer-based models and their applications
  • Strong understanding of LLM, LLM model selection, benchmarking, and optimization
  • Experience with RAG systems and vector databases
  • Proficiency in developing and deploying AI agents
  • Knowledge of open-source models and methods, including benchmarks for evaluating AI performance
  • Knowledge of security risks and mitigation strategies for autonomous AI agents, including OWASP guidelines
  • Proficiency in Python and experience with libraries such as Pandas, Tabula, and TensorFlow/PyTorch
  • Strong problem-solving skills and attention to detail
  • Excellent communication and documentation skills
Job Responsibility
Job Responsibility
  • Develop and implement enterprise scale cutting edge models such as visual document understanding and text2code
  • Implement and Optimize vector-based retrieval systems for RAG by covering embedding models, ANN indexing, hybrid search, and re-ranking
  • Implement autonomous AI agents to implement adaptive, error resistant data extraction, and content validation tasks
  • Develop and deploy enterprise software applications using state of the art practices, such as micro services, modular code, as well as proficiency in writing unit and integration tests to ensure the accuracy and reliability of the AI applications
  • Ensure data privacy and security in all AI-driven processes, adhering to OWASP guidelines and Citi’s stringent authentication and authorization policies
  • Collaborate with cross-functional teams to integrate AI solutions into existing workflows
  • Document the development process and create comprehensive technical specifications
  • Manage and maintain AI applications, ensuring best practices in model management and versioning
  • Deploy resulting AI applications using industrial strength framework and processes, including Kubernetes and OpenShift for scalable and efficient operations on-premises
  • Ability to research and develop and utilize transformer-based models for enhanced application performance
  • Fulltime
Read More
Arrow Right

AI Engagement Manager

We are looking for a highly-motivated senior IC who will work with some of Runwa...
Location
Location
United States , New York; San Francisco; Seattle
Salary
Salary:
145000.00 - 290000.00 USD / Year
runwayml.com Logo
Runway
Expiration Date
Until further notice
Flip Icon
Requirements
Requirements
  • 5+ years of overall experience in customer-facing roles, engaging C-level technical audiences with complex global organizations ideally within tech, finance, or consulting
  • Have led complex implementations of Generative AI/traditional ML solutions and can drive measurable business outcomes
  • Ability to work in a fast-paced, dynamic environment and manage multiple priorities with expansive ownership and ruthlessly prioritize
  • A strong bias towards action and belief that no task is too small for you
  • Have a track record of rapidly prototyping ideas and a willingness to work iteratively with your customers to deliver the right solution that solves their problem
  • Possess exceptional presentation and communication skills, particularly when engaging with executives and leaders
  • High technical aptitude for understanding complex technologies, tools, systems, and issues
  • Mindset to think creatively and think beyond past checklists and playbooks
  • Highly resourceful with the ability to create structure and work autonomously amidst ambiguity
  • Experience with data tooling and BI systems (SQL, BigQuery, Retool, etc)
Job Responsibility
Job Responsibility
  • Own and maintain strong relationships with our top partners ensuring long-term alignment and collaboration
  • Get embedded within accounts to learn and understand their (content) pain points and challenges today, then propose discrete ways in which Runway can solve those challenges
  • Identify and prototype specific and highly targeted new markets and new opportunities for our products
  • Develop a deep, functional and technical understanding of our products and future roadmap
  • Be hands-on in quickly onboarding new partners to our platform and/or API while also supporting their long-term expansion and technical understanding of how to leverage Runway
  • Provide high-impact feedback to Product, Engineering, and Creative which can in turn improve the product experience for our partners
  • Analyze, track, and report on the performance of existing partnerships leveraging BI tools and data
  • Fulltime
Read More
Arrow Right