Rubrics Expert Job at AquSag Technologies (Noida)

AI Research Engineer, Enterprise Evaluations

Scale AI is seeking a technically rigorous and driven AI Research Engineer to jo...

Location

United States , San Francisco; New York

Salary:

179400.00 - 224250.00 USD / Year

Scale

Expiration Date

Until further notice

Requirements

Bachelor’s degree in Computer Science, Electrical Engineering, a related field, or equivalent practical experience
2+ years of experience in Machine Learning or Applied Research, focused on applied ML systems or evaluation infrastructure
Hands-on experience with Large Language Models (LLMs) and Generative AI in professional or research environments
Strong understanding of frontier model evaluation methodologies and the current research landscape
Proficiency in Python and major ML frameworks (e.g., PyTorch, TensorFlow)
Solid engineering and statistical analysis foundation, with experience developing data-driven methods for assessing model quality

Job Responsibility

Partner with Scale’s Operations team and enterprise customers to translate ambiguity into structured evaluation data, guiding the creation and maintenance of gold-standard human-rated datasets and expert rubrics that anchor AI evaluation systems
Analyze feedback and collected data to identify patterns, refine evaluation frameworks, and establish iterative improvement loops that enhance the quality and relevance of human-curated assessments
Design, research, and develop LLM-as-a-Judge autorater frameworks and AI-assisted evaluation systems. This includes creating models that critique, grade, and explain agent outputs (e.g., RLAIF, model-judging-model setups), along with scalable evaluation pipelines and diagnostic tools
Pursue research initiatives that explore new methodologies for automatically analyzing, evaluating, and improving the behavior of enterprise agents, pushing the boundaries of how AI systems are assessed and optimized in real-world contexts

What we offer

Comprehensive health, dental and vision coverage
retirement benefits
a learning and development stipend
generous PTO
commuter stipend
equity grant

Fulltime

Manufacturing Standards Manager

Dandy is seeking a Manager of Manufacturing Quality to lead and scale manufactur...

Location

United States

Salary:

116450.00 - 150000.00 USD / Year

Dandy

Expiration Date

Until further notice

Requirements

8+ years in manufacturing quality, quality engineering, or technical operations (regulated manufacturing preferred)
Bachelor's degree in Engineering (Mechanical, Manufacturing, Industrial, Quality, Biomedical, etc.) or similar
Experience leading senior technical experts across multiple sites
Strong applied statistics experience (AAA, SPC, DOE)
Lean Six Sigma Green or Black Belt (or equivalent experience)
Proven ability to translate qualitative feedback into measurable standards
Willingness to travel ~30%

Job Responsibility

Lead and develop Principal Manufacturing Technicians across distributed sites
Drive consistent interpretation and application of manufacturing quality standards across locations
Define and maintain manufacturing quality rubrics that translate expert judgment into objective, auditable criteria
Convert clinician feedback (returns, remakes, qualitative notes) into quantitative specifications and CTQs (Critical to Quality)
Align manufacturing rubrics with design rubrics in partnership with the Manager of Design Quality
Lead Attribute Agreement Analysis (AAA) and calibration programs for inspection consistency
Establish SPC/control charts and action thresholds for key CTQs
Apply Lean Six Sigma methods to reduce variation and improve FPY
Partner with the Automation team to ensure end-of-line 3DQC measures the correct attributes and thresholds defined by Principal Technicians
Validate 3DQC performance (repeatability, drift, coverage) and ensure data is used for upstream improvement, not just sorting

What we offer

Healthcare
Dental
Mental health support
Parental planning resources
Retirement savings options
Generous paid time off

Fulltime

Learning Designer

We're looking for a Learning Designer to build the instructional infrastructure ...

Location

United States , San Francisco

Salary:

125000.00 USD / Year

Mercor

Expiration Date

Until further notice

Requirements

2+ years of experience in learning design, instructional design, curriculum development, or technical writing
Strong pedagogical foundation – you understand how people actually learn and can design experiences that build real skills, not just surface familiarity
Excellent writing and communication skills – you can make complex concepts clear and actionable for diverse audiences
Experience creating task documentation or procedural guides – you know how to write instructions that people can actually follow
Ability to diagnose instructional problems – when task quality suffers, you can identify whether it's a training gap, a documentation issue, or something else
Experience creating multi-format content (written documentation, video, interactive exercises, assessments)
Comfort with ambiguity and rapid iteration – we're building systems in a fast-moving environment, and you'll need to ship and improve continuously
Systems thinking orientation – you see how individual skills connect to broader operational excellence

Job Responsibility

Review and improve project guidelines, rubrics, and instruction documents to maximize clarity and task performance for our talent experts to complete data annotation and evaluation work
Transform complex task specifications into well-structured instructional materials that talent experts can follow accurately
Partner with project teams to identify where documentation gaps are causing quality issues and design solutions
Build centralized academy programs for complex data types related to post-training for LLMs
Create structured upskilling pathways that prepare talent experts for increasingly sophisticated project work
Develop assessments and certification frameworks that validate readiness for specific project types
Design and develop training curricula for internal Mercor employees across the full project lifecycle – from onboarding fundamentals to advanced client-facing skills
Build competency-based learning paths that progress from foundational concepts to applied expertise
Develop evaluation and certification systems to ensure consistent quality and skill development across teams

Fulltime

Well-Being Psychiatrist Expert

Join our team as a Well-Being Psychiatrists AI Trainer and help shape the future...

Location

India , Noida

Salary:

Not provided

AquSag Technologies

Expiration Date

Until further notice

Requirements

MD with a primary focus in psychiatry, and active licensure
Demonstrated experience practicing psychiatry in clinical or academic settings
Exceptional written and verbal communication skills with a genuine care for clarity and empathy
Strong analytical and problem-solving abilities relevant to mental health scenarios
Experience working with diverse teams and an enthusiasm for sharing psychiatric knowledge
High attention to detail and commitment to ethical standards in AI development
Comfort working in a remote, technology-driven environment with self-motivation

Job Responsibility

Design and develop detailed rubrics and frameworks for AI model training in psychiatric and well-being domains
Evaluate and provide expert feedback on AI-generated content, ensuring clinical accuracy and sensitivity
Collaborate closely with data scientists, AI engineers, and cross-functional specialists to optimize model performance
Contribute psychiatric insights to enhance AI understanding of mental health scenarios and patient care nuances
Continuously review and iterate on training materials and AI outputs to uphold high standards of quality and ethics
Document methodologies, edge cases, and improvements to facilitate knowledge transfer within the team
Champion clear, empathetic written and verbal communication in all interactions and documentation

Fulltime

Instructor, Academic Upgrading (Curriculum Development)

Keyano College is seeking qualified individuals to develop curriculum for Academ...

Location

Canada , Fort McMurray

Salary:

Not provided

Keyano College

Expiration Date

Until further notice

Requirements

Minimum of a Master’s in English, Mathematics, or a related field. A Bachelor’s degree with significant teaching experience with Alberta Education curriculum may be considered
Previous experience in developing program curriculum and content
Experience with the Moodle platform
Experience developing course content and delivering lessons aligned with the Alberta Government high school curriculum
Able to work independently to meet set timelines and produce deliverables with minimal supervision
Able to work collaboratively and accept constructive feedback in curriculum development and course development
Dependable and resourceful team member with a proven track record of managing multiple competing priorities with a calm professional approach
Commitment to inclusion and diversity

Job Responsibility

Map curriculum to requirements as outlined by Alberta Education and Keyano College curriculum development requirements
Develop course fundamentals including course description, measurable learning outcomes, course outline, and syllabi. Provide a detailed plan of the course structure, units/modules, topics, teaching strategies, assessment points, and instructional activities
Develop and curate teaching materials for each course module that align with the course content and objectives (presentations, slides, readings, videos, and interactive content)
Develop detailed lesson plans for each module including learning activities, content delivery methods, interaction points, and reflection or review of key concepts
Develop instructor guides including detailed instructions, notes, resources and best practices to help instructors effectively teach the course while adhering to the course’s learning objectives and structure
Develop an assessment structure and course learning outcomes. This includes a detailed assessment matrix, assessment samples and rubrics, quizzes, tests, projects/assignments, etc
Upload and organize all developed content into a LMS moodle course shell
Collaborate with subject matter experts (SMEs) to design and develop curricula
Maintain office hours and liaise with the Chair and Associate Dean regularly
Engage with community stakeholders and provincial stakeholders

What we offer

Placement on the wage grid is reflective upon educational and experiential qualifications

Research Engineer Intern

As a Research Engineer Intern at Mercor, you’ll work at the intersection of engi...

Location

United States , San Francisco

Salary:

40.00 - 75.00 USD / Hour

Mercor

Expiration Date

Until further notice

Requirements

Pursuing a degree in Computer Science or a related field (graduating 2025–2027)
Strong programming skills in Python, Go, or JavaScript, with an ability to write clean, reliable, production-grade code
Understanding of data structures, algorithms, backend systems, and core engineering fundamentals
Familiarity with APIs, SQL/NoSQL databases, and cloud platforms
Curiosity and passion for AI research, reinforcement learning, and fast-moving startups
Excitement to work in person and thrive in a high-intensity, high-ownership engineering environment

Job Responsibility

Work on post-training and RLVR pipelines to help Mercor understand how datasets impact model performance
Design and run reward-shaping experiments and algorithmic improvements (e.g., GRPO, DAPO) to improve LLM tool-use, agentic behavior, and real-world reasoning
Quantify data usability, quality, and uplift on key benchmarks
Build data generation and augmentation pipelines that scale with training needs
Create and refine rubrics, evaluators, and scoring frameworks that push the boundaries of what LLMs can learn
Collaborate closely with research engineers, applied AI teams, and experts producing data
Operate in a fast-paced, experimental research environment with rapid iteration cycles

What we offer

Competitive internship stipend
Mentorship from experienced engineers
Work on real, high-impact projects
$1K monthly stipend for meals
Free Equinox membership
Team events and offsites
Potential full-time return offer

Fulltime

Research Engineer

As a Research Engineer at Mercor, you’ll work at the intersection of engineering...

Location

United States , San Francisco

Salary:

130000.00 - 500000.00 USD / Year

Mercor

Expiration Date

Until further notice

Requirements

Strong applied research background, with a focus on post-training and/or model evaluation
Strong coding proficiency and hands-on experience working with machine learning models
Strong understanding of data structures, algorithms, backend systems, and core engineering fundamentals
Familiarity with APIs, SQL/NoSQL databases, and cloud platforms
Ability to reason deeply about model behavior, experimental results, and data quality
Excitement to work in person in San Francisco, five days a week (with optional remote Saturdays), and thrive in a high-intensity, high-ownership environment

Job Responsibility

Work on post-training and RLVR pipelines to understand how datasets, rewards, and training strategies impact model performance
Design and run reward-shaping experiments and algorithmic improvements (e.g., GRPO, DAPO) to improve LLM tool-use, agentic behavior, and real-world reasoning
Quantify data usability, quality, and performance uplift on key benchmarks
Build and maintain data generation and augmentation pipelines that scale with training needs
Create and refine rubrics, evaluators, and scoring frameworks that guide training and evaluation decisions
Build and operate LLM evaluation systems, benchmarks, and metrics at scale
Collaborate closely with AI researchers, applied AI teams, and experts producing training data
Operate in a fast-paced, experimental research environment with rapid iteration cycles and high ownership

What we offer

Generous equity grant vested over 4 years
A $20K relocation bonus (if moving to the Bay Area)
A $10K housing bonus (if you live within 0.5 miles of our office)
A $1K monthly stipend for meals
Free Equinox membership
Health insurance

Fulltime

Enrollment Communications Quality Specialist

Pansophic Learning is seeking an Enrollment Communications Quality Specialist to...

Location

United States

Salary:

50000.00 - 60000.00 USD / Year

EdTech Jobs

Expiration Date

Until further notice

Requirements

3–5+ years in quality assurance or related roles
experience in service or educational environments
strong desire to learn new technologies
quality assurance
coaching
training
analytical skills
effective communication
familiarity with technology
process improvement

Job Responsibility

Review and score team interactions
identify quality trends
develop QA tools
collaborate across teams to improve processes and training
design a new QA rubric and process
bring analytical rigor to surfacing trends
proactively recommend improvements to training and workflows
collaborate cross-functionally with Enrollment Leadership, School Operations and Analytics teams
drive strategic quality initiatives
serve as a quality expert and trusted partner

What we offer

medical insurance
dental insurance
vision insurance
401(k) retirement plan
life insurance
long-term disability insurance
short-term disability insurance
10 paid holidays annually
paid time off

Fulltime

Rubrics Expert

AquSag Technologies

Location:
India , Noida

Category:
Education

Contract Type:
Not provided

Salary:

Job Description:

Job Responsibility:

Requirements:

Nice to have:

Additional Information:

Job Posted:
March 21, 2026

Looking for more opportunities? Search for other job offers that match your skills and interests.

Similar Jobs for Rubrics Expert