Sr. SWE for Code Reviewing LLM Data Training

• Review and audit annotator evaluations of AI-generated Bash/PowerShell scripts
• Assess if the Bash/PowerShell code follows the prompt instructions, is functionally correct, and secure
• Validate code snippets using proof-of-work methodology
• Identify inaccuracies in annotator ratings or explanations
• Provide constructive feedback to maintain high annotation standards
• Work within Project Atlas guidelines for evaluation integrity and consistency

• 5–7+ years of experience in shell scripting, system administration, or DevOps with Bash/PowerShell
• Strong knowledge of Bash/PowerShell syntax, command-line utilities, debugging, edge cases, and testing
• Experience with both Unix/Linux and Windows environments
• Comfortable using code execution environments and debugging tools
• Excellent written communication and documentation skills
• Experience working with structured QA or annotation workflows
• English proficiency at B2, C1, C2, or Native level

• Experience in AI training, LLM evaluation, or model alignment
• Familiarity with annotation platforms
• Exposure to RLHF (Reinforcement Learning from Human Feedback) pipelines
• Background in automation, CI/CD pipelines, or infrastructure as code

Contributions are paid weekly once approved