Job Description

Position: Content Quality Reviewer

Date Posted: March 22, 2026

Industry: Information Technology / AI / Data Annotation

Employment Type: Contract (Full Time or Part Time)

Experience: Not Specified

Qualification: Bachelor’s Degree

Salary: $23 per hour

Location: United States (REMOTE)

Company: Mercor

Description:
Mercor is a global talent platform that connects highly skilled professionals with leading AI research organizations. Based in San Francisco and backed by top-tier investors, the company focuses on advancing artificial intelligence by leveraging expert human input to improve model performance.

Mercor is currently hiring Content Quality Reviewers, also referred to as Language Model Evaluators, to assess and enhance AI-generated responses. This role involves reviewing outputs from large language models (LLMs) and ensuring they meet high standards of accuracy, clarity, and usefulness.

Key Responsibilities:

• Review and evaluate AI-generated responses to determine how effectively they address user queries
• Perform fact-checking using reliable public sources and external verification tools
• Provide structured feedback by annotating strengths, weaknesses, and factual inaccuracies in responses
• Analyze reasoning quality, clarity, tone, and completeness of generated content
• Ensure outputs follow conversational guidelines and align with expected AI behavior
• Apply consistent evaluation standards using predefined taxonomies, benchmarks, and scoring systems

Required Qualifications:

• Bachelor’s degree in any relevant field
• Native-level proficiency in Arabic (C2/ILR Level 5) with excellent written communication skills
• Hands-on experience using large language models (LLMs)
• Strong attention to detail and ability to work across diverse subject areas
• Analytical mindset with experience in structured problem-solving environments
• Solid understanding of college-level mathematics

Preferred Qualifications:

• Experience in RLHF (Reinforcement Learning with Human Feedback), model evaluation, or data annotation
• Background in writing or editing high-quality content
• Ability to compare multiple outputs and make detailed qualitative judgments
• Familiarity with evaluation frameworks, scoring rubrics, or benchmarking systems

Application Process:

• Submit your resume for initial review
• Complete an AI-based interview tailored to your background
• Fill out and submit the required application form

Additional Information:

• Applications are reviewed on a daily basis
• Candidates are encouraged to complete all steps, including the AI interview, to be considered promptly
• Support is available for applicants throughout the process

Disclaimer: The job details above are structured for clarity and understanding based on publicly available content from recruiters/Company pages. All rights remain with the original source; names may be withheld for confidentiality. We are not involved in the hiring process

Job Tags

Similar Jobs