Remote | AI Data Quality & Evaluation Specialist — $45–$50/hour
24-MAG · New York, NY
$45–$50/hrfrom the description
Jul 22, 2026
New York, NY
Jul 23, 2026
What this job asks for AI summary
A part-time, fully remote contract role focused on reviewing and annotating AI-generated outputs against structured rubrics and detailed guidelines. Day-to-day work involves assessing responses for accuracy and consistency, identifying errors and edge cases, and writing reasoned feedback to support AI training pipelines. The role suits detail-oriented professionals with a background in AI evaluation, quality assurance, annotation, or similar precision-based review work.
Mid level · Remote · Contract
We read this from the posting text with AI. Skim the description below before ruling yourself out.
Why we read it this way (6)
This role is primarily AI output evaluation and annotation work — there is no single clean SOC code for this emerging occupation. 15-1299 (Computer Occupations, All Other) is the best available fit; 15-2031 (Operations Research Analysts) is a plausible runner-up given the structured review and analytical judgment emphasis.
The posting names no specific software tools or platforms — all skills are described in functional/process terms. Only the most concrete functional capabilities have been captured as skills; generic soft skills and methodologies have been omitted per extraction rules.
The bachelor's degree is explicitly framed as 'may be helpful' and equivalent practical experience is accepted, so the degree requirement is set to None.
The role is part-time and structured as an independent contractor engagement, fully remote with flexible scheduling.
The 'Nice to Have' section lists large-scale AI annotation programmes, QA/operations/trust-and-safety backgrounds, rubric-based evaluation, calibrated feedback, human data pipelines, and reviewing other annotators' work — all captured as preferred skills.
Posting is for a contract engagement — the market benchmarks below price full-time roles, so read the comp comparison with that in mind.
Read the full posting
The employer publishes the full description on their own site — read it there ↗. Or sign in to read it here — it's free, and it also lets you track this application.