AI Evaluation Lead
elly
$120,000–$140,000from the description
Jul 17, 2026
—
Jul 21, 2026
What this job asks for AI summary
This role centers on owning the evaluation framework for an AI-driven personal finance advice platform — determining whether the right things are being measured, interpreting scoring results, identifying failure modes, and translating findings into concrete recommendations for the AI/ML team. It suits someone with hands-on experience assessing AI or ML system quality in high-stakes contexts, enough familiarity with LLM behavior to reason about why outputs go wrong, and sufficient personal finance literacy to make independent first-pass quality judgments.
Senior level · Remote · Full-time
We read this from the posting text with AI. Skim the description below before ruling yourself out.
How this req sits in the market our data
Roughly 13,100 people nationally plausibly meet what this posting asks for (software quality assurance analysts and testers). range 5,500–19,600
Applicant volume Heavy — This req sits in a large pool with little in its requirements to thin it, and auto-apply tools fire at everything in the occupation. Applying early and leading with the rare skills below is what gets read.
What the occupation pays Median $106,594 (middle half $82,076–$136,109). This posting is about at that midpoint.
Estimated from BLS employment for this occupation and area, per-skill prevalence across our listing corpus, and published wage benchmarks — as of Jul 28, 2026. It is a model, not a headcount.
Why we read it this way (9)
The role sits at the intersection of AI/ML quality assurance and data analysis; 15-1253 (QA Analysts and Testers) is the closest SOC given the core function is evaluating and measuring AI output quality, but 15-2051 (Data Scientists) is a credible runner-up given the emphasis on pattern analysis, behavioral data, and production ML system reasoning.
No formal degree is required; the posting explicitly states credentials are not needed and equivalent experience is accepted.
No overall years-of-experience floor is stated; seniority is assessed as Senior based on the expectation of independent judgment, ownership of a framework, and cross-functional influence at an early-stage company.
The title carries no level word, so advertised seniority is Unspecified.
'Familiarity with evaluation and observability tooling' appears in the requirements narrative without naming a specific product; it is captured generically as 'AI observability tooling' and treated as preferred given the softer 'familiarity with' framing.
Personal finance literacy is called out as a substantive requirement but names no specific tool or technology, so it is omitted from the skills list per extraction rules.
Equity (early-stage options) is mentioned alongside the salary range but no figures are given for it.
Ignored 2 non-technology phrase(s) as skills (responsibilities/concepts, not named tools): automated scoring / eval frameworks, behavioral and interaction data analysis.
Caller marked this a fully-remote role — scored against the national candidate pool.
Read the full posting
The employer publishes the full description on their own site — read it there ↗. Or sign in to read it here — it's free, and it also lets you track this application.