Redmond, WA

Salary
—
Posted
Aug 17, 2026
Location
Redmond, WA
Last confirmed open
Sep 25, 2026

What this job asks for AI summary

This is a principal-level technical leadership role focused on designing and owning evaluation systems for AI, LLM, RAG, and agentic products across the full product lifecycle — from pre-launch through post-ship. The role spans architecture, methodology, and cross-functional influence: building eval frameworks, integrating them into CI/CD and release pipelines, and surfacing quality signals through dashboards and scorecards. It suits a senior researcher or applied scientist with deep experience in LLM/RAG evaluation, production system design, and the ability to drive technical strategy across engineering, applied science, and product teams.

Principal level · 6+ years · Bachelor's required · Full-time

Must have (5)
LLMsRAGagentic AI systemsML pipelinesCI/CD
Nice to have (9)
evaluation dashboardsonline experimentationhuman evaluationred teamingsynthetic datamodel monitoringenterprise searchdata visualizationresponsible AI

We read this from the posting text with AI. Skim the description below before ruling yourself out.

How this req sits in the market our data

What gives you an edge
RAG8%

Rare in this occupation — lead with these, and say what you built with them.

What won't set you apart
CI/CD45%

Most people in this occupation already list these. Still required — just not what gets you shortlisted.

What the occupation pays Median $122,874 (middle half $87,544–$162,374).

Estimated from BLS employment for this occupation and area, per-skill prevalence across our listing corpus, and published wage benchmarks — as of Aug 20, 2026. It is a model, not a headcount.

Why we read it this way (6)

The posting lists two overlapping experience tiers: a minimum path (Bachelor's + 6 years, Master's + 4 years, or Doctorate + 3 years) and a preferred path (Master's + 9 years or Doctorate + 6 years). The overall years minimum reflects the minimum gate (6 years with a Bachelor's); the preferred path implies a much more senior candidate.

The role explicitly frames itself as operating at 'principal, architect, or senior technical leadership level' — this is the basis for the Principal seniority classification despite no single word 'Principal' in the title.

No work location is stated in the posting; the CBSA is left blank. The context (Microsoft Copilot/IC3 references) suggests a US-based role but no city is confirmed.

SOC classification is Medium confidence: the role is primarily about evaluation methodology, statistical quality analysis, and research (pointing to 15-2051 Data Scientists), but it also requires substantial system architecture and production engineering work (pointing to 15-1252 Software Developers). The research publication and conference-speaking requirements tip the balance toward 15-2051.

The 5+ years of publications, 2+ years of conference speaking, and 5+ years of research program experience are listed under what appears to be a 'preferred' or senior tier of qualifications — they are treated as preferred rather than hard gates, as the minimum qualification block does not include them.

Many skills in this posting are methodological concepts (eval dataset design, failure mode analysis, quality metrics) rather than named tools or platforms; those are omitted per extraction rules. Only concrete named technologies and systems are captured.

Read the full posting

The employer publishes the full description on their own site — read it there ↗. Or sign in to read it here — it's free, and it also lets you track this application.

Apply

Apply on employer site ↗