AI Evaluation Scientist at Steampunk
McLean, VA
$105,000–$145,000from the description
Jul 5, 2026
McLean, VA
Jul 21, 2026
What this job asks for AI summary
This role centers on building and running evaluation systems for machine learning and generative AI models — including LLMs and RAG pipelines — used in federal government contexts. Day-to-day work involves creating test datasets, automated evaluation pipelines, and behavioral audits, then translating findings into improvement recommendations aligned with responsible AI and governance standards. It suits someone with hands-on ML evaluation experience and familiarity with Python-based AI tooling.
Mid level · 2+ years · National · Full-time
“or” means any one of them counts — you don't need all of them.
Posted 3 times — it's one opening, so apply once.
We read this from the posting text with AI. Skim the description below before ruling yourself out.
How this req sits in the market our data
Rare in this occupation — lead with these, and say what you built with them.
Most people in this occupation already list these. Still required — just not what gets you shortlisted.
What the occupation pays Median $122,874 (middle half $87,544–$162,374). This posting is about at that midpoint.
Estimated from BLS employment for this occupation and area, per-skill prevalence across our listing corpus, and published wage benchmarks — as of Jul 28, 2026. It is a model, not a headcount.
Why we read it this way (10)
No work location is specified in the posting. Steampunk is a federal contractor headquartered in the Washington, DC area, and the role requires the ability to hold a US government public trust position, suggesting a DC-metro or on-site placement is likely — but no city or remote status is explicitly stated. Remote has been set to false as a conservative default.
The clearance requirement is 'ability to hold a position of public trust,' which is a suitability/background-investigation requirement, not a formal security clearance (Confidential/Secret/TS). Clearance has been set to None accordingly.
The degree requirement lists Bachelor's or Master's but also accepts 'a related field,' and the JD does not explicitly reject candidates without a degree or state 'required.' Degree has been set to None (no hard minimum) given the typical federal-contractor framing where equivalent experience is implicitly accepted.
PyTorch, Hugging Face, scikit-learn, and LangChain are listed together under a 'Proficiency in Python and relevant libraries such as…' clause in the Qualifications section. The 'such as' qualifier means the specific tools are interchangeable examples, but proficiency in Python and at least one such ML/NLP library is a hard gate; each named tool is captured as required per policy.
Ragas is listed under 'Proficiency in AI evaluation frameworks such as Ragas' — same reasoning as above; it is the only named framework and is treated as a hard gate with no named alternatives.
OWASP LLM Top 10 appears under 'Familiarity with…' framing in the Qualifications section, which is softer language than the other requirements; marked as preferred.
AWS/Azure/Google ML Certifications are listed under 'Relevant certifications (helpful but not required)' — marked as preferred. AWS is used as the primary name with Azure and GCP as alternatives.
NIST AI RMF certification is listed under the same 'helpful but not required' certifications block — marked as preferred.
Agile/iterative development experience is explicitly called 'a plus' in the posting and has been omitted as it names no specific tool or technology.
Ignored 1 non-technology phrase(s) as skills (responsibilities/concepts, not named tools): OWASP LLM Top 10.
Read the full posting
The employer publishes the full description on their own site — read it there ↗. Or sign in to read it here — it's free, and it also lets you track this application.