Remote | Python Software Engineer — $55–$85/hour at 24-MAG
New York, NY
$55–$85/hrfrom the description
Jul 25, 2026
New York, NY
Jul 26, 2026
What this job asks for AI summary
A contract role focused on building and evaluating benchmarks for frontier AI coding agents. Day-to-day work involves designing multi-step Python engineering tasks, writing complete reference solutions with tests and validation, and assessing how AI agents handle complex coding and debugging challenges. Peer review of other contributors' tasks is also part of the work. Suits experienced software engineers comfortable with independent, technically demanding problem-solving.
Mid level · 1+ years · Remote · Contract
“or” means any one of them counts — you don't need all of them.
We read this from the posting text with AI. Skim the description below before ruling yourself out.
How this req sits in the market our data
Roughly 306,500 people nationally plausibly meet what this posting asks for (software developers). range 128,500–459,500
Applicant volume Heavy — This req sits in a large pool with little in its requirements to thin it, and auto-apply tools fire at everything in the occupation. Applying early and leading with the rare skills below is what gets read.
Most people in this occupation already list these. Still required — just not what gets you shortlisted.
What the occupation pays Median $138,970 (middle half $107,524–$175,762). This posting is about at that midpoint.
Estimated from BLS employment for this occupation and area, per-skill prevalence across our listing corpus, and published wage benchmarks — as of Jul 29, 2026. It is a model, not a headcount.
Why we read it this way (9)
This is a W-2 contract engagement (described as 'full-time W-2 contingent employment') through staffing firm 24-MAG LLC, not a direct hire — classified as Contract accordingly.
The role is fully remote within the United States but no specific metro is given; CBSA fields are left blank.
The posting requires only 1+ year of experience and frames the educational bar as 'highly relevant' (not required), with equivalent practical experience explicitly accepted — the degree requirement is set to None.
The title carries no seniority level; the 1-year minimum and scope of work (task design, reference solutions, peer review) map to a Mid-level band.
Debugging is listed as a core required skill throughout the required qualifications but names no specific tool — captured as a named capability rather than a generic concept given its repeated, explicit emphasis as a primary technical requirement.
AI coding assistants and prompt engineering appear only under the 'Nice to Have' section and are marked preferred.
CI/CD and containers appear only under 'Nice to Have' and are marked preferred.
The alt SOC 15-1299 (Computer Occupations, All Other) is noted because the role is a hybrid of software engineering and AI model evaluation/benchmark development, which does not map cleanly to a single standard occupation.
Posting is for a contract engagement — the market benchmarks below price full-time roles, so read the comp comparison with that in mind.
Read the full posting
The employer publishes the full description on their own site — read it there ↗. Or sign in to read it here — it's free, and it also lets you track this application.