Senior Software Engineer, Model Evaluation
At a glance
Senior Software Engineering role at Clera. Remote · full-time · $180,000–$250,000 base.
remoteroles summary, based on the employer's posting.
What you'll do
- Build benchmarks and evaluation suites for sourcing, supplier qualification, compliance paperwork, purchasing, freight and order tracking
- Work with procurement and manufacturing specialists to define tasks, write reference answers and rubrics, and create graders
- Train and fine-tune models directly, then use evaluation findings to shape experiments
- Track agent failures, make targeted changes, and verify their effects with measured results
- Help put research and evaluation practices in place as an early team member
What you bring
- At least three years of experience, with the judgment and scope expected at senior or staff level
- You have designed benchmarks or evaluations and built tasks with reference solutions, rubrics and quality checks
- You have hands-on model training and fine-tuning experience, not only experience using models through APIs
- You can turn knowledge from non-ML specialists into tasks that can be measured
Who this fits
This role suits a senior or staff-level engineer who wants hands-on model training and evaluation work alongside procurement and manufacturing experts. It is fully remote within the United States.
From the employer
About the Role
Join an early-stage AI company building agents for manufacturing and supply chain operations. In this research-focused engineering role, you will work with procurement and manufacturing experts to create benchmarks, train models, and improve agents used in real operational workflows.
What You'll Do
Design benchmarks and evaluation suites for supplier sourcing, qualification, compliance documents, purchase orders, freight, and order tracking.
Work with domain experts to scope tasks, write reference answers and rubrics, and build graders.
Train and fine-tune models hands-on, using evaluation results to guide experiments.
Identify agent failures, develop improvements, and validate those improvements with measured results.
Help establish research and evaluation practices as an early member of the team.
What We're Looking For
At least 3 years of experience, with senior- or staff-level capability.
Experience designing and writing benchmarks or evaluations.
Experience authoring tasks, including defining scope, creating reference solutions or rubrics, and checking task quality.
Hands-on experience training and fine-tuning models, beyond using models through APIs.
Ability to work closely with non-ML domain experts and translate their knowledge into measurable tasks.
Supply chain, procurement, manufacturing, or early-stage startup experience is a plus.
Compensation & Benefits
Base salary is $180,000 to $250,000 annually, plus equity. Visa sponsorship policy was not provided.
Location
Fully remote in the United States.