Machine Learning Engineer, Core Evaluations
At a glance
Mid-level Machine Learning Engineering role at Cantina. Remote, United States · full-time · no salary stated · we estimate $142,500–$229,847 across the United States.
Cantina has not published a salary for this role. The figure below is our estimate, not theirs.
remoteroles summary, based on the employer's posting.
What you'll do
- Build evaluation pipelines for speech generation and recognition models in development and production
- Turn product needs into measurable model metrics, including subjective assessments through user studies
- Create dashboards that automate evaluation reporting and let teams compare model results
- Train models to assess new metrics, then use findings to guide model and data choices
- Coordinate evaluation work with model, data and product teams, and help establish the evaluation function
What you bring
- You have strong judgment in choosing metrics that capture model performance
- You have designed user studies using Mechanical Turk or comparable platforms
- You bring experience training and fine-tuning models for evaluation, plus statistical skills to compare results
- You have strong engineering and programming ability, with experience training ASR or TTS models
Who this fits
This mid-level role suits an engineer focused on evaluating speech generation and recognition models. It is remote within the United States. You would join as the evaluation function’s founding member, with an expectation to lead the team as it grows.
From the employer
About Cantina:
Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning.
If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future!
About the Role:
We are seeking an experienced Machine Learning Engineer (MLE) to focus on audio model evaluation, specifically for speech generation and recognition models.
This role involves designing and developing comprehensive model evaluation pipelines for both development and production environments, as well as creating automated dashboards for reporting evaluation results.
As the founding member of our evaluation team, the ideal candidate is expected to leverage their experience to lead our evaluation efforts and play a key role in the future growth of the evaluation team.
What You’ll Do:
Designing model evaluation pipelines for models in development and production
Designing user studies for subjective model evaluations.
Converting requirements into measurable metrics.
Designing and developing automated evaluation dashboard to see model performances and compare results.
Training new models to capture new and different evaluation metrics.
Communicating with the model team to help design better models based on the evaluation results.
Communicating with the data team to help decide the type of data necessary to improve model performance.
Communication with the product-manager to make sure product requirements are correctly measured.
Help grow the evaluation team as the founding member.
Lead the evaluation team in the future.
What You’ll Bring:
Strong experience and intuition for designing metrics that capture model performance.
Strong experience with designing user studies on Mechanical Turk or similar platforms. .
Strong experience with model training and fine-tuning for model evaluation.
Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
Very strong engineering and programming skills.
Experience with training ASR, TTS models.
Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)
Compensation:
When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.
Benefits for U.S.-based roles:
Competitive salary and generous company equity
Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina
42 days of paid time off, including:
15 PTO days
10 sick days
15 company holidays
2 floating holidays
Generous parental leave & fertility support
401(k) retirement savings plan
Lifestyle spending account – $500/month to use however you’d like
Complimentary lunch and snacks for in-office employees
One Medical membership, and more!
Machine Learning Engineering work from LinkedIn, X, Reddit and Upwork, in one list.
Each morning you get everything that fits, strongest first.
Machine Learning EngineeringRemote
Find me workBy the team behind remoteroles.