Role: AI Linguist
Location: Mountain View, CA (3 Days hybrid)
Duration: 12 Months
 
Job Description:
We are seeking AI Linguists to join our diverse and interdisciplinary team to focus on AI quality, evaluation, and annotation and to contribute with their expertise to the annotation/human judgment pipeline development that lay at the foundation of the Client’s AI model development and improvement.
 
Responsibilities:

• Design and scale AI quality, evaluation, and annotation workflows for models and agentic products
• Build human-in-the-loop and AI-assisted judgment systems using clear rubrics, decision rules, and quality controls
• Define measurable criteria for model behavior, including search relevance, task success, and agent performance
• Run annotation and evaluation workflows across internal and vendor teams, including sampling, QA, adjudication, and calibration
• Conduct error analysis and experiments to improve prompts, datasets, rubrics, and workflow efficiency
• Partner cross-functionally to align on quality goals, tooling, and continuous product improvement
• Document processes, standardize workflows, and report evaluation results to improve quality, efficiency, and scale

 
Basic Qualifications:

• B.S./B.A Degree in Computational Linguistics, Language Technologies, Linguistics, Speech Science or a related field.
• 2+ years relevant industry experience.
• 2+ years experience with large-scale, end-to-end human judgment flows and tools for the training/evaluation of AI models
• 2+ years experience to breakdown linguistic data requirements into concise, clear instructions and designs
• 1+ years experience programming in Python

 
Preferred Qualifications:

• MS or Ph.D. in Computational Linguistics, Language Technologies, Linguistics or a related field.
• Previous experience with agentic products is a strong plus
• Willingness to learn technical concepts and new tools and a keen interest in technology
• Experience with human judgment projects and flows
• Experience with complex and scalable data annotations and evaluations
• Experience with programming languages in data annotation space
• Excellent comprehension and written communication skills in English

Suggested Skills
- AI Evaluation & Annotation Design
- Linguistic Data Analysis
- Human in the Loop Workflow Development 
 
At SPECTRAFORCE, we are committed to maintaining a workplace that ensures fair compensation and wage transparency in adherence with all applicable state and local laws. This position’s starting pay is: $ 58.70/hr.