AI QA Analyst

At Ryz Labs , we are looking for an AI QA Analyst to join one of our clients, an AI-first travel and hospitality company. In this role, you will act as the bridge between model development and user experience. Your primary goal is to ensure our generative models are accurate, safe, and contextually aware by implementing rigorous evaluation frameworks and data-driven insights.

Qualifications:

• Ability to spot subtle logical inconsistencies or factual inaccuracies in AI-generated content.
• Experience managing, cleaning, or labeling datasets for machine learning or NLP projects.
• Familiarity with the AI lifecycle, Prompt Engineering, and tools like Google Sheets/Excel (SQL or Python is a significant plus).

Key Responsibilities:

• Review and annotate complex conversation traces to rate response quality based on metrics such as helpfulness, honesty, and harmlessness (HHH).
• Build and maintain high-q...