All open roles QA AI CONVERSATION ANALYST Bengaluru (Hybrid) Full-time 0 to 3 years Applications opening soon ABOUT THE ROLE We are looking for an AI Conversation Analyst to become the quality guardian of our AI agents: someone who can deeply analyse conversations, identify why an interaction succeeded or failed, uncover recurring patterns, and work with product and engineering teams to continuously improve agent performance. If you are naturally curious about how AI thinks, enjoy analysing conversations, and love solving complex problems, this is the role for you. If you have ever wondered how we know whether an AI actually had a good conversation, this is where you answer that question every day, and help us make every AI conversation better than the last. WHAT YOU'LL DO * Analyse AI conversations: review AI-generated voice calls and in-app conversations and evaluate quality across accuracy, context understanding, relevance, naturalness, tone and empathy, conversation flow, goal completion, instruction adherence, objection handling, and customer satisfaction. * Investigate AI failures: identify and document hallucinations, gibberish responses, incorrect information, context loss, repetitive replies, poor reasoning, incorrect pronunciation, speech interruptions, misread intent, weak objection handling, conversation loops, and failures to complete business objectives. * Perform root cause analysis: pinpoint the underlying cause, whether prompt-related, knowledge base gaps, tool failures, context management problems, speech recognition errors, text-to-speech issues, workflow limitations, or product bugs. * Identify patterns at scale: review hundreds or thousands of conversations to surface recurring failure modes, customer-specific issues, quality trends, and emerging edge cases. * Build quality frameworks: contribute to conversation scorecards, evaluation rubrics, quality benchmarks, failure taxonomies, review guidelines, and human-in-the-loop evaluation processes. * Deliver actionable insights: prepare recurring quality reports covering failure categories, severity, trends, agent performance, and improvement recommendations. * Work with AI to improve AI: use LLMs, transcription tools, analytics platforms, and internal observability tools to make analysis faster and more scalable. WHAT WE'RE LOOKING FOR You're a great fit if you have: * Strong analytical and problem-solving skills, with excellent attention to detail. * Strong written and verbal communication, and the ability to work with both technical and business teams. * The ability to spot subtle conversation issues and explain why they matter. * Structured thinking and strong documentation skills. * Genuine curiosity about AI and Large Language Models. * The ability to work independently with high ownership, and to review large volumes of conversations. * Comfort thriving in a fast-paced startup environment. NICE TO HAVE * Experience with Voice QA or Customer Support QA. * Experience with AI products, chatbots, or conversational AI. * Familiarity with ChatGPT, Claude, Gemini, or similar LLMs, and concepts like prompting, hallucinations, context windows, and AI evaluation. * Experience with Excel, Google Sheets, SQL, or Python. * Experience designing QA scorecards or evaluation frameworks. * Experience working with enterprise customers. SUCCESS IN THIS ROLE Your impact will be measured by: * The quality and accuracy of your conversation reviews and root-cause identification. * The number of actionable insights you deliver, and the improvements implemented from them. * A measurable reduction in recurring AI failures and improvement in AI quality scores. * Healthy review turnaround time and conversation coverage. * Client satisfaction with your quality reports. WHY JOIN REVRAG AI * Work on cutting-edge Voice and Conversational AI used in production. * Learn directly from experienced AI engineers and product leaders. * Influence how enterprise AI agents are built, evaluated, and improved. * Gain exposure to LLMs, AI evaluation, prompt engineering, and observability. * Be part of a fast-growing startup solving real customer problems with AI. ABOUT REVRAG AI RevRag AI is a Bengaluru-based deep-tech startup building production-grade Voice AI and LLM-powered automation for customer-facing workflows. Our multilingual, in-app AI agents plug directly into BFSI journeys, improving digital onboarding, sales, and support while cutting operational cost by up to 70%. We work with leading BFSI organisations including Mahindra Finance, Motilal Oswal, PayTM Insurance, PhonePe, KreditBee, Razorpay, Udaan Capital, Kissht, PolicyBazaar, InsuranceDekho, Open Capital, and EaseBuzz. See other roles COMPANY * Home * About Us * Careers * Book a Demo * Blog SOLUTIONS * Lending * Credit Cards * Insurance * Wealth * Banking AI AGENTS * In-App Agents * Voice & WhatsApp Agents * Agentic Workflow * Actionable Analytics * Integrations USE CASES * Onboarding * Sales * Support * Collections * Feature Adoption RESOURCES * Podcasts * Case Studies * Knowledge Base * Compare Vendors * Newsletter CONTACT US Office 8 The Green Dover, DE 19901, USA India Office HSR Layout, Bengaluru, Karnataka 560102 ---------------------------------------- Empowering Financial Apps to Scale Smarter, Faster, Better! ---------------------------------------- ISO Certified RevRag AI utilizes real-time APIs to fetch the information you need. Your data remains securely in its original location while we provide seamless access across our platform. SOC 2 Compliance: RevRag AI is aligned with globally recognized SOC 2 standards, guaranteeing exceptional data security and management practices. Role-Based Access Control Our platform adheres to the highest standards of data protection and operational integrity, validated by rigorous ISO certifications. ---------------------------------------- Terms of usePrivacy PolicyResponsible AI© 2025 RevRag AI All Rights Reserved