Why This Changes Everything
Before AI evaluation, checking voice call quality meant one thing: someone had to listen to the calls. For a hotel receiving 50 calls a day, that’s hours of work — and even then, you’d only sample a fraction. Most calls were never reviewed. With RecepAI’s Guest Satisfaction Criteria, your receptionist evaluates every single call — automatically, the moment it ends. You get structured results for 100% of voice interactions, not a random 5% sample.Voice evaluation is uniquely valuable because phone calls leave no written record for guests. A chat guest can scroll back and re-read — a phone guest relies entirely on what they heard and remember. If something goes wrong on a call, you’d never know without evaluation. This system catches problems that would otherwise be completely invisible.
How It Works
You define satisfaction criteria
On your Voice Agent page, you create criteria that describe what a successful phone call looks like for your hotel.
Every call gets evaluated
After each voice call ends, the conversation transcript is analyzed against every criterion you’ve defined. This happens automatically — the evaluation engine reviews the full conversation and checks each criterion independently.
Results appear in History
Open the History page, Voice tab. Each call shows evaluation results — you can see at a glance which calls met your standards and which need attention.
Voice evaluation analyzes the transcript, not the audio. Your receptionist’s voice AI generates a transcript of every call, and the evaluation runs against this text. This means evaluation catches what was said — content accuracy, completeness, whether the right information was shared — but not how it sounded (tone of voice, speaking pace, pronunciation quality, emotional warmth). For voice quality tuning beyond what evaluation can measure, use the recommendations in your Voice Prompt guide and listen to a few calls periodically.
What this means for writing criteria: Write your criteria about the content of the conversation, not the delivery. “The receptionist provided the correct check-in time” works well. “The receptionist sounded friendly” won’t evaluate reliably because the evaluation engine reads text, not listens to audio. If you care about delivery quality (and you should), combine evaluation results with occasional manual call reviews.
Setting Up Guest Satisfaction Criteria
Go to Voice Agent and scroll to the “Guest Satisfaction Criteria” section.Step 1: Click “Add Criteria”
Click the “Add Criteria” button to open the criteria editor.Step 2: Write a Clear Name
The name should be short and descriptive — up to 50 characters. This is what you’ll see in evaluation results, so make it immediately recognizable.Step 3: Write the Success Description
Describe what success looks like for this criterion — up to 500 characters. Be specific about what the evaluation engine should look for in the call transcript. You can define up to 10 criteria per voice agent.Example Criteria (5 Tested Templates)
These criteria are designed specifically for voice interactions. They account for the unique challenges of phone calls — limited attention span, no visual aids, and the inability to re-read information.1. Guest Question Answered
1. Guest Question Answered
Name: Guest Question AnsweredSuccess Description: The caller’s primary question received a clear, specific answer. The receptionist did not leave the question unanswered or give a vague response like “I’m not sure” without offering an alternative way to get the information.Why this works for voice: On a phone call, a vague answer is worse than in chat — the guest can’t re-read or click a link. This criterion ensures every caller gets a concrete answer or a clear next step.
2. Contact Details Provided Clearly
2. Contact Details Provided Clearly
Name: Contact Details ProvidedSuccess Description: When the caller needed to take action (make a reservation, reach a department, send an email), the receptionist provided a specific phone number, email address, or clear instruction — spoken slowly enough to be noted down.Why this works for voice: Sharing contact information on a call is tricky — the guest needs to write it down in real time. This criterion catches cases where the receptionist rattled off a number too quickly or used “contact us” without specifics.
3. Conversation Kept Concise
3. Conversation Kept Concise
Name: Conversation Kept ConciseSuccess Description: The receptionist’s responses were brief and focused — no longer than 2-3 sentences per turn. Information was summarized rather than listed exhaustively. When multiple options existed, the receptionist offered the most relevant one first and asked if the caller wanted to hear more.Why this works for voice: Long responses on a phone call cause callers to lose focus or interrupt. This criterion ensures your receptionist uses the “offer-then-drill-down” pattern recommended in the Voice Prompt guide.
4. Caller Acknowledged Satisfaction
4. Caller Acknowledged Satisfaction
Name: Caller Acknowledged SatisfactionSuccess Description: The caller expressed satisfaction or acknowledgment during or at the end of the call — through phrases like “thank you,” “that’s helpful,” “great,” or similar positive feedback. The caller did not express frustration, confusion, or dissatisfaction.Why this works for voice: Unlike chat, phone calls have real-time verbal cues. A “thank you” or “perfect” is a strong signal that the call went well. Conversely, “I don’t understand” or an abrupt hang-up signals a problem.
5. Proper Greeting and Closure
5. Proper Greeting and Closure
Name: Proper Greeting and ClosureSuccess Description: The receptionist opened the call with a professional greeting that included the hotel name, and closed the call by asking if there was anything else the caller needed before saying goodbye warmly.Why this works for voice: First and last impressions matter most on phone calls. A professional greeting sets expectations, and a proper closure ensures the caller didn’t hang up with unresolved questions.
Understanding Evaluation Results
After each voice call, the evaluation engine checks the transcript against your criteria. You can see results on the History page under the Voice tab.Status Overview
| Status | What It Means | What to Do |
|---|---|---|
| Success | The criterion was clearly met during the call | Nothing — your receptionist handled this well |
| Failure | The criterion was clearly not met | Review the call transcript and consider improvements |
| Unknown | The call didn’t provide enough information to evaluate | Usually means a very short call, disconnection, or the criterion simply wasn’t relevant to this call |
What “Unknown” Really Means
The “Unknown” result deserves special attention. It occurs when:- The call was too short — A caller who immediately said “wrong number” and hung up can’t be meaningfully evaluated
- The criterion wasn’t relevant — A “Reservation Handled” criterion on a call about pool hours
- The transcript was unclear — Technical issues that resulted in a partial or garbled transcript
From an AI perspective, “Unknown” is actually the honest answer. A binary pass/fail system would force a judgment even when there’s not enough evidence — leading to misleading results. The three-outcome model (success/failure/unknown) gives you cleaner, more trustworthy data.Short call bias: Hotels typically receive a mix of very short calls (10-30 seconds — wrong number, quick single question, immediate hang-up) and substantive calls (2+ minutes). Short calls will almost always return “Unknown” for most criteria, and that’s correct. Don’t worry about a high “Unknown” rate if your hotel gets many brief calls — focus on the success/failure ratio of substantive conversations instead. If you see a high percentage of “Unknown” on a specific criterion even for longer calls, that criterion might be too narrow or doesn’t apply to most of your call types.
Voice vs. Chat Evaluation: Key Differences
If you’re also using Chat Quality Criteria, here’s how voice evaluation differs:| Aspect | Chat Evaluation | Voice Evaluation |
|---|---|---|
| Where to set up | Conversation Agent page | Voice Agent page |
| Section name | ”Chat Quality Criteria" | "Guest Satisfaction Criteria” |
| Result statuses | Success, Failed, Needs Training, Partial, Pending | Success, Failure, Unknown |
| Knowledge gap detection | Yes — identifies missing Training Materials topics | Not available — focus is on conversation quality |
| ”Mark as Trained” workflow | Yes — clears actionable training items | Not applicable |
| Description limit | 2,000 characters | 500 characters |
| Criteria limit | No fixed limit | 10 criteria maximum |
Why the differences? Chat evaluation was built specifically for RecepAI with knowledge gap analysis — it understands your Training Materials and can tell you exactly what’s missing. Voice evaluation focuses on the conversation experience itself — how well the call went from the guest’s perspective. Both are valuable; they just measure different things.A practical tip: If the same topic (e.g., spa information) fails in both chat and voice evaluations, it’s almost certainly a Training Materials gap — upload the missing document and both channels improve. If something only fails in voice, the issue is likely in how the information is delivered verbally, not what information is available. In that case, adjust your voice prompt rather than your Training Materials.
Improving Voice Quality Based on Results
When evaluation results reveal issues, here’s where to make changes:Callers aren't getting answers → Add Training Materials
Callers aren't getting answers → Add Training Materials
If your receptionist frequently can’t answer questions on calls, the information is likely missing from your knowledge base. Go to Training Materials and add documents covering the missing topics.Tip: Voice calls tend to be about urgent, practical topics — directions, hours, availability. Prioritize adding this operational information.
Responses are too long → Adjust your voice prompt
Responses are too long → Adjust your voice prompt
If the “Conversation Kept Concise” criterion keeps failing, your voice prompt may not emphasize brevity strongly enough. Add or strengthen the 2-3 sentence rule in your “How they should talk” field.
Callers sound confused → Simplify information delivery
Callers sound confused → Simplify information delivery
If callers frequently express confusion, your receptionist might be sharing too much information at once. Update your voice prompt to use the “offer-then-drill-down” pattern: give the most relevant answer first, then ask if the caller wants more details.
Many 'Unknown' results → Review criteria relevance
Many 'Unknown' results → Review criteria relevance
A high percentage of “Unknown” results usually means your criteria are too specific for the types of calls you receive. Consider broadening the criteria descriptions or removing criteria that only apply to rare call types.
Calls end abruptly → Check greeting and pacing
Calls end abruptly → Check greeting and pacing
If calls are ending before your receptionist can properly help, the issue might be the greeting (too long, too robotic) or overall pacing. Listen to a few call recordings and compare with your voice prompt settings.
Best Practices
Start with 3 criteria, expand based on data
Start with 3 criteria, expand based on data
Begin with the most universal criteria: Guest Question Answered, Contact Details Provided, and Proper Greeting/Closure. After reviewing a week of results, add criteria that target specific issues you noticed.
Write criteria for the transcript, not the audio
Write criteria for the transcript, not the audio
Remember: the evaluation engine reads the call transcript, not listens to the audio. Criteria about “tone of voice” or “speaking speed” won’t be evaluated accurately. Focus on what was said, not how it sounded.
Account for different call types
Account for different call types
Your hotel likely receives different types of calls — quick FAQ, detailed reservation inquiries, complaints, directions. Write criteria descriptions that make sense across these different scenarios, or accept that some criteria will return “Unknown” for certain call types.
Use evaluation alongside voice prompt testing
Use evaluation alongside voice prompt testing
Evaluation tells you what’s happening. To fix issues, combine evaluation insights with the Voice Prompt Testing Checklist. Test changes, then watch the next week’s evaluation results to verify improvement.
Compare chat and voice results
Compare chat and voice results
If the same topic (e.g., spa information) fails in both chat and voice evaluations, it’s definitely a Training Materials gap. If it only fails in voice, the issue might be how the information is delivered verbally — a prompt adjustment rather than a knowledge gap.