Conversation Analytics

REDA AI's Conversation Analytics feature empowers admins to deeply understand how individual conversations are using credits and where optimisation opportunities exist. By analysing a conversation, you gain insights into the efficiency of your agents, identify costly interactions, spot configuration issues, and receive AI-powered recommendations for improvement. Analysis is on-demand (not automatic) — admins deliberately choose which conversations to analyse, paying a small, fixed one-time credit cost to unlock detailed insights.

How It Works

Each conversation record in REDA AI stores a complete transcript of all messages and tool calls. When you click the Analyse Conversation button, the system:

1. Processes the full conversation history and message-by-message token usage.
2. Calculates per-message credit consumption and identifies which tools were called in each turn.
3. Computes global efficiency metrics and identifies optimisation patterns.
4. Generates an efficiency rating (Optimal, Good, Needs Review, or High Cost) based on credits per exchange.
5. Produces actionable suggestions for both individual messages and overall agent configuration.

Analysis results are displayed in a detailed breakdown that admins can use to tune agents, adjust tool assignments, refine prompts, or reconsider channel strategies.

Accessing Conversation Analytics

1. Open the REDA AI app and navigate to Conversations (or open an agent and go to its Conversations related list).
2. Click on a conversation record to open it.
3. Click the Analyse Conversation button.
4. Wait a few seconds for the analysis to complete.
5. Review the efficiency rating, per-message breakdown, token metrics, and optimisation suggestions.

Efficiency Ratings

The efficiency rating classifies the conversation's cost-effectiveness based on credits per exchange (message sent by user, response from agent):

Per-Message Breakdown

Each message in the analysis report includes:

• Message Content Preview: The first 80 characters of the message, helping you quickly identify which turn you're reviewing.
• Credits Used: The credit cost of that message, including any tool-call surcharges.
• Tools Called: A list of tools the agent invoked in that turn (e.g. describe_object, query_related_records, get_picklist_values).
• Colour-Coded Tier: Visual indicators show credit intensity — green (low, 0–2 credits), amber (medium, 2–6 credits), red (high, 6+ credits).
• Per-Message Suggestions: Specific recommendations for that turn (e.g. "consider adding this schema to the prompt to avoid repeated describe_object calls").

Global Optimisation Suggestions

Beyond per-message insights, the analysis provides organisation-level recommendations:

• Avoidable Tool Calls: Identifies patterns where the agent repeatedly calls describe_object or get_picklist_values. Suggestion: Add object schema or picklist definitions to the agent's system prompt or context fields to eliminate these costly queries.
• Excessive Tool Loading: If the agent has more than 15 tools assigned, the analysis flags this. Suggestion: Reduce the tool set to only the most essential tools — fewer tools = fewer decisions the agent must make, lower credit consumption.
• Cache Efficiency: Indicates when prompt caching is reducing cost on repeated content. If similar conversations repeat without benefiting from cache, the suggestion will recommend stabilising the system prompt across messages.

Key Metrics

Common Patterns & Actions

Pattern 1: Avoidable Schema Queries

Finding: The conversation shows repeated describe_object and get_picklist_values calls for the same objects.

Action: Update the agent's system prompt to include object schema (e.g. "The Opportunity object has fields: Amount, Close_Date, Account_ID..."). Alternatively, add these objects to the agent's Context Fields, which embeds their schema in every request automatically.

Pattern 2: Oversized Tool Set

Finding: The efficiency report flags 20+ tools assigned to the agent, but the conversation only used 3 of them.

Action: Review the agent's Tool Assignments and remove rarely-used tools. Fewer tools = simpler decisions for the AI, lower token usage, faster responses.

Pattern 3: Long, Verbose Responses

Finding: The agent is generating lengthy responses, driving up the average credits per exchange.

Action: Update the agent's persona or prompt to encourage conciseness (e.g. "Respond in 2–3 sentences" or "Be direct and action-focused"). For automated agents, add "Respond in automated mode — brief and to the point."

Pattern 4: Cache Efficiency Opportunity

Finding: The optimisation suggestions flag low cache reuse despite repeated similar queries.

Action: Check if the agent's system prompt is stable across messages. Per-turn timestamps, randomised ordering, or churning context fields can invalidate the prompt cache. Stabilise the prompt content to maximise cache reuse on repeated conversations.

Using Analytics to Improve Agents

Follow this workflow to optimise your agents:

1. Sample Conversations: Pick 3–5 representative conversations and analyse each. Look for common patterns.
2. Prioritise Findings: Identify the biggest cost drivers — avoidable tool calls, oversized tool sets, or verbose responses.
3. Make Targeted Changes: Update the agent (refine prompt, adjust tools, add schema to context). Test on new conversations.
4. Re-analyse: Run analytics on subsequent conversations to measure the impact of your changes.
5. Iterate: Repeat until you achieve your target efficiency rating.

Analytics Use Cases

• Cost Budgeting: Analyse high-volume agent conversations to estimate monthly costs. If costs are too high, optimise before rolling out to more users.
• Agent Comparison: Analyse conversations from different agents (support, sales, internal) to identify which are most efficient. Share best practices with lower-performing agents' teams.
• Channel Efficiency: Compare conversation costs across channels (Internal vs. External vs. Community) to decide where to invest in optimisation.
• Troubleshooting: When a user reports that the agent is too slow or giving bad answers, analyse their conversation. Tool overload or excessive tool calls often point to configuration issues.
• Compliance & Auditing: Analyse conversations involving sensitive data to confirm the agent is handling queries appropriately and not calling unnecessary tools that expose data.

Best Practices

• Analyse Regularly: Make conversation analytics part of your weekly or monthly admin routine. Trends emerge over time.
• Sample Across Agents & Dates: Don't analyse only the oldest or newest conversations. Sample across all your active agents and date ranges to get a representative picture.
• Document Changes: When you optimise an agent based on analytics, note the change in the agent's description or a changelog. This helps admins and stakeholders track agent improvements.
• Share Insights: Present analytics findings to your team — show which agents are efficient and which need work. Use data to justify configuration changes.
• Set Efficiency Targets: Aim for "Good" or "Optimal" ratings as a baseline. "Needs Review" conversations are learning opportunities; "High Cost" conversations warrant immediate investigation.

Troubleshooting

Understand the Economy credit change

Economy ordinary usage now applies a 0.40 multiplier to the base credit rate. For unchanged token quantities and base rates, an example costing 100 base credits costs 40 Economy credits. At the previous 0.10 multiplier it cost 10: the new debit is four times the previous Economy debit, and 60% below base. This changes credit consumption, not the subscription price. File pre-extraction is charged separately at full rate. A provider fallback can use a different effective tier; inspect the actual transaction rather than applying 0.40 to every mixed operation. New credit transactions record Processing Tier; historical rows are not backfilled. Economy still requires consent and compatible licence/provider configuration; a missing consent is refused, not silently processed at another tier.