Home Conversations and Dashboard | Monitoring ClaudIA
👀

Conversations and Dashboard | Monitoring ClaudIA

Conduct audits to optimize ClaudIA's content and monitor all your conversations and generated data
Fabrício Rissetto
Ariana Carvalho
By Fabrício Rissetto and 5 others
• 11 articles

📊 How to Improve Claudia's Retention?

Explanation of each tab, chart, and table, along with important notes on how to use and interpret the data. 🧭 General Structure of the Retention Section 📌 Important: - The start and end date filters do not apply; they only apply to the main metrics tab. Data is organized in fixed and complete weeks to ensure performance and standardization of analyses. - All percentages shown in the tables are relative to the total tickets for that week. For example, if an escalation reason appears with 2%, it means that 2% of all tickets in that week had that cause. 🧩 Tab 1 – Retention Metrics - Overall This is the managerial tab, used to monitor progress, goals, and offenders in a consolidated manner. 🔶 Chart: “Total Tickets × % Retained per Week” - Orange bars: total tickets received per week. - Orange line: percentage of tickets retained by Claudia (without escalation). - Helps understand volume trends and AI effectiveness. 🔶 Side indicators: - Retention goal (set in the current operation's target) - Difference from last week vs. goal - Last week's retention - Maximum retention in the last 8 weeks - Average retention in the last 8 weeks 🔶 Table: “Percentage of Tickets by N2 Reason” - Shows the main reasons for escalation to N2, week by week. - Highlights the main offender from the past 4 weeks, with an average value. - Helps identify persistent patterns or reasons, such as use of N2 content or transfer by Eddie. 🛠 Tactical / Operational Tabs These tabs aim to support fine-tuning, content review, and case analysis. The structure is similar across all: 📁 Tab: Used N2 Content 🔶 Table: “Retention Gain Potential by Content – Last 8 Weeks” - Displays the most recurring N2 articles in escalated tickets. - Weekly columns show the percentage contribution of each content item relative to total tickets handled by AI. - Use to prioritize adjustments and migration of content to N1 or interactive (Eddie). 🔶 Table: “Retention Gain Potential by Tags – Last 8 Weeks” - Shows the most recurring tags in escalated tickets. - Weekly columns indicate the contribution percentage of each tag relative to total tickets handled by AI. - Use to prioritize adjustments related to themes/tags. 🔶 Chart: “Top Contents and % Retention Potential – Last 14 Days” - Shows the accumulated retention percentage that could be achieved by optimizing the Top N contents. - Example: optimizing the top 10 contents could yield an approximate 7.7% gain in retention. - Clearly demonstrates the impact of prioritizing adjustments on the right articles. 🔶 Table: “Section Selection Errors – Used vs. Correct” - Shows when Claudia used incorrect content, based on support feedback. - Displays which section was used and which should have been (N1 or INTERACTIVE). - Helps identify content usage issues. 📁 Tab: Transferred by Eddie (by design) 🔶 Table: “Retention Potential (%) by Eddie Flow” - Indicates which Eddie flows appear most frequently in escalations. - Values per week, allowing trend analysis. - Used to prioritize content or flow structure adjustments. 🔶 Table: “Tickets Escalated by Flow” - Lists real tickets, transfer reasons, and flow links. - Serves for validation and context review. 📁 Tab: Customer Requested Human 🔶 Chart: “Agent Interactions Before Handover” - Shows how many messages the customer exchanged with Claudia before requesting human support. - If most cases fall in the 3 to 5 interactions range, it may indicate lack of initial engagement or ineffective content. 🔶 Table: “Tickets by Interaction Range” - Lists tickets corresponding to the interaction range depicted in the chart. 🔶 Chart: “Top 5 Topics Customers Ask for Human Support” - Shows the most common topics where customers request human support. - Column: absolute volume of tickets | Line: % of total requests. - Helps identify topics with perceived low performance. 🔶 Table: “Content Used Before Customer Requests Human” - Indicates which contents were shown by Claudia before the customer requested support. - Useful to understand if content contributed to frustration or confusion. 📁 Tab: Transferred Due to Eddie Call Failure 🔶 Table: “Eddie Call Error % by Flow” - Shows flows that failed in technical calling, leading to escalation. - Used to identify infrastructure issues or bugs. 🔶 Table: “Tickets with Call Error by Flow” - Lists tickets and flows with errors. - Ideal for validation with the technical team and specific fixes. 📁 Tab: Transferred Due to Eddie Repetition 🔶 Table: “% of Eddie Flow Repetitions by Period” - Indicates which flows reached the attempt limit without resolution. - Shows problems with content effectiveness or the need to adjust the number of attempts. 🧠 Best Practices for Analysis - Use the main tab to monitor overall progress, goals, and major offenders. - Use the tactical tabs to explore specific causes and prioritize content, flow, or response model adjustments. - Always consult ticket links when validating with real examples. If you have questions on how to interpret any data or want to suggest dashboard improvements, contact our team! We’re here to help 🧡

Last updated on Aug 24, 2025

Como auditar as mensagens da ClaudIA

Nessa FAQ e vídeo apresentamos em detalhe como funciona a lógica por trás da seleção de seções utilizadas pela ClaudIA e a funcionalidade de tagueamento (Tagger) ao encerrar um ticket. Também explicamos como auditar esse dado diretamente no Hub, para que você possa receber insights através do Dashboard da Claudia para melhorar o tagueamento. https://www.loom.com/share/f6601cbe38b6400da3830e8fd0d52230 Sessões utilizadas pela ClaudIA: o que são, por que importam e como auditar O que são “sessões utilizadas”? Toda vez que a ClaudIA responde a um cliente, ela consulta uma ou mais seções do seu conteúdo base (N1 ou N2) para construir a resposta. Essas seções são chamadas de sessões utilizadas. Ou seja, são os blocos de conhecimento que a IA usou para formular a resposta enviada ao cliente. 🎯 Por que isso é importante? Essa informação é fundamental para: 1. Entender como a ClaudIA está raciocinando em cada resposta. 2. Auditar se ela está de fato utilizando o conteúdo certo. 3. Treinar e melhorar o comportamento da IA, identificando quando sessões estão sendo usadas incorretamente. 4. Definir a tag da conversa: a sessão com maior peso influencia o algoritmo de tagueamento automático. 5. Detectar quando há transferência para N2, e se ela foi feita com base em conteúdo correto. O que foi lançado Agora, você pode: - Ver diretamente quais sessões foram utilizadas em cada resposta da ClaudIA. - Dar feedback positivo ou negativo sobre o uso dessas sessões. - Auditar a tag final da conversa com base nas sessões utilizadas. Como visualizar as sessões utilizadas 1. Acesse qualquer tíquete no Hub. 2. Encontre a resposta da ClaudIA. 3. Clique no ícone ℹ️ para abrir o detalhe da resposta. 4. Você verá: - O raciocínio da ClaudIA (termo consultado + intenção). - As sessões utilizadas para compor a resposta. - Um ícone identificando quais sessões foram usadas (ex: ✅ ou ícone destacado). - Se houver sessões N2, elas também aparecem com destaque. Exemplo prático Pergunta do cliente: “Vocês integram com o Zendesk e o Intercom?” Resposta da ClaudIA: “Sim! Temos integração nativa com o Intercom. No caso do Zendesk…” Sessões utilizadas: - N1 - Integrações Intercom - N2 - Zendesk Integração via API Resultado: - A ClaudIA usou conteúdo de duas sessões. - Como uma delas era N2, a conversa foi automaticamente escalada. - A tag da conversa ficou como: Interesse em contratação, pois era a sessão mais relevante. Como auditar sessões utilizadas 1. Ao revisar uma conversa, clique no ícone ℹ️ de qualquer resposta. 2. Veja quais sessões foram marcadas como utilizadas. 3. Avalie se realmente fazem sentido com base na resposta. 4. Clique em 👍 ou 👎 para dar seu feedback sobre o uso da sessão. Importante: Isso avalia apenas se a sessão marcada foi usada corretamente, não se a resposta foi boa ou ruim. 5. Caso a resposta esteja errada, siga com a auditoria normal no fim do tíquete. O que acontece com esse feedback? - Permite avaliar se a ClaudIA está utilizando as sessões mais adequadas para cada tipo de solicitação. - Apoia a revisão do tagueamento automático, indicando quando ajustes manuais podem ser necessários. - Ajuda nosso time a entender se sessões N2 estão sendo usadas corretamente, o que impacta a performance da IA e os critérios de escalada. Importante: a auditoria, por si só, não altera automaticamente o comportamento da ClaudIA. Qualquer melhoria depende de ajustes de conteúdo, configuração ou estratégia definidos a partir dessa análise. Dicas rápidas - ⚠️ Às vezes, ClaudIA consulta uma sessão para confirmar algo, mesmo que a resposta não tenha trechos literais do conteúdo. Isso ainda conta como uso válido. - Se você não reconhece o conteúdo de uma sessão na resposta, marque 👎 e descreva brevemente por quê. - O feedback de sessão não substitui a auditoria da conversa. Ele complementa. O que influencia a tag da conversa? Ao fim do ticket, ClaudIA atribui uma tag baseada na sessão utilizada com maior score (relevância). Você pode revisar essa tag no processo de auditoria: 1. Vá até o final da conversa. 2. Revise se a tag atribuída corresponde ao conteúdo e intenção. 3. Caso queira alterar, selecione a tag correta e envie a auditoria. Onde tudo isso impacta? - Dashboards de temas atendidos - Indicadores de retenção e escalada - Performance da ClaudIA por sessão ou assunto - Qualidade das sugestões automáticas de melhoria - Faturamento (em alguns contratos baseados em sessões)

Last updated on Apr 13, 2026

Filters and Search System in Conversation

Filters 1. What is the Session ID used filter? This filter allows you to search for conversations based on the session ID (i.e., what content or section Claudia used in the response). To use this filter, you need to enable it in the Hub settings: go to Settings → Other Settings → Show Session ID filter. 2. How does the Helpdesk ID filter work? This filter allows searching for conversations by the external identifier assigned to the ticket by your support system (such as Zendesk, Freshchat, etc.). In the case of Freshchat, there is a peculiarity: the ID that appears in the URL (the "internal ID") may be different from the ID used in Claudia's filter. To perform the correct search, it may be necessary to convert the internal ID to the ID used by the filter using the Freshchat API or request this conversion from support. 3. What are the Start Date / End Date filters for? These filters limit the time period of the analysis, showing only conversations that were closed within the specified range. They are essential for time-based searches, monthly comparisons, trend analysis, and specific audits. 4. What does the Tags filter mean? Allows filtering only those conversations that received certain tags during their flow or at closing. For example: login_error, billing, complaint. If no tag filter is specified (or "All" is selected), all conversations will be displayed regardless of whether they have tags or not. 5. What is the purpose of the Audit filter? This filter can be used to segment conversations that have already been audited from those that have not. Useful for quality teams, reviews, or controls, to focus only on conversations that have been analyzed or those that need to be analyzed. 6. What can the Identifier (Customer / Agent / AI) filter mean? This field probably defines who initiated the conversation: - Customer — conversation started by the end user - Playground — test conversations created in the Playground screen - Optimizer — batch-created conversations based on system history for evaluation. Usually done before launching new operations This filter helps differentiate automated flows from those initiated by humans. 7. What is the Resolution (N1 / N2) filter for? Although not explicitly documented, it is reasonable to assume this filter classifies the type of resolution of the conversation: - N1 — conversations resolved by AI (ClaudIA) - N2 — conversations that required human intervention (handover) With this, you can measure Claudia's autonomy versus cases that demanded escalation. 8. What does Reason for Resolution mean? This filter lists the recorded reasons when closing a conversation. It indicates which functionality was used to finalize in N1 or to transfer to N2 in each conversation. For a clearer parameter, you can access our FAQ category "Conversation Closure" or view the "Resolution Reason" in the internal note of each conversation. This helps you understand why conversations were closed, in addition to their status. 9. What is the purpose of the A/B Test filter? This filter allows selecting conversations belonging to one of the experiment groups (A or B), useful when you or your team are testing different versions of flow, prompts, or AI models. It is a helpful tool for performance comparison between variations. 10. What does Intent / Topics mean? This filter likely allows selecting conversations based on the LAST topic or intent identified by Claudia. The main use case is filtering cases where: Claudia identified the end and needed to close OR "Bot Sale (Forward to Human Check and Forward to Human Multi-prompt)" 11. What could Gen CX Score be? This field represents a score of experience or sentiment generated by the AI (similar to a customer rating or an internal metric). If available, it allows filtering conversations based on the AI's assessment (positive, neutral, negative). It helps prioritize audits at risk of being detrimental or cases where the AI did not behave as most appropriate. 12. What is the purpose of the Eddie Only filter? It is also a reasonable assumption that this filter allows viewing only conversations responded to by the "Eddie" model — an advanced or specific version of Claudia. This would be useful to compare Eddie's performance versus the standard AI responses.

Last updated on Oct 06, 2025

How to Make the Best Use of the Gen CX Score

What is the Gen CX Score The Gen CX Score is a "Sentiment" metric assessing customer satisfaction by analyzing the entire conversation context. We use Large Language Models (LLMs) for this evaluation, which can range from: - Good (Score 5) - Neutral (Scores 2, 3, or 4) - Bad (Score 1) We categorize the metric this way instead of the traditional CSAT classification to identify extremes. Want a ticket with excellent service, clearly indicating that the issue was resolved? Look for "Good" Want to see tickets with a high risk of negative experience? Look for "Bad" Fluctuations within Neutral mean it's unclear whether there was a negative experience involving service, process, or product. Why Use the Gen CX Score? The goal of the Gen CX Score is to serve as an auxiliary metric for auditing, quality assessment, and CSAT prevention. It doesn't replace any of the above but can generate valuable auxiliary insights. It helps us gain a clear understanding of quality even when the volume of CSAT responses or audits is low. Additionally, when we start analyzing detailed metrics, the number of evaluations and audits disperses, making conclusions difficult (e.g., Eddie performance or Content) Where and How to Use the Gen CX Score? Identify conversations with a high risk of negative experience: Access Conversations > Filter > Gen CX Score ![Image] (https://cloudchat.cloudhumans.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBMktaRmc9PSIsImV4cCI6bnVsbCwicHVyIjoiYmxvYl9pZCJ9fQ==--e6cdd11c91e1e8eb06fd5c852a2c9223afc5f2f6/image.png) Filtering for "Bad" will help identify cases with high friction potential for auditing, content adjustments, or configuration changes. Monitor operation with an auxiliary indicator: Access Dashboard > CSAT and GenCXScore ![Image] (https://cloudchat.cloudhumans.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBM0NaRmc9PSIsImV4cCI6bnVsbCwicHVyIjoiYmxvYl9pZCJ9fQ==--42252a30ffd73cafeb57fa3c0891dc0f574f2287/image.png) Keep track of operational performance with an additional indicator. Granular insights into content or Eddie: Access Dashboard > Content or Eddie ![Image] (https://cloudchat.cloudhumans.com/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBNEdaRmc9PSIsImV4cCI6bnVsbCwicHVyIjoiYmxvYl9pZCJ9fQ==--2406bab85ff86f6b4ee4823688f0d7b8ba24cbdf/image.png) Compare specific content or Eddie results to ensure you have a high number of evaluations per interaction.

Last updated on Feb 05, 2026