Monitoring Amazon Q Business user conversations with Amazon CloudWatch Logs - Amazon Q Business
Amazon Q Business chat message and feedback log examples

Monitoring Amazon Q Business user conversations with Amazon CloudWatch Logs

You can use Amazon CloudWatch Logs to deliver user conversations and response feedback in Amazon Q Business for you to analyze. These logs can be delivered to multiple destinations, such as Amazon CloudWatch, Amazon S3, or Amazon Data Firehose (standard rates apply). We recommend that you set up conversation and feedback logging within five minutes of creating your Amazon Q Business application environment.

The following are examples of tasks you can complete with logs from conversations and response feedback in Amazon Q Business:

  • Identify common user queries and pain points by reviewing the chat message content.

  • Monitor the quality of responses by looking at metrics like isMessageWithNoAnswer.

  • Understand user sentiment and satisfaction by analyzing the feedback data, including comments and usefulness ratings.

  • Generate custom dashboards and reports to track key metrics and trends over time.

Amazon Q Business supports the EVENT_LOGS log type that tracks the specifics of conversations in an application. You can use EVENT_LOGS to monitor Amazon Q Business in all AWS regions where Amazon Q Business is offered. For more information about the AWS Regions and endpoints currently supported by Amazon Q Business, see Supported Regions.

Logs from conversations might include sensitive or personally identifiable data passed in the chats. You can filter out this information from your logs with the Amazon Q Business console. Or you can mask this data on your logs using CloudWatch Logs masking policies. For more information, see Help protect sensitive log data with masking.

Topics

Amazon Q Business chat message and feedback log examples

The following are examples of Amazon Q Business chat message and feedback logs in CloudWatch Logs.

Chat message

{
    "application_id": "<ApplicationId>",
    "event_timestamp": "2024-09-05T19:43:58Z",
    "log_type": "Message",
    "account_id": "<AccountId>",
    "conversation_id": "<conversationId>",
    "system_message_id": "<systemMessageId>",
    "user_message_id": "<userMessageId>",
    "customer_message": string,
    "system_message": string,
    "output_metrics_is_message_blocked": boolean,
    "output_metrics_is_message_with_no_answer": boolean
}

Feedback

{
    "application_id": "<ApplicationId>",
    "event_timestamp": "2024-09-05T13:13:27Z",
    "log_type": "Feedback",
    "account_id": "<AccountId>",
    "conversation_id": "<conversationId>",
    "system_message_id": "<systemMessageId>",
    "user_message_id": "<userMessageId>",
    "customer_message": string,
    "system_message": string,
    "usefulness_reason": "NOT_FACTUALLY_CORRECT" | "HARMFUL_OR_UNSAFE" | "INCORRECT_OR_MISSING_SOURCES" | "NOT_HELPFUL" | "FACTUALLY_CORRECT" | "COMPLETE" | "RELEVANT_SOURCES" | "HELPFUL" | "NOT_BASED_ON_DOCUMENTS" | "NOT_COMPLETE" | "NOT_CONCISE" | "OTHER",
    "usefulness": "NOT_USEFUL" | "USEFUL",
    "comment": string
}