Viewing AWS Panorama logs

AWS Panorama reports application and system events to Amazon CloudWatch Logs. When you encounter issues, you can use the event logs to help debug your AWS Panorama application or troubleshoot the application's configuration.

To view logs in CloudWatch Logs

1. Open the Log groups page of the CloudWatch Logs console.

2. Find AWS Panorama application and appliance logs in the following groups:

   • Device logs – /aws/panorama/devices/device-id

   • Application logs – /aws/panorama/devices/device-id/applications/instance-id

When you reprovision an appliance after updating the system software, you can also view logs on the provisioning USB drive.

Viewing device logs

The AWS Panorama Appliance creates a log group for the device, and a group for each application instance that you deploy. The device logs contain information about application status, software upgrades, and system configuration.

Device logs – /aws/panorama/devices/device-id

• occ_log – Output from the controller process. This process coordinates application deployments and reports on the status of each application instance's nodes.

• ota_log – Output from the process that coordinates over-the-air (OTA) software upgrades.

• syslog – Output from the device's syslog process, which captures messages sent between processes.

• kern_log – Events from the device's Linux kernel.

• logging_setup_logs – Output from the process that configures the CloudWatch Logs agent.

• cloudwatch_agent_logs – Output from the CloudWatch Logs agent.

• shadow_log – Output from the AWS IoT device shadow.

Viewing application logs

An application instance's log group contains a log stream for each node, named after the node.

Application logs – /aws/panorama/devices/device-id/applications/instance-id

• Code – Output from your application code and the AWS Panorama Application SDK. Aggregates application logs from /opt/aws/panorama/logs.

• Model – Output from the process that coordinates inference requests with a model.

• Stream – Output from the process that decodes video from a camera stream.

• Display – Output from the process that renders video output for the HDMI port.

• mds – Logs from the appliance metadata server.

• console_output – Captures standard output and error streams from code containers.

If you don't see logs in CloudWatch Logs, confirm that you are in the correct AWS Region. If you are, there might be an issue with the appliance's connection to AWS or with permissions on the appliance's AWS Identity and Access Management (IAM) role.

Configuring application logs

Configure a Python logger to write log files to /opt/aws/panorama/logs. The appliance streams logs from this location to CloudWatch Logs. To avoid using too much disk space, use a maximum file size of 10 MiB and a backup count of 1. The following example shows a method that creates a logger.

Example application.py – Logger configuration

def get_logger(name=__name__,level=logging.INFO):
    logger = logging.getLogger(name)
    logger.setLevel(level)
    LOG_PATH = '/opt/aws/panorama/logs'
    handler = RotatingFileHandler("{}/app.log".format(LOG_PATH), maxBytes=10000000, backupCount=1)
    formatter = logging.Formatter(fmt='%(asctime)s %(levelname)-8s %(message)s',
                                    datefmt='%Y-%m-%d %H:%M:%S')
    handler.setFormatter(formatter)
    logger.addHandler(handler)
    return logger

Initialize the logger at the global scope and use it throughout your application code.

Example application.py – Initialize logger

def main():
    try:
        logger.info("INITIALIZING APPLICATION")
        app = Application()
        logger.info("PROCESSING STREAMS")
        while True:
            app.process_streams()
            # turn off debug logging after 150 loops
            if logger.getEffectiveLevel() == logging.DEBUG and app.frame_num == 150:
                logger.setLevel(logging.INFO)
    except:
        logger.exception('Exception during processing loop.')

logger = get_logger(level=logging.INFO)
main()

Viewing provisioning logs

During provisioning, the AWS Panorama Appliance copies logs to the USB drive that you use to transfer the configuration archive to the appliance. Use these logs to troubleshoot provisioning issues on appliances with the latest software version.

Important

Provisioning logs are available for appliances updated to software version 4.3.23 or newer.

Application logs

• /panorama/occ.log – AWS Panorama controller software logs.

• /panorama/ota_agent.log – AWS Panorama over-the-air update agent logs.

• /panorama/syslog.log – Linux system logs.

• /panorama/kern.log – Linux kernel logs.

Egressing logs from a device

If your device and application logs don't appear in CloudWatch Logs, you can use a USB drive to get an encrypted log image off of the device. The AWS Panorama service team can decrypt the logs on your behalf and assist in debugging.

Prerequisites

To follow the procedure you will need the following hardware:

• USB drive – A FAT32-formatted USB flash memory drive with at least 1 GB of storage, for transferring the log files off the AWS Panorama Appliance.

To egress logs from the device

1. Prepare a USB drive with a managed_logs folder inside of a panorama folder.

   /
    └── panorama
        └── managed_logs

2. Connect the USB drive to the device.

3. Power off the AWS Panorama Appliance.

4. Power on the AWS Panorama Appliance.

5. The device copies logs to the device. The status LED blinks blue while this is in progress.

6. Log files can then be found inside managed_logs directory with the format panorama_device_log_v1_dd_hh_mm.img

You can't decrypt the log image yourself. Work with customer support, a technical account manager for AWS Panorama, or a solutions architect to coordinate with the service team.