Refreshing Data - Amazon QuickSight

Refreshing Data

When refreshing data, Amazon QuickSight handles datasets differently depending on the connection properties and the storage location of the data. If QuickSight connects to the data store by using a direct query, the data automatically refreshes when you open an associated dataset, analysis, or dashboard. To refresh SPICE datasets, QuickSight must independently authenticate using stored credentials to connect to the data. QuickSight can't refresh manually uploaded data—even from S3 buckets, even though it's stored in SPICE—because QuickSight doesn't store its connection and location metadata. If you want to automatically refresh data that's stored in an S3 bucket, create a dataset by using the S3 data source card.

For files that you manually uploaded to SPICE, you refresh these manually by importing the file again. If you want to reuse the name of the original dataset for the new file, first rename or delete the original dataset, then give the preferred name to the new dataset. Also, check that the field names are the same name and data type. Open your analysis, and replace the original dataset with the new dataset. For more information, see Replacing Datasets.

You can refresh your SPICE datasets at any time. Refreshing imports the data into SPICE again, so the data includes any changes since the last import.

You can refresh SPICE data by using any of the following approaches:

  • You can use the options on Datasets page.

  • You can refresh a dataset while editing a dataset.

  • You can schedule refreshes in the dataset settings.

  • You can use the CreateIngestion API operation to refresh the data.

When you create or edit a SPICE dataset, you can enable email notifications about data loading status. This option notifies the owners of the dataset if the data fails to load or refresh. To turn on notifications, select the Email owners when a refresh fails option that appears on the Finish data set creation screen. This option isn't available for datasets that you create by using Upload a File on the datasets page.

In the following sections, you can find an explanation of different approaches to refreshing data.

Refreshing a Dataset

Use the following procedure to refresh a SPICE dataset based on an Amazon S3 or database data source on the Datasets page.

To refresh SPICE data from the Datasets page

  1. On the Datasets page, choose the dataset, and then choose Refresh Now.

  2. Keep the refresh type as Full refresh.

  3. If you are refreshing an Amazon S3 dataset, choose one of the following options for S3 Manifest:

    • To use the same manifest file you last provided to Amazon QuickSight, choose Existing Manifest. If you have changed the manifest file at the file location or URL that you last provided, the data returned reflects those changes.

    • To specify a new manifest file by uploading it from your local network, choose Upload Manifest, and then choose Upload manifest file. For Open, choose a file, and then choose Open.

    • To specify a new manifest file by providing a URL, enter the URL of the manifest in Input manifest URL. You can find the manifest file URL in the Amazon S3 console by opening the context (right-click) menu for the manifest file, choosing Properties, and looking at the Link box.

  4. Choose Refresh.

  5. If you are refreshing an Amazon S3 dataset, choose OK, then OK again.

    If you are refreshing a database dataset, choose OK.

Refreshing a Dataset During Data Preparation

Use the following procedure to refresh a SPICE dataset based on an Amazon S3 or database data source during data preparation.

To refresh SPICE data during data preparation

  1. On the Datasets page, choose the dataset, and then choose Edit Data Set.

  2. On the dataset screen, choose Refresh now.

  3. Keep the refresh type set to Full refresh.

  4. (Optional) If you are refreshing an Amazon S3 dataset, choose one of the following options for S3 Manifest:

    • To use the same manifest file you last provided to Amazon QuickSight, choose Existing Manifest. If you have changed the manifest file at the file location or URL that you last provided, the data returned reflects those changes.

    • To specify a new manifest file by uploading it from your local network, choose Upload Manifest, and then choose Upload manifest file. For Open, choose a file, and then choose Open.

    • To specify a new manifest file by providing a URL, enter the URL of the manifest in Input manifest URL. You can find the manifest file URL in the Amazon S3 console by opening the context (right-click) menu for the manifest file, choosing Properties, and looking at the Link box.

  5. Choose Refresh.

  6. If you are refreshing an Amazon S3 dataset, choose OK, then OK again.

    If you are refreshing a database dataset, choose OK.

Refreshing a Dataset on a Schedule

Use the following procedure to schedule refreshing the data. If your dataset is based on a direct query and not stored in SPICE, you can refresh your data by opening the dataset. You can also refresh your data by refreshing the page in an analysis or dashboard.

To refresh SPICE data on a schedule

  1. On the Datasets page, choose the dataset, and then choose Schedule refresh.

  2. For Schedule Refresh, choose Create.

  3. On the Create a Schedule screen, choose settings for your schedule.

    1. For Time zone, choose the time zone that applies to the data refresh.

    2. For Repeats, choose one of the following:

      • For Standard or Enterprise editions, you can choose Daily, Weekly, or Monthly.

        • Daily: Repeats every day.

        • Weekly: Repeats on the same day of each week.

        • Monthly: Repeats on the same day number of each month. To refresh data on the 29th, 30th or 31st day of the month, choose Last day of month from the list.

      • For Enterprise edition only, you can choose Hourly. This setting refreshes your dataset every hour, beginning at the time you choose. So, if you select 1:05 as the starting time, the data refreshes at five minutes after the hour, every hour.

        If you decide to use an hourly refresh, you can't also use additional refresh schedules. To create an hourly schedule, remove any other existing schedules for that dataset. Also, remove any existing hourly schedule before you create a daily, weekly, or monthly schedule.

    3. For Starting, choose a date for the refresh to start.

    4. For At, specify the time that the refresh should start. Use HH:MM and 24-hour format, for example 13:30.

  4. Choose Create.

Using the Amazon QuickSight console, you can create five schedules for each dataset. When you have created five, the Create button is disabled.