Uploading your data into Amazon S3 - Amazon Lookout for Equipment

Uploading your data into Amazon S3

Amazon Lookout for Equipment needs to access your .csv files to create a dataset. You must store the files in the correct location and provide Lookout for Equipment with the information that it needs to access them.

To understand how to properly store your .csv files, you need to be familiar with the following Amazon Simple Storage Service (Amazon S3) concepts:

  • Bucket – A container to store your objects.

  • Object – The entity stored in the S3 bucket. In this case, it's your training or tuning text files.

  • Key – The unique identifier for an object within a bucket.

  • Prefix – Any portion of a key up to the final delimiter. You use prefixes to organize your data and specify objects in the S3 bucket.

To upload your data, you must first create an Amazon S3 bucket. To learn how to create an S3 bucket, see Create your first S3 bucket.

Amazon S3 uses prefixes to organize the storage of your files. A prefix is a logical grouping of objects in a bucket that is separated by a delimiter. The prefix value is similar to a directory name that you use to store similar data under the same directory in a bucket. For more information about prefixes, see Organizing objects using prefixes.

The Amazon S3 console supports the folder concept to help you see how prefixes organize your data. When you create an additional folder, you modify the prefix of your object key by adding another delimiter. To learn more about using folders, see Organizing objects in the Amazon S3 console using folders.

The Amazon S3 locations that you use to store your .csv files and the prefixes that Amazon Lookout for Equipment uses to access them depend on how your data is organized. For more information, see the following sections.

You use the following prefix to store the data for each sensor on your asset.

s3://DOC-EXAMPLE-BUCKET/AssetName/SensorName/SensorName.csv

If you had sensors on your asset with the following sensor names:

  • Sensor1

  • Sensor2

  • Sensor3

You would use the following objects keys for the sensors:

  • s3://DOC-EXAMPLE-BUCKET/AssetName/Sensor1/Sensor1.csv

  • s3://DOC-EXAMPLE-BUCKET/AssetName/Sensor2/Sensor2.csv

  • s3://DOC-EXAMPLE-BUCKET/AssetName/Sensor3/Sensor3.csv

The object key s3://DOC-EXAMPLE-BUCKET/AssetName/Sensor1/Sensor1.csv has the following prefixes:

  • s3://DOC-EXAMPLE-BUCKET/AssetName/

  • s3://DOC-EXAMPLE-BUCKET/AssetName/Sensor1/

By using the s3://DOC-EXAMPLE-BUCKET/AssetName/ prefix, you can access Sensor1.csv, Sensor2.csv, and Sensor3.csv.

The following procedure shows you how to use the Amazon S3 console to upload a .csv file containing data from one sensor to an Amazon S3 bucket.

To upload a .csv file of one sensor to an Amazon S3 bucket

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.

  2. In the Buckets list, choose the name of the bucket that you've created to store the data for your sensor.

  3. Choose Create folder.

  4. Specify the name of the asset as the folder name (for example, Pump1). Then choose Create folder.

  5. Choose the folder with the name of the asset.

  6. Choose Create folder.

  7. Specify the name of the sensor as the folder name (for example, Sensor1).

  8. Choose the folder with the sensor name.

  9. Choose Upload. Then choose Add files.

  10. Choose the .csv file containing your sensor data.

You've successfully uploaded your data. To learn how to use a schema to create a dataset from your .csv files, see Creating a dataset from multiple .csv files.

You use the following object key structure to store the data for each sensor on your asset:

s3://DOC-EXAMPLE-BUCKET/FacilityName/AssetName/AssetName.csv

For FacilityName, you specify the location of your asset. For example, Powerplant1 or Factory1 could be facility names.

The object key s3://DOC-EXAMPLE-BUCKET/FacilityName/AssetName/AssetName.csv has the following prefixes:

  • s3://DOC-EXAMPLE-BUCKET/FacilityName/

  • s3://DOC-EXAMPLE-BUCKET/FacilityName/AssetName/

When you ingest your model, you use the s3://DOC-EXAMPLE-BUCKET/FacilityName/ prefix to access the .csv file.

To upload a .csv file of one sensor to an Amazon S3 bucket

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/

  2. In the Buckets list, choose the name of the bucket that you've created to store the data for your sensor.

  3. Choose Create folder.

  4. Specify the name of the facility as the folder name (for example, Powerplant1). Then choose Create folder.

  5. Choose the folder with the name of the facility.

  6. Choose Create folder.

  7. Specify the name of the asset as the folder name (for example, Motor1).

  8. Choose the folder with the asset name.

  9. Choose Upload. Then choose Add files.

  10. Choose the .csv file containing the sensor data from your asset.

You've successfully uploaded your data. To learn how to use a schema to create a dataset from your .csv files, see Creating a dataset from one .csv file.

After you upload the sensor data, you can upload label data to highlight any abnormal equipment behavior. The following is an Amazon S3 prefix that you should use for your label data:

s3://DOC-EXAMPLE-BUCKET1/label-data

The following are example S3 locations where you would store the .csv files of your label data:

s3://DOC-EXAMPLE-BUCKET1/label-data/label-data01.csv

s3://DOC-EXAMPLE-BUCKET1/label-data/label-data02.csv

After you've uploaded the files, you can create a dataset from your .csv files. For more information, see Creating a dataset in Amazon Lookout for Equipment.

Next step

Creating a dataset in Amazon Lookout for Equipment