Use the solution
Using the microservices
Use the solution microservices to interact with the Amazon Ads and Selling Partner API, and hydrate your data lake. When you’re finished, business stakeholders can use Amazon Athena to access the data returned by the workflow run.
-
Navigate to the AWS CloudFormation console
. -
Select your deployed AMC Insights on AWS stack.
-
Navigate to the Outputs tab.
-
Choose the link in the Value section, next to the *platformmanagerSageMakerNotebookInstance61A5A1A0 * key.
Building QuickSight dashboards
This section details how to build an Amazon QuickSight dashboard with AMC data from your data lake. For more information on using QuickSight features, refer to What is Amazon QuickSight?
Create a QuickSight account
To build your first dashboard, you must create a QuickSight account. If you do not have a QuickSight account already, create one by following the steps in Setting up for Amazon QuickSight.
Note
Lake Formation integration with Amazon QuickSight is supported only for Amazon QuickSight Enterprise edition. Ensure that you have an Enterprise account before continuing with the remaining steps.
Authorize QuickSight to access AWS services
For QuickSight to access Athena, Amazon S3, and Lake Formation, a QuickSight administrator must configure the AWS resource permissions. These permissions apply to all QuickSight users. If you’re a QuickSight administrator (in which case you will see the Manage QuickSight option in your profile menu at the upper right), you can authorize QuickSight access to AWS services using the following two procedures.
Authorize QuickSight to access Athena and Amazon S3
-
In the QuickSight console
, select your profile name and choose Manage QuickSight. -
Navigate to Security & Permissions.
-
Under QuickSight access to AWS services, choose Manage.
-
Find Athena in the list. Select the box by Athena, then choose Next.
-
Under S3 Bucket, choose the solution stage bucket to grant QuickSight read access. The name of the bucket can be found in the Outputs section of your CloudFormation stack.
-
Choose Finish, and save your settings.
Authorize QuickSight to access Lake Formation database and tables
-
Find the Amazon Resource Names (ARNs) of the QuickSight users and groups that need access to Lake Formation data by following the steps in Authorizing connections through AWS Lake Formation.
-
Grant each user or group access by following the steps in Granting database permissions using the Lake Formation console and the names resource method.
Create a dataset in QuickSight
After you’ve authorized QuickSight to access AWS services, as described in Authorize QuickSight to access AWS services, you can create custom datasets in QuickSight using Athena by following the steps in Creating a datset using Amazon Athena data.
You can then create, publish, and share your custom dashboard.
Adding new datasets
This section details how to extend the solution to add a new dataset and its custom transformation to the deployed insights pipeline.
-
Fork the solution’s repository and clone the forked repository.
-
Specify configurations in dictionary format for a new dataset in file
source/infrastructure/datasets_parameters.json
, for example:{ "dev": [ { "dataset": "newdataset", "pipeline": "insights", "config": { "stage_a_transform": "new_dataset_light_transform", "stage_b_transform": "new_dataset_heavy_transform" } } ] }
-
Create custom transformation code for the dataset to be used in stage A and stage B Step Functions for processing the dataset. Place the transformation code under the following paths for stage A and stage B, respectively:
source/infrastructure/data_lake/lambda_layers/data_lake_library/python/datalake_library/transforms/stage_a_transforms/new_dataset_light_transform.py source/infrastructure/data_lake/lambda_layers/data_lake_library/python/datalake_library/transforms/stage_a_transforms/new_dataset_heavy_transform.py
Note
The
new_dataset_light_transform.py
andnew_dataset_heavy_transform.py
are actual transformation applied in stage A and stage B, and the filename is a reference for the Serverless Data Lake Framework(SDLF) pipeline to pick the transformation for the dataset. The names of the files have to match the configurations specified in stage_a_transform
andstage_b_transform
indatasets_parameters.json
of step 2. -
Create script for a glue job in Python and place the script in the following path:
source/infrastructure/data_lake/glue/lambdas/sdlf_heavy_transform/adtech/<DATASET_NAME>/main.py
Note
The
DATASET_NAME
is a reference to create Glue job for the dataset. TheDATASET_NAME
must match the configurations specified indataset
indatasets_parameters.json
of step 2. -
Follow the steps in the README.md
file to deploy or update AMC Insights on AWS.
Downloading the user scripts
-
Navigate to the CloudFormation console
. -
Select your deployed AMC Insights on AWS stack.
-
Navigate to the Outputs tab.
-
Copy the command in the Value section, next to the UserScriptOutput key.
Note
This command uses the AWS CLI to copy files locally from your deployed solution S3 Artifacts bucket. Ensure you have a compatible version of the AWS CLI installed to run this command.