Monitor, update and delete Amazon EMR Studio resources
This section includes instructions to help you monitor, update, or delete an EMR Studio resource. For information about assigning users or updating user permissions, see Assign and manage EMR Studio users.
View Studio details
Monitor Amazon EMR Studio actions
View EMR Studio and API activity
EMR Studio is integrated with AWS CloudTrail, a service that provides a record of
actions taken by a user, by an IAM role, or by another AWS service in EMR Studio.
CloudTrail captures API calls for EMR Studio as events. You can view events using the CloudTrail
console at https://console.aws.amazon.com/cloudtrail/
EMR Studio events provide information such as which Studio or IAM user makes a request, and what kind of request it is.
Note
On-cluster actions such as running notebook jobs do not emit AWS CloudTrail.
You can also create a trail for continuous delivery of EMR Studio CloudTrail events to an Amazon S3 bucket. For more information, see the AWS CloudTrail User Guide.
Example CloudTrail Event: a user Calls the DescribeStudio API
The following is an example AWS CloudTrail event that is created when a user,
admin
, calls the DescribeStudio API. CloudTrail records the user name as
admin
.
Note
To protect Studio details, the EMR Studio API event for
DescribeStudio excludes a value for responseElements
.
{
"eventVersion":"1.08",
"userIdentity":{
"type":"IAMUser",
"principalId":"AIDXXXXXXXXXXXXXXXXXX",
"arn":"arn:aws:iam::653XXXXXXXXX:user/admin
",
"accountId":"653XXXXXXXXX",
"accessKeyId":"AKIAIOSFODNN7EXAMPLE",
"userName":"admin
"
},
"eventTime":"2021-01-07T19:13:58Z",
"eventSource":"elasticmapreduce.amazonaws.com",
"eventName":"DescribeStudio",
"awsRegion":"us-east-1",
"sourceIPAddress":"72.XX.XXX.XX",
"userAgent":"aws-cli/1.18.188 Python/3.8.5 Darwin/18.7.0 botocore/1.19.28",
"requestParameters":{
"studioId":"es-9O5XXXXXXXXXXXXXXXXXXXXXX"
},
"responseElements":null,
"requestID":"0fxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"eventID":"b0xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"readOnly":true,
"eventType":"AwsApiCall",
"managementEvent":true,
"eventCategory":"Management",
"recipientAccountId":"653XXXXXXXXX"
}
View Spark user and job activity
To view Spark job activity by Amazon EMR Studio users, you can configure user impersonation on a cluster. With user impersonation, each Spark job that is submitted from a Workspace is associated with the Studio user who ran the code.
When user impersonation is enabled, Amazon EMR creates an HDFS user directory on the
cluster's primary node for each user that runs code in the Workspace. For example,
if user studio-user-1@example.com
runs code, you can connect to the primary
node and see that hadoop fs -ls /user
has a directory for
studio-user-1@example.com
.
To set up Spark user impersonation, set the following properties in the following configuration classifications:
-
core-site
-
livy-conf
[ { "Classification": "core-site", "Properties": { "hadoop.proxyuser.livy.groups": "*", "hadoop.proxyuser.livy.hosts": "*" } }, { "Classification": "livy-conf", "Properties": { "livy.impersonation.enabled": "true" } } ]
To view history server pages, see Debug applications and jobs with EMR Studio. You can also connect to the primary node of the cluster using SSH to view application web interfaces. For more information, see View web interfaces hosted on Amazon EMR clusters.
Update an Amazon EMR Studio
After you create an EMR Studio, you can update the following attributes using the AWS CLI:
-
Name
-
Description
-
Default S3 location
-
Subnets
To update an EMR Studio using the AWS CLI
Use the update-studio
AWS CLI command to update an EMR Studio. For more
information, see the AWS CLI Command
Reference.
Note
You can associated a Studio with a maximum of 5 subnets. These subnets must
belong to the same VPC as the Studio. The list of subnet IDs that you submit to
the update-studio
command can include new subnet IDs, but must also include
all of the subnet IDs that you already associated with the Studio. You can't
remove subnets from a Studio.
aws emr update-studio \ --studio-id
<example-studio-id-to-update>
\ --name<example-new-studio-name>
\ --subnet-ids<old-subnet-id-1 old-subnet-id-2 old-subnet-id-3 new-subnet-id>
\
To verify the changes, use the describe-studio
AWS CLI command and specify
your Studio ID. For more information, see the AWS CLI Command
Reference.
aws emr describe-studio \ --studio-id
<id-of-updated-studio>
\
Delete an Amazon EMR Studio and Workspaces
When you delete a Studio, EMR Studio deletes all of the IAM Identity Center user and group assignments that are associated with the Studio.
Note
When you delete a Studio, Amazon EMR does not delete the Workspaces associated with that Studio. You must delete the Workspaces in your Studio separately.
Delete Workspaces
Delete an EMR Studio