Cloudera EDH on AWS
Cloudera EDH Quick Start

Step 3. Configure the Cluster and EDH Services

In this step, you will use SSH tunneling to connect to Cloudera Director, which is running on the cluster launcher instance you created in step 2, and configure EDH services.

  1. Find the SSH command to connect to the cluster launcher instance.

    To do this, on the EC2 dashboard, click the Connect tab under EC Instances, as shown in Figure 7. You will need your private key to launch the instance.

                        Connecting to the cluster launcher using SSH

    Figure 7: Connecting to the cluster launcher using SSH

  2. Set up an SSH tunnel to connect to Cloudera Director.

    When you launch the cluster launcher instance, it will automatically download Cloudera Director and build a configuration file based on the resources created by the AWS CloudFormation template, such as VPC, private subnet, and public subnet. You can then modify the configuration file to launch the most appropriate cluster for your scenario. The launcher instance is automatically assigned an IAM root role to grant access to all the AWS resources that may be needed by the default configuration created in step 1.

    In addition, the template creates a 2048-bit RSA key pair with the naming pattern cloudera-aws-quickstart-mm-dd-YYYY on the cluster launcher node. This key pair will be used during the launch of EDH nodes. See the AWS CLI documentation for more information.

    Because the launcher instance is started with an IAM role, there is no need to distribute AWS credentials to deploy the EDH cluster. Because role credentials are temporary and rotated automatically, you don't have to manage credentials. For example, you don't have to worry about rotating credentials. For more information about the benefits of the IAM role, see Using IAM Roles to Delegate Permissions to Applications that Run on Amazon EC2 in the AWS documentation.

    Figure 8 lists the files that are downloaded automatically during launch.

                        Deployment scripts and configuration files

    Figure 8: Deployment scripts and configuration files

    Use the following command to set up an SSH tunnel into Cloudera Director running on port 7189. This command allows you to access Cloudera Director via a browser running on your local system.

    ssh -i "mykeyfile.pem" -L 7189:localhost:7189

    If you want to use the Cloudera Director client and deploy manually, use the following SSH command and bootstrap the cluster via the command line interface:

    ssh -i "mykeyfile.pem"


    Note that the auto-generated key pair file is necessary to connect to the new nodes being launched by using SSH. However, the cluster laucher node needs the key pair that was used during the initial AWS CloudFormation template launch. The SSH command above refers to the key pair used during the AWS CloudFormation launch, and not to the auto-generated key pair file.

  3. Modify the configuration of the cluster.

    The Quick Start builds two baseline configuration files that are customizable during deployment (either manually or through the Cloudera Director server web UI):

    • aws.simple.conf for configuring simple clusters

    • aws.reference.conf for configuring complex clusters

    You can make additional changes to the deployment configuration (for example, you can choose instance types, node count, subnet type, EDH services, or installation versions) by further modifying the configuration file or by using the web UI. The configuration files include baseline values based on the various resources (such as VPC ID and subnet ID) created during the launch of the Cloudera stack. By default, all Cloudera nodes are launched in the private subnet for security reasons. For more information about configuration parameters, see the Cloudera Director User Guide.