Understanding Cluster Network Security Characteristics (AWS)

Each cluster is associated with a unique security group which acts as a virtual firewall that controls the traffic for the nodes within the cluster. The ports that allow inbound traffic are:

  • Ports that allow inbound traffic from Qubole’s security group sg-a8c407c0 in the AWS us-east-1 region are:

    • Port 22, which is the SSH port.

      Note

      Hadoop 2 requires only port 22 to allow inbound traffic from Qubole’s security group.

    • Port 9000, which is the NameNode port

    • Port 9001, which is the JobTracker port (Hadoop 1)

    • Port 50030, which is the JobTracker web port (Hadoop 1)

    • Port 50070, which is the NameNode web port

    • Port 50060, which is the TaskTracker web port (Hadoop 1)

    • Port 50075, which is the DataNode web port

    • Port 8081, which is the Presto server port

    • Port 8443, which is the HTTPs port for a Presto server

    • Port 8082, which is the Zeppelin server port

    • Port 18080, which is the Spark History Server port

  • Port 22 allows inbound traffic from:

    • Qubole’s security group sg-a8c407c0 in the AWS us-east-1 region and the EC2-classic platform.

    Note

    CIDR 0.0.0.0/0 (world) for all other cases. (These include clusters in an AWS VPC, including the default VPC in the us-east-1 region, and AWS regions other than us-east-1).

    Create a ticket with Qubole Support if you want to restrict SSH port (port 22) access to limited IP addresses. For more information, see Creating a Security Group in the VPC.

  • Within a cluster, participating nodes can communicate with each other on all ports.