Choosing between a Cross-account IAM Role and Dual IAM Roles

QDS supports cross-account IAM Role-based authentication and dual IAM Roles-based authentication, which are briefly described in the following section.

Cross Account IAM Role

Qubole allows you to configure a cross-account IAM role which is a recommended over using AWS Keys to interact with AWS resources.

Set up AWS integration with Qubole with cross account roles by following steps provided in Getting Started with Qubole on AWS, It provides a detailed step-by-step procedure on how to go about creating IAM roles on the AWS end as well as the Qubole end. Using these steps, you can provide better security than with AWS IAM keys. However, the pitfall with this approach is that Qubole would have access to the S3 buckets that may contain sensitive data. So, if you are concerned about security, you can create an additional IAM Role that would have access to the sensitive data as described in Dual IAM Role.

For more information, see Managing Roles and Getting Started with Qubole on AWS/Creating a Cross-account IAM Role for QDS.

Dual IAM Role

You can use Dual IAM Role-based authentication if you want one role to access sensitive data.

In this IAM Role-based authentication, you must have two IAM Roles, let us call the two roles as Role A and Role B. Getting Started with Qubole on AWS provides a detailed step-by-step procedure on how to go about creating IAM roles on the AWS end as well as the Qubole end.

Role A will act as the cross-account IAM role. However, Role B would have access to the other S3 buckets with the sensitive data. Qubole cannot assume the Role B to access S3 data and hence, it provides much more data security. Use the steps described in Creating Dual IAM Roles for QDS to update this Role on those clusters that must have access to the sensitive data. With this approach, you can restrict access to data to only those commands and queries that have to operate on this data and deny all action access to all Qubole users in the account.

For more information, see Creating Dual IAM Roles for QDS.

However, with this approach, there are limitations that are in the following list:

  • Only Hive on master works.
  • Explore and Scheduler dependencies do not work. The dependencies only work over the default location or S3 path to which the cross-account IAM role has access to.
  • Importing and exporting data do not work.
  • An Amazon S3 path in commands works only when the S3 path is accessible to the cross-account IAM role.
  • Data in results and logs are accessible as the default location stores them.