FAQs

The topics that follow provide answers to commonly asked questions:

• Presto FAQs provides answers to commonly asked questions on Presto.

• General Questions
  • 1. What is the pricing model for Qubole?
  • 2. What are some examples of policies I should use to delegate access to Qubole for my Cloud accounts?
  • 3. How should I set up networking for my Cloud accounts?
  • 4. How does Qubole access data in my Cloud object store?
  • 5. How do I set access control to a specific object in Qubole?
  • 6. How do I install custom Python libraries through the node bootstrap?
  • 7. How do I renew the QDS account password after its expiry and what is the password policy?
  • 8. What are the mimimum Operating System and browser requirements for using QDS?
  • 9. Why is my Spark application not using the Package Management?
  • 10. How do I avoid other users of the QDS account from seeing my commands?
• Questions about Airflow
  • 1. Do I need to provide access to Qubole while registering Airflow datastore in QDS?
  • 2. What does this error - Data store was created successfully but it could not be activated mean?
  • 3. How do I put the AUTH_TOKEN into the Qubole Default connection?
  • 4. How are API token and custom default data store related on Airflow Clusters?
  • 5. Is there any button to run a DAG on Airflow?
  • 6. Can I create a configuration to externally trigger an Airflow DAG?
  • 7. Why must I reenter the database password/AUTH-token at a cluster restart?
  • 8. Questions on Airflow Service Issues
  • 9. Deleting a DAG on an Airflow Cluster
• Questions about Hive
  • 1. What version of Hive does Qubole provide?
  • 2. How can I create a Hive table to access data in object storage?
  • 3. What is the difference between an external table and a managed table?
  • 4. How different is a Qubole Hive Session from the Open Source Hive Session?
  • 5. How can I create a table in HDFS?
  • 6. What file formats does Qubole’s Hive support out of the box?
  • 7. What is the default InputFormat used by Qubole’s Hive?
  • 8. Does Qubole remember my tables even when my cluster goes away?
  • 9. I have my data in RDS. Can I use Hive to process the data?
  • 10. Can I use Excel/Tableau/BI tools on top of Qubole’s Hive tables?
  • 11. Can I plug in my own UDFs and SerDes?
  • 12. How do I handle the exception displayed with a Tez Query while querying a JDBC table?
  • 13. What are the unsupported features in Qubole Hive?
• Questions about QDS Clusters
  • 1. Why didn’t my cluster come up?
  • 2. How long does a Qubole Hadoop Cluster take to come up?
  • 3. In whose account are clusters launched?
  • 4. When are clusters brought up and shut down?
  • 5. When are clusters auto-scaled?
  • 6. Should I have one large auto-scaling cluster or multiple smaller clusters?
  • 7. Are files from a Hadoop archive extracted to a specific folder by default?
  • 8. Will HDFS be affected by cluster auto-scaling?
  • 9. How do I check if a node is a master node or a worker node?
  • 10. Does Qubole store any data?
  • 11. Can the data stored on a Cloud instance be encrypted?
  • 12. Can I use Python 2.7 for Hadoop tasks?
  • 13. Why are so many idle mappers seen on an Hadoop Cluster?
  • 14. Can I submit Hive Commands to a Spark Cluster and is it supported?
  • 15. Do I need to bake a Paravirtual Image for bringing up the clusters?
• Questions about AWS
  • 1. What AWS regions are supported by Qubole?
  • 2. Can I use AWS Spot instances with Qubole?
  • 3. How can I use AWS Reserved Instances with Qubole?
  • 4. What are the purchasing options for AWS cluster nodes?
  • 5. What is the disadvantage of creating a Spot-only cluster or a cluster with 100% Spot nodes?
  • 6. Can I enable SSE-KMS encryption on EBS Volumes?
• Questions about Security
  • 1. Where do my Hive queries/Hadoop jobs run?
  • 2. Does Qubole store any of my data on its machines?
• Questions about Package Management
  • 1. How does Qubole Package Management pick a library version? Is it dependent on the Conda version or does it pick the latest version?
  • 2. Does the Qubole Package Management install dependencies of a package?
  • 3. Does the Qubole Package Management upgrade the underlying dependent libraries if they are already installed?
  • 4. How do I install packages that are not available in a Package Management environment?