Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com

 

Hitachi Vantara Lumada and Pentaho Documentation

Create Snowflake warehouse

Parent article

The Create Snowflake warehouse entry creates a new Snowflake virtual warehouse from within a PDI job. This entry builds the warehouse with the attributes you specify. For more information about working with Snowflake in PDI, see PDI and Snowflake.

You can create a virtual warehouse to address your specific business needs. Within the step, you can set size, scaling, automated suspension, and other properties for your warehouse. Later, if your business needs change, you can adjust the specifications of your warehouse using the Modify Snowflake warehouse job entry.

For more information about Snowflake warehouse settings, see the Snowflake documentation.

General

The following field is general to this job entry:

  • Entry Name: Specify the unique name of the Create Snowflake warehouse entry on the canvas. You can customize the name or leave it as the default.

Options

The Create Snowflake warehouse entry includes several options for creating a virtual warehouse and defining its attributes. Each option is described below.

Create Snowflake warehouse

Database connection and warehouse

In this section, specify the Snowflake database connection and identify the virtual warehouse you want to create. For more information about naming virtual warehouses, see the Snowflake documentation.

OptionDescription
Database connectionSelect an existing Snowflake database from the list.

If you do not have an existing connection, click New. If you need to modify an existing connection, click Edit. See Define Data Connections for instructions.

Note: If timeout errors occur, see Snowflake timeout errors to troubleshoot.

Warehouse nameSpecify a valid identifier for the virtual warehouse you are creating. The name must be unique for your Snowflake account.
Warehouse commentSpecify a description or other information to identify or distinguish this virtual warehouse or its purpose.

This information is useful in configurations that include multiple virtual warehouses.

Warehouse settings

This section of the Create Snowflake warehouse entry includes the warehouse options that can be specified from within a PDI job when you create the virtual warehouse.

The Auto suspend (in seconds) option verifies that when the warehouse is inactive, it stops consuming credits. The Auto resume warehouse option verifies that the warehouse starts up again as soon as it is needed.

After you create the virtual warehouse, you can use the Modify Snowflake warehouse entry to change these settings at any time, even while it is running, to accommodate the need for more or less computing resources.

For information about virtual warehouse sizes, scaling policies, and resource monitor permissions, see the Snowflake documentation.

OptionDescription
Warehouse sizeSelect a size for this new virtual warehouse:

  • X-Small (default)
  • Small
  • Medium
  • Large
  • X-Large
  • 2X-Large
  • 3X-Large
  • 4X-Large
CautionThe size determines both the number of servers in each cluster in the warehouse and the number of credits consumed while the warehouse is running.
Scaling policySelect the policy for automatically starting and shutting down clusters in a multi-cluster warehouse running in Auto-scale mode.

The following policies are available:

  • Standard
  • Economy
Auto suspend (in seconds)Specify the time, in seconds, for this virtual warehouse to auto-suspend when it becomes inactive. When a warehouse is suspended, it does not accrue any credit usage.

The default value is 600 seconds (10 minutes).

You must specify a number of 60 or greater because the minimum amount of time a warehouse can run before it suspends is 60 seconds (1 minute).

CautionIf you leave this option blank, the warehouse will never suspend. Leave this option blank only if your query workloads require a continually running virtual warehouse.
Resource monitorSelect the name of an existing resource monitor to assign to this existing virtual warehouse.

The default value is blank. No resource monitor will be assigned to this warehouse.

NoteA resource monitor is a permissions-based control to govern the monthly credits used by the warehouse and all other warehouses to which the monitor is assigned.
Auto resume warehouseSpecify whether to automatically resume a warehouse when a SQL statement, such as QUERY, is submitted.
  • If this check box is clear (default), the warehouse resumes only when using the Start Warehouse job entry.
  • If this check box is selected, the warehouse resumes when a new query is submitted.

Cluster settings

The maximum and minimum cluster settings work together. Depending on your processing needs, you can use the cluster settings to run a multi-cluster warehouse in either Maximized mode or Auto-scale mode.

  • If you set the minimum and maximum cluster count with the same value, then the warehouse runs in Maximized mode.
  • If you set the minimum cluster count less than the maximum cluster count, then the warehouse runs in Auto-scale mode.

See the Snowflake documentation for more details.

OptionDescription
Minimum cluster countIf you are creating a multi-cluster warehouse, select the minimum number of server clusters for this virtual warehouse.

Valid values are 1 to 10. The default value is 1. This value must be equal to or less than the Maximum cluster count.

Maximum cluster countSelect the maximum number of server clusters for this virtual warehouse. Valid values are 1 to 10. The default value is 1.
NoteHigher values require the Snowflake Enterprise Edition.

Activity settings

You can define the initial state of the warehouse once it is created. You can also determine what happens within the PDI job if the requested new warehouse already exists.

OptionDescription
If warehouse existsSelect a result for the PDI job if the identified warehouse already exists.
  • Fail (default)

    The PDI job stops. No warehouse is created.

  • Replace

    The PDI job replaces the existing warehouse and the job continues.

  • Continue

    The PDI job continues.

Initially suspend warehouse after creationSpecify whether the warehouse is created initially in the suspended state.
  • Leave this checkbox cleared (default) to start the warehouse after it is created.
  • Select this checkbox to suspend the warehouse after it is created.