Settings for Snowflake

This topic describes how to configure the Snowflake destination connector in DataSync.

What is Snowflake

Snowflake is a fully managed, cloud-based OLAP database that is optimized for batch processing and large datasets. Key optimization techniques include:

  • Separation of storage and compute: Storage and compute resources are scaled independently to optimize performance and costs.

  • Automatic partitioning: Snowflake automatically partitions data and stores it in columnar format. Each partition (typically 50 MB to 100 MB) includes statistics such as minimum and maximum values.

  • Parallel processing: Due to its distributed compute nodes, Snowflake can process queries in parallel across multiple nodes.

Snowflake doesn’t support triggers or indexes and only enforces NULL/NOT NULL constraints on standard tables.

Configure a Snowflake connection

To configure a destination connection for Snowflake:

  1. In DataSync, create a new Snowflake destination connection.

  2. Fill out the fields in the Connection Properties panel.

Connection parameters

Property Description
Account

Enter the full account identifier. In Snowflake, open the account selector. Select the desired account, then View Account details. The account identifier appears under Account Identifier. See Account identifiers for more information.
Database Enter the name of the database to connect to.
Insecure Mode Disable the certificate revocation list checks. Use only as a temporary workaround during service disruptions caused by OCSP issues.
Disable Retry Enable or disable retries on HTTP requests to Snowflake API endpoints.
Max HTTP Retries Set the maximum number of retry attempts for failed HTTP requests. Acceptable values range from 1 to 999 (if retry is enabled). The default is 7.
Retry Timeout Set the total timeout in seconds for applicable endpoints covered by the retry policy. The default is 300 seconds. Any other value must be greater.
Connection Timeout Set the standard timeout in seconds for connectors. The default is 300 seconds.
Authentication

DataSync currently supports only basic authentication and key-pair authentication.

Basic Authentication:

  • Username: Enter the Snowflake database user name.

  • Password: Enter the user's password.

    Note: The user type must be LEGACY_SERVICE. See User Management for more information. Snowflake is currently in the process of deprecating single-factor basic authentication. Use this method only if no other authentication method is available.

Key-Pair Authentication:

  • Key User: Enter the Snowflake database user name.

  • Private key: Select the private key file associated with the user’s public key.

  • Passphrase: Enter the password protecting the private key file. Leave blank if the key is not encrypted.

    Note: Snowflake requires a 2048-bit RSA key pair at minimum. See Key-pair authentication for information on generating and rotating keys.

Limitations

When adding a Snowflake source table to an extraction, source indexes (if any) won’t be included. You also can’t manually create an index.