About Snowflake data source

The Snowflake data source is in Early Access and is only available to select customers. If you are interested in trying this feature, contact your Tealium Support representative.

Requirements

This feature requires the following:

• Tealium EventStream or Tealium AudienceStream
• Snowflake account
• Snowflake role with USAGE privileges and user type of LEGACY_SERVICE.
  For more information, see Authentication.

How it works

Use the Snowflake data source to import bulk data from a Snowflake table or view at near real-time speeds. Once imported, the data can be transformed, mapped to Tealium attributes, and used to create enhanced audiences for targeted activation.

To get started, create one Snowflake data source per Snowflake table or view and customize the query mode and SQL WHERE clause for your dataset. Then map your database columns to Tealium attributes. Each imported row from a Snowflake source is processed as an event in Tealium.

A maximum of 10 cloud data warehouse data sources per profile can be enabled at a time.

Authentication

Key-pair

To authorize a connection using an RSA key pair, Tealium can generate an unencrypted key for you. This generated public key must be assigned to the Snowflake user with access to your Snowflake table or view.

You can select from the following key algorithms based on your security requirements: RSA-2048, RSA-3072, or RSA-4096.

After the key is generated, download the public key file and assign the key to a Snowflake user. The key fingerprint is also provided if you need to verify that the user’s public key was assigned correctly.

For details, see Step 1: Establish a Snowflake connection.

Single-factor password (legacy)

This option is not available for new connections and is only supported for existing customers with this type of connection.

To authorize a connection using a basic username and password, create a service user specifically for this integration. A service user is declared in the Snowflake user object with TYPE = SERVICE or LEGACY_SERVICE. These types of users are not subject to Snowflake MFA policies.

For more information, see Snowflake: Phased approach to block single-factor authentication.

Here is an example SQL command to create a service user:

CREATE USER my_legacy_service_user
    COMMENT = 'Service user to connect to Tealium'
    PASSWORD = 'YOUR_LONG_PASSWORD_HERE'
    DISABLED = FALSE
    MUST_CHANGE_PASSWORD = FALSE
    TYPE = LEGACY_SERVICE;

GRANT ROLE TEALIUM_READER TO USER my_legacy_service_user;

SHOW USERS;

Rate limits

Imports from Snowflake are typically limited to 500 events per second per account, but may vary. Standard attribute size limits still apply. For more information, see About attributes > Size limits

Batch size

The Snowflake data source imports data in batches, with a maximum of 1,000 rows per batch. This behavior is important to consider when selecting a query mode.

Snowflake tables

Each Snowflake data source supports importing data from one Snowflake table or view. To import data from multiple Snowflake tables, create a view in Snowflake and select the view in the data source configuration. For more information about views in Snowflake, see Snowflake: Overview of Views.

Data types

The Snowflake data source supports all Snowflake data types. To ensure data is imported correctly, map the Snowflake data types according to the following guidelines:

For more information about Snowflake data types, see Snowflake: Summary of Data Types.

Events

In the default Tealium data collection order of operations, events from a Snowflake data source are processed before the Event received step and do not change the order of operations.

Snowflake data source events are sent to EventStream and AudienceStream in the same way as events from other data sources with the following important exceptions:

• Browser-specific attributes: Browser-specific attributes, such as user agent, are not populated.
• Enrichments: Enrichments on preloaded attributes in AudienceStream are not run, except for the First visit attribute.
• Functions: Data transformation functions are not run.
• Single-page visits: Incoming events are exempt from the single-page visit/visitors criteria. Single-page visits and visitors from other data sources are not persisted in AudienceStream. For more information, see How are single-page visits processed in AudienceStream? (requires Tealium login).
• Visit length: A visit started by a Snowflake data source event lasts for 60 seconds.
• Visitor ID mapping: If you map an AudienceStream visitor ID attribute in your Snowflake data source configuration, the visitor ID is set directly to the value of the column you choose and no additional enrichment is needed.

Query Modes

The Snowflake data source supports three query modes to control how data is imported from your Snowflake table or view. Each mode uses a timestamp column, an increment column, or both to determine which rows to import.

Requirements for columns

To use query modes effectively, you must have one or both of the following:

• Timestamp column
  A timestamp column set to the current time when a row is added or modified. This column must be one of the following Snowflake data types: TIMESTAMP_LTZ, TIMESTAMP_TZ, or TIMESTAMP_NTZ.
  For more information, see Snowflake: Date & time.
• Increment column
  A numeric column that increments in value for every row added. A recommended definition for an auto-increment column is:

  COL1 NUMBER AUTOINCREMENT START 1 INCREMENT 1
  

  For more information, see Snowflake: Numeric data types.

Available Query Modes

Select a mode that aligns with the requirements of your use case. For an example of how these modes work, see Query mode example.

Timestamp + Incrementing (Recommended)

This mode uses both a timestamp column and an incrementing column to import new or modified rows. Rows are imported if they have:

• A newer timestamp than the previous import, and/or
• A larger increment value than the last imported row.

This mode provides the highest reliability for ensuring that all rows are imported as intended.

Timestamp

This mode uses a timestamp column to import new or updated rows. Rows are imported if they have a newer timestamp than the previous import.

Use this mode if your table has a timestamp column set on every insert or update operation.

Be aware that if the number of rows with the same timestamp exceeds the batch size, then some rows will not be imported.

Incrementing

This mode uses an incrementing column to import rows. Rows are imported if they have a larger increment value than the last imported row. Rows with an increment value less than or equal to the maximum value from the previous import are skipped. This mode does not detect modifications to existing rows.

Use this mode if your table does not have a timestamp column.

If you maintain your own increment column, as opposed to using an auto-increment column, ensure that the values always increase.

Query mode example

The following example shows how batch processing of rows and query modes work together. In the following table, modification_time is the timestamp column and customer_id is the incrementing column.

The Snowflake data source fetches data 1,000 rows at a time and marks the maximum value of the timestamp and/or incrementing column(s) from the batch of data.

• Using Timestamp + Incrementing mode: The data source fetches rows 1-1000. The next time the data source fetches the data, it looks for rows where either
  modification_time is 01Apr 13:00 and the customer_id is greater than 1000
  or
  modification_time is greater than 01Apr 13:00
• Using Incrementing mode: The data source fetches rows 1-1000 and marks the maximum incrementing value of 1000. On the next import, rows 1-1000 are skipped even if they were modified since the first import. Only new rows that increment the incrementing column (customer_id in the example) are processed.
• Using Timestamp mode (modification_time in the example): The data source fetches rows 1-1000 and marks the maximum timestamp of 01Apr 13:00. On the next import, rows with a timestamp greater than 01Apr 13:00 are imported. In this case, row 1001 is skipped because it has the same timestamp value that was fetched in the previous batch of data.

SQL Query

In the Snowflake data source Query Configuration, select the columns you want to import into Tealium. To add additional conditions for processing, use the SQL WHERE clause. This option adds a WHERE statement to your query. WHERE statements support basic SQL syntax.

The WHERE clause does not support subqueries from multiple tables. To import data from multiple Snowflake tables, create a view in Snowflake and select the view in the data source configuration. For more information, see Snowflake: Overview of Views.

Column mapping

The column mapping configuration determines the event attributes that correspond to each column in the Snowflake table.

Column names are often different from the attribute names in the Customer Data Hub, so this mapping ensures that the data is imported properly. For example, a table might have a column named postalCode, but the matching event attribute is named customer_zip, so you need a mapping to associate them.

For information about mapping Snowflake data types to Tealium data types, see the Data Types section.

Visitor ID mapping

To ensure your imported data is stitched with other sources, such as web, mobile, or HTTP API, ensure that every row in the Snowflake table has a column with a unique visitor ID. You can then map the visitor ID column and corresponding event attribute to a visitor ID attribute (a unique attribute type for visitor identification in AudienceStream). The value in the mapped event attribute is assigned to the tealium_visitor_id attribute and matched directly to any existing visitor profiles.

For more information about Visitor ID Mapping in AudienceStream, see Visitor Identification using Tealium Data Sources.

IP Addresses to allow

If your Snowflake account has strict rules about which systems it accepts requests from, add the Tealium IP addresses to your Snowflake allow list.