Snowflake Streaming Connector Setup Guide
This article describes how to set up the Snowflake Streaming connector.
API Information
This connector uses the following vendor SDK:
- Snowflake Ingest SDK Version: 2.1.1
Actions
Action name | AudienceStream | EventStream |
---|---|---|
Send Custom Event Data | ✗ | ✓ |
Send Entire Event Data | ✗ | ✓ |
Send Custom Visitor Data | ✓ | ✗ |
Send Entire Visitor Data | ✓ | ✗ |
How it works
The Snowflake Streaming connector leverages the powerful Snowflake Snowpipe Streaming feature to enable near real-time importing of event and visitor data directly into Snowflake staging tables, enabling immediate availability for processing and analytics with data latency under 10 seconds. Send either the entire dataset or specific attributes to the staging table, ensuring customizable data integration tailored to business needs.
Using Snowpipe Streaming creates a streamlined pathway to enhance data accessibility and analytical capabilities within Snowflake while decreasing latency and costs. By removing third party integrations and using the Snowpipe Streaming serverless compute model, the Snowflake Streaming connector provides a cost effective, secure connection.
The Snowflake Streaming connector enables you to unlock a number of use cases, including:
- Identity: Populate and activate identity graphs using cleansed, normalized, real-time identity data.
- Insights: Improve and activate differentiated intelligence with rich, high quality, integrated data.
- AI: Inform and activate AI initiatives with a real-time data layer.
- Consent: Ensure insights and activation always respect customer privacy and preferences.
Staging tables
The staging table in Snowflake that is intended to received data from Tealium must contain specific columns depending on the selected connector action.
When using either Send Entire Event Data or Send Entire Visitor Data actions, the staging table must have a VARIANT
column to receive the dataset and a column to receive the timestamp. The timestamp column must support the the timestamp data format that is sent.
When using either Send Custom Event Data or Send Custom Visitor Data, the staging table must have a column for each data type. Each data attribute must be assigned to a column. Ensure that each attribute is an accepted Snowflake data type and that the column is formatted properly to receive this data type. To map a timestamp, a separate timestamp column must be added.
For more information, see Snowflake: Supported Java data types.
Data types
The following table summarizes the data types supported by the Snowflake Streaming connector:
Snowflake Data Type | Supported |
---|---|
Numeric data types | ✓ |
String and binary data types | ✓ |
Logical data types | ✓ |
Date and time data types | ✓ |
Arrays | ✓ |
Object | ✓ |
Variant | ✓ |
Vector | ✗ |
Geography | ✗ |
Geometry | ✗ |
Table and column configurations
The connector does not support the following table or column configurations:
- Columns with collations
TRANSIENT
orTEMPORARY
tables- Tables with
AUTOINCREMENT
orIDENTITY
column settings - A default column value that is not
NULL
IP addresses to allow
Snowflake has strict rules about which systems it accepts requests from. You will need to add the Tealium IP addresses to your Snowflake allow list.
You must add the us-west-1
and the server-side profile region addresses to your allowlist. If you do not add these addresses to your allowlist, you will see errors when you try to fetch data.
Best practices
We recommend the following Snowflake table configurations for the Snowflake Streaming connector:
- 1 event feed per table
- 1 audience per table
Concurrent writing of tables by more than one feed or audience may result in performance errors.
Configuration
Navigate to the Connector Marketplace and add a new connector. For general instructions on how to add a connector, see About Connectors.
After adding the connector, configure the following settings:
-
Username
The Snowflake account username. Must either haveOWNERSHIP
orINSERT
rights to the Snowflake table that is receiving the data. -
Role
Access control role to use for the session. -
URL
The Snowflake account URL in the following format:<account_identifier>.snowflakecomputing.com
-
Private Key
The customer-generated private key. Supports both encrypted and unencrypted private keys. For instructions on generating the Snowflake private key, see Snowflake: Key-pair authentication and key-pair rotation.If the private key is encrypted, you must provide the Private Key Passphrase.
-
Private Key Passphrase
The encrypted private key passphrase for use with an encrypted private key. Do not assign a value if the private key is unencrypted.
To complete the connector authentication, use the following steps to alter the above user with public key details in Snowflake:
- Generate a public key in Snowflake. For information, see Generate a Public Key.
- Assign the public key to the above user by using an
ALTER USER
command in Snowflake. Only owners of users or users with SECURITYADMIN roles or higher can alter a user. For more information, see Assign the public key to a Snowflake user.
To successfully assign the public key to the user, ensure the following:- Enter the Snowflake username in double quotes (
"
). For example,"SNOWFLAKE.USER"
. - Copy and paste the public key without line breaks.
- Enter the Snowflake username in double quotes (
- Run the query to update the user with the new public key.
Snowflake supports public and private key rotations. For more information, see Configuring key-pair rotation.
Actions
The following section lists the supported parameters for each action.
Send Custom Event Data
Parameters
Parameter | Description |
---|---|
Database Name | The Snowflake database that contains the required table. |
Schema Name | The name of the schema used in the table. |
Table Name | The name of the table you want to insert data into. |
Event Parameters | Map the event parameters to the columns in the Snowflake table. |
Send Entire Event Data
Parameters
Parameter | Description |
---|---|
Database Name | The Snowflake database that contains the required table. |
Schema Name | The name of the schema used in the table. |
Table Name | The name of the table you want to insert data into. |
Column to record the payload | Choose the VARIANT column to record the event data. |
Timestamp | Select the column to send the timestamp variable to. |
Timestamp Attribute | (Optional) The default sends the current timestamp for the action. Select an attribute to assign as the timestamp if you want to send a different format. For more information, see Snowflake: Supported Java data types. If an attribute is assigned and produces an empty value, we will send the current timestamp. |
Send Custom Visitor Data
Parameters
Parameter | Description |
---|---|
Database Name | The Snowflake database that contains the required table. |
Schema Name | The name of the schema used in the table. |
Table Name | The name of the table you want to insert data into. |
Visitor Parameters | Map the visitor parameters to the columns in the Snowflake table. |
Send Entire Visitor Data
Parameters
Parameter | Description |
---|---|
Database Name | The Snowflake database that contains the required table. |
Schema Name | The name of the schema used in the table. |
Table Name | The name of the table you want to insert data into. |
Column to record the visitor data | Choose the VARIANT column to record the visitor data. |
Include Current Visit Data | Selecting this option will include both visitor data and current visit data in the data sent to Snowflake. |
Timestamp | Select the column to send the timestamp variable to. |
Timestamp Attribute | (Optional) The default sends the current timestamp for the action. Select an attribute to assign as the timestamp if you want to send a different format. For more information, see Snowflake: Supported Java data types. If an attribute is assigned and produces an empty value, we will send the current timestamp. |
This page was last updated: October 2, 2024