Snowflake Streaming Connector Setup Guide

API information

This connector uses the following vendor SDKs:

• Current actions (HPA): com.snowflake:snowpipe-streaming version 1.8.1
• Classic actions: Snowflake Ingest SDK version 4.4.2

How it works

The Snowflake Streaming connector leverages the powerful Snowflake Snowpipe Streaming feature to enable near real-time importing of event and visitor data directly into Snowflake staging tables, enabling immediate availability for processing and analytics with data latency under 10 seconds. Send either the entire dataset or specific attributes to the staging table, ensuring customizable data integration tailored to business needs.

Using Snowpipe Streaming creates a streamlined pathway to enhance data accessibility and analytical capabilities within Snowflake while decreasing latency and costs. By removing third party integrations and using the Snowpipe Streaming serverless compute model, the Snowflake Streaming connector provides a cost effective, secure connection.

The Snowflake Streaming connector unlocks a number of use cases, including:

• Identity: Populate and activate identity graphs using cleansed, normalized, real-time identity data.
• Insights: Improve and activate differentiated intelligence with rich, high quality, integrated data.
• AI: Inform and activate AI initiatives with a real-time data layer.
• Consent: Ensure insights and activation always respect customer privacy and preferences.

Architectures

The connector supports two Snowpipe Streaming architectures:

• High-performance architecture (HPA): Pipe-based. HPA is the current architecture and the default for new action configurations.
• Classic: Table-based. Classic actions are preserved for existing configurations and are labeled (Legacy) in the connector. Snowflake is deprecating the Classic architecture. For more information, see Snowflake: Snowpipe Streaming Classic architecture overview.

A channel is a streaming connection that appends rows to a Snowflake target. HPA uses Snowflake Elastic Channels: Snowflake creates one implicit channel per pipe, and multiple writers can append to that channel in parallel. Elastic Channels do not guarantee ordered ingestion or deduplication. For more information, see Snowflake: Elastic Channels limitations.

Classic actions open a separate channel per connector action instead of using Elastic Channels. The Drop classic architecture channels admin task removes those channels.

HPA reports row-level errors asynchronously. The connector reports a successful send after Snowflake durably accepts the batch. If Snowflake later rejects a row (for example, an invalid timestamp or a type mismatch), Trace and the action still show a successful send and the row is not written. To review rejected rows, check the row-level error reporting in Snowflake. For more information, see Snowflake: Elastic Channels row-level error handling. Classic actions fail synchronously in these cases.

Staging tables

The staging table in Snowflake that receives data from Tealium must contain specific columns depending on the selected connector action. For HPA actions, the staging table is the pipe’s target table, and the connector reads the column list from that target table.

When using either Send Entire Event Data or Send Entire Visitor Data actions, the staging table must have a VARIANT column to receive the dataset and a column to receive the timestamp. The timestamp column must support the timestamp data format that is sent.

When using either Send Custom Event Data or Send Custom Visitor Data, the staging table must have a column for each data type. Each data attribute must be assigned to a column. Ensure that each attribute is an accepted Snowflake data type and that the column is formatted properly to receive this data type. To map a timestamp, a separate timestamp column must be added.

For more information, see Snowflake: Supported Java data types.

Data types

The following table summarizes the data types supported by the Snowflake Streaming connector:

Table and column configurations

The connector does not support the following table or column configurations:

• Columns with collations
• TRANSIENT or TEMPORARY tables
• Tables with AUTOINCREMENT or IDENTITY column settings
• A default column value that is not NULL

IP addresses to allow

Snowflake has strict rules about which systems it accepts requests from. Add the Tealium IP addresses to your Snowflake allow list.

You must add the us-west-1 and the server-side profile region addresses to your allowlist. If you do not add these addresses to your allowlist, you will see errors when you try to fetch data. Tealium uses the us-west-1 IP addresses during connector configuration.

Best practices

For Classic actions, we recommend the following Snowflake table configurations:

• 1 event feed per table
• 1 audience per table

For Classic actions, concurrent writing to the same table by more than one feed or audience may result in performance errors.

Current (HPA) actions use Snowflake Elastic Channels, which support concurrent writers to the same pipe. The per-table concurrency recommendation does not apply to HPA actions.

HPA delivery can result in duplicate rows. If Snowflake does not confirm a batch within 10 seconds, the connector retries the batch. If Snowflake later commits the original attempt, the rows land twice. We recommend that you deduplicate downstream on an event ID.

Snowflake native app

Tealium provides a free native app in the Snowflake marketplace: Tealium Snowflake Connector.

The app includes helper utilities for setting up and managing streamed data from Tealium in Snowflake. It automates initial landing table setup tasks, including creating schemas, tables, and views.

The native app is optional and does not replace the configuration steps in this guide. To perform the setup manually, follow the instructions below.

The Tealium Snowflake App currently transforms visitor payload data only. Event payload data is not supported.

Configuration

Go to the connector marketplace and add a new connector. For more information, see About connectors.

After adding the connector, configure the following settings:

• Authentication Type: Select the authentication type for the connection. Possible values are: Private key or Public key.
• User Name: The Snowflake account username. Must either have OWNERSHIP or INSERT rights to the Snowflake table that is receiving the data.
• Role: (Optional) Specify an account role if your user needs to switch roles after connecting. Leave blank if using the Tealium Snowflake Connector native app and the application role has been granted directly to your user.
• Account identifier: Account identifier is used to construct your account URL in the following format: ACCOUNT_IDENTIFIER.snowflakecomputing.com. If your account identifier contains underscores, use the dashed format instead:
  • URL with underscores: https://acme-marketing_test_account.snowflakecomputing.com
  • URL with dashes: https://acme-marketing-test-account.snowflakecomputing.com
• For private key authentication:

  • Private Key: The customer-generated private key. Supports both encrypted and unencrypted private keys. For instructions on generating the Snowflake private key, see Snowflake: Key-pair authentication and key-pair rotation.

    If the private key is encrypted, you must provide the Private Key Passphrase.

  • Private Key Passphrase: The encrypted private key passphrase for use with an encrypted private key. Do not assign a value if the private key is unencrypted.

  • To use the private key:

    1. Generate a public key in Snowflake. For more information, see Generate a Public Key.
    2. Assign the public key to the above user by using an ALTER USER command in Snowflake. Only owners of users or users with SECURITYADMIN roles or higher can alter a user. For more information, see Assign the public key to a Snowflake user.
       To successfully assign the public key to the user, ensure the following:
       • Enter the Snowflake username in double quotes ("). For example, "SNOWFLAKE.USER".
       • Copy and paste the public key without line breaks.
    3. Run the query to update the user with the new public key.
• For public key authentication:
  • Key Algorithm: Select an algorithm to generate the key pair, then click Generate. A key fingerprint appears below. Click Regenerate to generate a new key. RSA-2048, RSA-3072, or RSA-4096 are supported.
  • To use the public key:
    1. Click Download Public Key to download your public key below.
    2. Copy the key from the file.
    3. Execute the following query on your Snowflake instance:
       ALTER USER [your username] SET RSA_PUBLIC_KEY = [your public key from the file].
    4. Click Done to create the new connection.
• Enable Snowpipe Streaming for Iceberg Tables: Enable this option if your target table in Snowflake is a managed Iceberg table. This connection-level option applies to both current (HPA) and Classic actions.

Snowflake supports public and private key rotations. For more information, see Configuring key-pair rotation.

Actions

The connector exposes current (HPA) actions for new configurations and Classic actions for backward compatibility.

Current actions (HPA)

Each current action uses the Snowflake Snowpipe Streaming high-performance architecture (HPA) with Elastic Channels and streams data into the selected pipe.

Classic actions

The following actions use the Classic (table-based) architecture and are deprecated. Existing configurations continue to work. For new configurations, use the current (HPA) action.

Choose an action

For new configurations, use the current (HPA) action. Existing Classic configurations continue to work on the Classic action. For a comparison of the two architectures, see Architectures.

The following sections list the supported parameters for each action.

Send Custom Event Data

Parameters

Send Entire Event Data

Parameters

Send Custom Visitor Data

Parameters

Send Entire Visitor Data

Parameters

Send Entire Log Event

Parameters

Send Log Event

Parameters

Send Custom Event Data (Legacy)

This action is deprecated. For the current action, see Send Custom Event Data.

Parameters

Send Entire Event Data (Legacy)

This action is deprecated. For the current action, see Send Entire Event Data.

Parameters

Send Custom Visitor Data (Legacy)

This action is deprecated. For the current action, see Send Custom Visitor Data.

Parameters

Send Entire Visitor Data (Legacy)

This action is deprecated. For the current action, see Send Entire Visitor Data.

Parameters

Send Entire Log Event (Legacy)

This action is deprecated. For the current action, see Send Entire Log Event.

Parameters

Send Log Event (Legacy)

This action is deprecated. For the current action, see Send Log Event.

Parameters

Migrate a Classic action to HPA

To move an existing Classic action to the current (HPA) version:

1. Create the pipe for the HPA action, or plan to use the default <TABLE_NAME>-STREAMING pipe. For privileges, see Required Snowflake privileges. To create a named pipe, see Create a pipe.
2. Add the current (HPA) action to the connector with the same mappings as the Classic action.
3. In the same save, disable or remove the Classic action.
4. After new data begins arriving through the HPA action, use Drop classic architecture channels to remove the leftover channels that the Classic action created.

Do not run the current (HPA) action and the Classic action against the same target table at the same time. Every event is written twice for as long as both actions overlap. To test the HPA action first, point it at a different target table.

The connector only sends events that are processed after the action is saved. The migration does not replay data, and there is no backfill mechanism.

When you publish the save that removes the Classic action, the connector flushes any buffered or retrying batches to Snowflake before the action stops. Those rows are preserved.

For tables with VARIANT or ARRAY columns, the HPA action writes the same values as the Classic action, so the migration preserves existing data.

Admin tasks

Create Pipe

Use the Create Pipe admin task to create a Snowflake pipe for an HPA action. The task is available from the Pipe Name field in any current (HPA) action, or under Remote Configuration on the connector.

Required Snowflake privileges

The role used by the connector owns the pipe it creates, and Snowflake loads data using the owning role’s privileges.

To create a pipe, grant the following privileges to the role:

GRANT USAGE ON DATABASE <database_name> TO ROLE <role>;
GRANT USAGE ON SCHEMA <database_name>.<schema_name> TO ROLE <role>;
GRANT CREATE PIPE ON SCHEMA <database_name>.<schema_name> TO ROLE <role>;
GRANT INSERT ON TABLE <database_name>.<schema_name>.<target_table> TO ROLE <role>;

To stream data into an existing pipe (created by another role), grant the following privileges to the role:

GRANT USAGE ON DATABASE <database_name> TO ROLE <role>;
GRANT USAGE ON SCHEMA <database_name>.<schema_name> TO ROLE <role>;
GRANT OPERATE ON PIPE <database_name>.<schema_name>.<pipe_name> TO ROLE <role>;
GRANT INSERT ON TABLE <database_name>.<schema_name>.<target_table> TO ROLE <role>;

The Pipe Name field uses SHOW PIPES to populate the dropdown, which only lists pipes the role owns or holds MONITOR or OPERATE on. The column list for mapping uses DESCRIBE PIPE and DESCRIBE TABLE, so the role also needs MONITOR on the pipe and SELECT on the target table. If the connector’s role created the pipe, pipe ownership covers the pipe-side requirements.

For the default <TABLE_NAME>-STREAMING pipe that Snowflake auto-provisions, INSERT on the target table is sufficient. Table ownership is not required.

Create a pipe

1. Go to Connect > Event Connectors or to Connect > Audience Connectors.
2. Select a Snowflake connector.
3. Under Remote Configuration, click Create Pipe.
4. Enter the following information:
   • Database Name
     (Required) The Snowflake database that contains the target table.
   • Schema Name
     (Required) The name of the schema used in the target table.
   • Pipe Name
     (Required) The name of the pipe to create.
   • Target table
     (Required) The name of the Snowflake table that the pipe writes to.
5. Click Create.

The task runs the following statement in Snowflake, which maps columns by name, case-insensitively, to all columns of the target table:

CREATE PIPE "<database_name>"."<schema_name>"."<pipe_name>"
AS
COPY INTO "<database_name>"."<schema_name>"."<target_table>"
FROM TABLE(DATA_SOURCE(TYPE=>'STREAMING'))
MATCH_BY_COLUMN_NAME = CASE_INSENSITIVE

The statement is CREATE PIPE, not CREATE OR REPLACE PIPE. If a pipe with the specified name already exists in the schema, Snowflake returns an error and the admin task surfaces that error.

Drop classic architecture channels

This admin task applies only to Classic actions. Current (HPA) actions use Snowflake Elastic Channels, which do not create per-connector channels and do not require a drop task.

To drop channels that Tealium has created, provide a list of channels in CSV format. You can retrieve the list of channels by running SHOW CHANNELS in your Snowflake instance. To drop all channels created by Tealium, leave the list blank.

To configure channels to drop, use the following steps:

1. Go to Connect > Event Connectors or to Connect > Audience Connectors.
2. Select a Snowflake connector.
3. Under Remote Configuration, click Drop classic architecture channels.
4. Enter the following information:
   • Database Name
     (Required) The Snowflake database name for the table.
   • Schema Name
     (Required) The name of the schema used in the table.
   • Table Name
     (Required) The name of the Snowflake table.
   • Channel Name
     The names of the Snowflake channels to drop, in CSV format. Channel Name is required for Iceberg tables. If no channel name is provided, all channels are dropped.
5. Click Drop.

If your list of tables is large and the deletion takes longer than 30 seconds, a false error stating that channels were not deleted may occur. We recommend that you validate that channels were successfully deleted by running SHOW CHANNELS in your Snowflake instance.