About file import

Prerequisites

• Tealium AudienceStream CDP OR
• Tealium EventStream API Hub

To be successfully imported, you must format your CSV files correctly. For information on file and data formats, see Prepare a CSV file for import.

How it works

Use the Tealium server to connect to a file service (such as an Amazon S3 bucket), read the file, and import the data. Each imported row is processed as an event. After importing, the data can then be enriched, stitched to existing visitor profiles, and sent to other vendors.

Rate limits

Rows processed per second will vary. Rate limits will typically be a maximum of 100 rows per second at peak rate and 50 rows per second at a minimum rate.

Events

In the default Tealium data collection order of operations, events from a file import data source are processed before the Event received step and do not change the order of operations.

File import events are sent to EventStream and AudienceStream in the same way as events from other data sources with the following important exceptions:

• Browser-specific attributes: Browser-specific attributes, such as client IP address or user agent, are not populated.
• Enrichments: Enrichments on pre-loaded attributes in AudienceStream are not run, except the First Visit pre-loaded attribute.
• Functions: Data transformation functions are not run.
• Single-page visits: Incoming events are exempt from the single-page visit/visitors criteria. Single-page visits and visitors from other data sources are not persisted in AudienceStream. For more information about single-page visits/visitors, see How are single-page visits processed in AudienceStream? (requires Tealium login).
• Visit length: A visit started by a file import event lasts for 60 seconds.
• Visitor ID mapping: If you map an AudienceStream visitor ID attribute in your file import data source configuration, the visitor ID is set directly to the value of the column you choose.

Additionally, you can use the following event attributes with incoming data from a file import data source:

• _source_file (String) - The name of the file associated with the event.
• _file_line_num (Integer) - The line of the file associated with the event.

Set up a file import data source

There are two steps in setting up a file import as a data source:

1. Column mapping and file transfer service configuration
   To set up a file import data source, map columns from your CSV file to event attributes in Tealium and select a file transfer service.

   To assist with column mapping, upload a sample CSV file with your column labels and up to 1,000 rows of data.

2. Upload files to your file transfer service
   After you set up your data source, upload your CSV files to your file service. File uploading is done outside of Tealium.

Step 1: CSV column mapping and file transfer service configuration

Column mapping

The column mapping configuration determines the event attributes that correspond to each column in the CSV file. The column names are often different from the attribute names in the Customer Data Hub, so this mapping ensures that the data is imported properly. For example, a CSV file might have a column named postalCode, but the matching event attribute is named customer_zip, so you need to map these columns to associate them.

Mappings can be configured based on an existing event specification or as a custom event.

Event specification mapping

Each row of the CSV file is processed as an event. When you select an event specification mapping, event attributes are pre-selected in the Column Mapping table. Each row is processed as an event of the selected specification, for example tealium_event = "purchase".

Custom mapping

If you select a custom event mapping, specify the event attribute that corresponds to each CSV column. Each row is processed as an event with the following event identifier:

tealium_event = "imported"

Visitor ID mapping

To ensure your imported data is stitched with other sources, such as web, mobile or HTTP API, ensure that every row in a file import source file has a column with a unique visitor ID. You can then map the visitor ID column and corresponding event attribute to a visitor ID attribute (a unique attribute type for visitor identification in AudienceStream). The value in the mapped event attribute is assigned to the tealium_visitor_id attribute and matched directly to any existing visitor profiles.

For more information about Visitor ID Mapping in AudienceStream, see Visitor Identification using File Import Data Source.

File transfer service

The file transfer service is a secure location where you upload the files to be imported. Tealium supports the following file transfer services:

• Amazon S3 (Tealium bucket or your own bucket) - Tealium uses VPC endpoints to access Tealium S3 buckets directly through the AWS network.

  Storing large numbers of files in your S3 bucket may slow read times. We recommend regularly removing processed files from your S3 bucket to maintain efficient read times.

• Microsoft Azure File/Blob Storage

• FTP/SFTP - Supports the following authentication methods:

  • Password
  • Upload Private Key File
  • Generate Key Pair

Tealium S3 file retention

By using the Tealium S3 bucket in your file import configuration, you agree to the following file retention policy:

• 1-3 years - One year after uploading files to the Tealium S3 bucket, the files will be moved to a repository for long-term storage. Files will be accessible for up to three years. Retrieving files from long-term storage will incur a minimal to moderate service charge depending on the size and number of items requested. To retrieve any file, contact Tealium Support.
• 3 years or more - Three years after uploading files to the Tealium S3 bucket, files will be deleted permanently. If your business case requires storage for longer than three years, a longer data retention time must be explicitly stated in the customer contract. To request a longer retention policy, contact Tealium Support.

Step 2: Upload your files to a file transfer service

After you complete your file import data source configuration, upload files to the file service you have selected.

Tealium uses the following order of operations to import files:

1. Check for new files
   The system checks the file transfer service for new files every 10 minutes.
2. Copy new files
   When a new file is detected, it is copied from the file transfer location and processed in the Customer Data Hub.
3. Match Filename Prefix to File Import data source
   The prefix of the filename is used to identify which file import data source to use when importing the data in the file.
4. Process files
   The header line is read to identify the attributes being imported. After reading the header line, the following processing is performed:

   1. Visitor lookup — The visitor ID is used to look up the visitor record in AudienceStream. If an existing visitor record is not found, a new one is created.

      Grouping rows with the same visitor ID decreases the import time.

   2. Attribute enrichment - The visitor record is enriched according to the attributes imported and the existing enrichments in your account.